-
Notifications
You must be signed in to change notification settings - Fork 631
Pre 0.10.0-rc1 #2905
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Pre 0.10.0-rc1 #2905
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
33490f5
Updated version and changelog
chernser 73156f8
Added script to build examples
chernser 3668d52
Added migration guide and updated changelog
chernser 2f3848a
updated developers list
chernser ad47b52
Updated changelog
chernser 9be6713
added VERSION and updated workflow to take version from there
chernser File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| #!/bin/sh | ||
| LIB_VER=$(grep '<revision>' pom.xml | sed -e 's|[[:space:]]*<[/]*revision>[[:space:]]*||g') | ||
| find `pwd`/examples -type f -name pom.xml -exec sed -i -e "s|\(<clickhouse-java.version>\).*\(<\)|\1$LIB_VER\2|g" {} \; | ||
| for d in $(ls -d `pwd`/examples/*/); do \ | ||
| if [ -e $d/pom.xml ]; then cd $d && mvn --batch-mode --no-transfer-progress clean compile; fi; | ||
| if [ -e $d/gradlew ]; then cd $d && ./gradlew clean build; fi; | ||
| done | ||
|
chernser marked this conversation as resolved.
chernser marked this conversation as resolved.
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| # Release 0.10.0 | ||
|
|
||
| # Migration Guide | ||
|
|
||
| ## JDBC-V2: `async_insert=1` not Set by Default | ||
|
|
||
| The previous version of the JDBC driver set `async_insert=0`, effectively making any insert operation synchronous. | ||
| This was done to get the insert summary that is then returned by the family of `java.sql.Statement#executeUpdate(java.lang.String)` methods. | ||
| Now these methods will not return an accurate number of records (in most cases `0`) if the server is using `async_insert=1`. | ||
|
|
||
| Set `clickhouse_setting_wait_for_async_insert=1` at the connection level to make the server wait for the async insert to complete. In this | ||
| case the client will get a response and return control once the operation is done on the server side. However, this option does not produce an accurate summary. | ||
|
|
||
| Set `clickhouse_setting_async_insert=0` at the connection level (or pass it via the JDBC connection string) to fall back to synchronous inserts. | ||
|
|
||
| See more about async inserts: https://clickhouse.com/docs/optimize/asynchronous-inserts | ||
|
|
||
| ## CLIENT-V2 & JDBC-V2: User/Password Credentials not Allowed with Custom `Authorization` HTTP Header | ||
|
|
||
| The client builder throws an exception if both user credentials and a custom `Authorization` header are set. This is done to prevent configuration | ||
| errors. Previously this combination was accepted and the custom `Authorization` header overrode the user/password at request time. | ||
|
|
||
| If an application wants to use both mechanisms, we recommend creating separate Client instances for each authentication method. This will | ||
| also protect against using the default user credentials instead of the custom `Authorization` header. | ||
|
|
||
|
|
||
| ## CLIENT-V2 & JDBC-V2: Access Token and Bearer Token Clarification | ||
|
|
||
| There are two configuration parameters: | ||
| - `com.clickhouse.client.api.ClientConfigProperties#ACCESS_TOKEN` - accepts a raw value and passes it to the `Authorization` HTTP header. | ||
| - `com.clickhouse.client.api.ClientConfigProperties#BEARERTOKEN_AUTH` - adds `Bearer ` before the passed value and writes it to the `Authorization` HTTP header. | ||
|
|
||
| The previous version had an issue that made the client ignore `ACCESS_TOKEN`. This is now fixed, and its value is passed to the `Authorization` HTTP header as is. | ||
| `com.clickhouse.client.api.ClientConfigProperties#BEARERTOKEN_AUTH` is processed as an alias to `com.clickhouse.client.api.Client.Builder#useBearerTokenAuth`, | ||
| which sets `com.clickhouse.client.api.ClientConfigProperties#ACCESS_TOKEN` to `Bearer ` + `token`. | ||
|
|
||
| In most cases `com.clickhouse.client.api.ClientConfigProperties#BEARERTOKEN_AUTH` works better because it adds `Bearer `, so there is no need to do it in the application. | ||
|
|
||
|
|
||
| ## CLIENT-V2 & JDBC-V2: `ClientConfigProperties.MAX_EXECUTION_TIME` (`executionTimeout`) is Always Treated in Milliseconds now | ||
|
|
||
| There was a bug where the `executionTimeout` parameter was set in milliseconds but read in seconds. So it caused a bigger timeout than expected. | ||
|
|
||
| If your application uses a workaround by setting lower values, then you need to adjust it before migrating to the new version. | ||
|
|
||
| See also https://github.com/ClickHouse/clickhouse-java/pull/2841/changes. | ||
|
|
||
| ## CLIENT-V2 & JDBC-V2: SSLMode (`ssl_mode`) Changes | ||
|
|
||
| There was an `ssl_mode` setting with two possible values, `none` and `strict`. It was ignored in V2 for a long time, until now we are re-introducing it. | ||
|
|
||
| The most important change is that V1's `none` maps to `TRUST` in the V2 client and driver because it matches the trust-all logic. There was a problem enabling encryption | ||
| with self-signed certificates. `none` is still supported by JDBC (only) and is an alias for `TRUST`. Please migrate to the new value. | ||
|
|
||
| The V2 client and driver now support these values: | ||
| - `DISABLED` - disables encryption. Do not use it with HTTP. This makes sense only for TCP, where the protocol doesn't provide encryption. HTTP has HTTPS for that. | ||
| - `TRUST` - valid for all protocols. Means encryption only. | ||
| - `VERIFY_CA` - valid for all protocols. Makes the client verify only the CA signature. No identity verification. | ||
| - `STRICT` - valid for all protocols. Default when HTTPS is used. Means complete verification of the server identity. | ||
|
|
||
|
|
||
| ## JDBC-V2: **Deprecation** of `custom_http_params` and `custom_settings` | ||
|
|
||
| We still see usage of `custom_http_params` and `custom_settings`, which comes from V1 of the client and JDBC driver. | ||
|
|
||
| Please migrate to using the `com.clickhouse.client.api.ClientConfigProperties#SERVER_SETTING_PREFIX` (`clickhouse_setting_`) prefix because | ||
| - it is a more solid way to pass settings with a list of values (we had an issue parsing complex properties when they were used within `custom_http_params`). | ||
| - it makes the configuration clearer to understand, as it clearly belongs to server-side settings. | ||
|
|
||
|
|
||
| ## JDBC-V2: **Deprecation** of `use_server_time_zone_for_dates` | ||
|
|
||
| The driver does not apply a timezone to `Date` values anymore, as they have no such information stored in the database. | ||
|
|
||
|
|
||
| ## JDBC-V2: **Deprecation** of `http_connection_provider` | ||
|
|
||
| Driver V2 ignores this setting and uses only the available HTTP client. | ||
| There are plans to support other HTTP clients and protocols, but they will be configured a different way. | ||
|
|
||
|
|
||
| ## JDBC-V2: **Deprecation** of `typeMappings` | ||
|
|
||
| This property (not a feature) is deprecated. Please use `com.clickhouse.jdbc.DriverProperties#JDBC_TYPE_MAPPINGS` (`jdbc_type_mappings`) instead. | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,2 @@ | ||
|
|
||
| ch_java_client_version=0.9.8 | ||
| ch_java_client_version=0.10.0-rc1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| # Floating-point precision in jdbc-v2 | ||
|
|
||
| This note documents how `Float32` and `Float64` values behave when written through the | ||
| `jdbc-v2` driver, why a `Float32` value can come back differing by up to one ULP, and how | ||
| tests should assert floating-point round-trips. | ||
|
|
||
| ## Background | ||
|
|
||
| `PreparedStatement` parameters in `jdbc-v2` are **not** sent in ClickHouse `RowBinary` | ||
| form. Instead the driver serializes each parameter into a **textual SQL literal** and | ||
| substitutes it into the statement before sending it to the server | ||
| (`com.clickhouse.jdbc.PreparedStatementImpl#encodeObject`). | ||
|
|
||
| For numeric primitives there is no dedicated branch, so the value falls through to the | ||
| default rendering: | ||
|
|
||
| ```java | ||
| return SQLUtils.escapeSingleQuotes(x.toString()); | ||
| ``` | ||
|
|
||
| This means: | ||
|
|
||
| - `setFloat(i, x)` → `Float.toString(x)` (e.g. `-3.402823E38`) | ||
| - `setDouble(i, x)` → `Double.toString(x)` (e.g. `-1.7976931348623157E308`) | ||
|
|
||
| `Float.toString` / `Double.toString` produce the *shortest* decimal string that round-trips | ||
| back to the **same** value *of that width* (`Float.parseFloat` for a float, `Double.parseDouble` | ||
| for a double). | ||
|
|
||
| ## Why a `Float32` can shift by one ULP | ||
|
|
||
| When the server reads a decimal literal into a `Float32` column it parses it as `Float64` | ||
| first and then narrows the result to `Float32`. That is a **double rounding**: | ||
|
|
||
| ``` | ||
| decimal text -> Float64 (round #1) -> Float32 (round #2) | ||
| ``` | ||
|
|
||
| The shortest decimal emitted by `Float.toString` is only guaranteed to round-trip via a | ||
| **single** `decimal -> Float32` rounding. Routing it through `Float64` first can land on an | ||
| adjacent `Float32` value. The error is bounded by **one ULP** of the `Float32`, and in | ||
| practice only shows up for extreme magnitudes where the ULP is large. | ||
|
|
||
| Observed example: | ||
|
|
||
| | Java value inserted | Text sent | Value read back | | ||
| | ------------------- | ------------- | ----------------- | | ||
| | `-3.402823E38f` | `-3.402823E38`| `-3.4028229E38f` | | ||
|
|
||
| `Float64` columns are **not** affected: the literal is parsed straight to `Float64` with a | ||
| single rounding, and `Double.toString` already round-trips exactly through that path. | ||
|
|
||
| ## Read side | ||
|
|
||
| On retrieval the binary reader returns boxed primitives that mirror the column type | ||
| (`com.clickhouse.client.api.data_formats.internal.BinaryStreamReader`): | ||
|
|
||
| | Column type | `getObject` returns | | ||
| | ----------- | ------------------- | | ||
| | `Float32` | `java.lang.Float` | | ||
| | `Float64` | `java.lang.Double` | | ||
|
|
||
| So `ResultSet#getObject("float32")` is a `Float`, and `getObject("float64")` is a `Double`. | ||
|
|
||
| ## Implications for callers | ||
|
|
||
| - A `Float32` written via `setFloat` may read back differing by up to one ULP. If you need | ||
| bit-exact `Float32` round-trips, avoid the text path — for example insert through the | ||
| `client-v2` binary writer, or store the value in a `Float64` column. | ||
| - `Float64` written via `setDouble` round-trips exactly. | ||
|
|
||
| ## Testing guidance | ||
|
|
||
| Floating-point round-trips should not be asserted with exact equality for `Float32`. Use a | ||
| one-ULP tolerance for `Float32` and exact equality for `Float64`: | ||
|
|
||
| ```java | ||
| // Float32: allow up to one ULP because of decimal -> Float64 -> Float32 double rounding | ||
| assertEquals(rs.getFloat("float32"), expected32, Math.ulp(expected32)); | ||
|
|
||
| // Float64: exact | ||
| assertEquals(rs.getDouble("float64"), Double.valueOf(expected64)); | ||
| ``` | ||
|
|
||
| The same tolerance applies when comparing the boxed `Float` returned by `getObject`: | ||
|
|
||
| ```java | ||
| Object actual32 = rs.getObject("float32"); | ||
| assertTrue(actual32 instanceof Float); | ||
| assertEquals((float) (Float) actual32, expected32, Math.ulp(expected32)); | ||
| ``` | ||
|
|
||
| See `com.clickhouse.jdbc.JdbcDataTypeTests#testFloatTypes` for a data-set-driven example | ||
| that exercises minimum, maximum, zero, unit, subnormal, constant, and random values. | ||
|
|
||
| ## Related: ClickHouse Cloud read-after-write | ||
|
|
||
| `JdbcDataTypeTests` writes test rows and reads them back. On ClickHouse Cloud, separate | ||
| connections may be routed to different replicas, so a value written on one connection is not | ||
| guaranteed to be immediately visible to a `SELECT` on another. To keep these tests | ||
| deterministic without per-query consistency tuning, perform the write and the verifying read | ||
| on the **same** connection. `JdbcIntegrationTest#runQuery(String, Connection)` is provided to | ||
| run DDL on that same connection and avoid opening extra ones. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.