docs(bigquery-jdbc): add user guide with connection property and custom endpoint reference#13878
docs(bigquery-jdbc): add user guide with connection property and custom endpoint reference#13878Neenu1995 wants to merge 9 commits into
Conversation
…om endpoint reference
There was a problem hiding this comment.
Code Review
This pull request adds a comprehensive User Guide (USER_GUIDE.md) for the Google BigQuery JDBC Driver, detailing configuration, architecture, and common code recipes. The review feedback correctly identifies several inconsistencies in the document, specifically regarding incorrect OAuthType values in the authentication examples (confusing OAuthType=0 for Service Accounts with OAuthType=3 for Application Default Credentials) and mismatches in the Table of Contents headings.
…rties # Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive User Guide for the Google BigQuery JDBC Driver, detailing setup, configuration properties, architecture, and common code recipes. The review feedback highlights several critical corrections needed in the documentation's code examples: updating the transaction prerequisite exception from IllegalStateException to SQLException to match the JDBC contract, adding null and type safety checks to the complex types example to prevent runtime exceptions, replacing the stored procedure example with a multi-statement script since standard JDBC OUT parameters are unsupported, and wrapping the Spring Boot YAML connection URL in double quotes to avoid parsing issues.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive User Guide (USER_GUIDE.md) for the Google BigQuery JDBC Driver, detailing configuration, authentication, connection properties, data type mappings, architecture, and common user journeys with code recipes. The review feedback correctly identifies two key improvements: updating the Table of Contents to accurately match the headers in Section 9, and adding a final batch execution check in the ETL ingestion code recipe to prevent potential data loss when the batch size is not a multiple of 500.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…ogle-cloud-java into pr-1-user-guide-properties
|
|
||
| The driver supports multiple OAuth 2.0 and identity workflows specified via the `OAuthType` connection property. | ||
|
|
||
| | `OAuthType` | Authentication Strategy | Required Properties | |
There was a problem hiding this comment.
Need to re-structure this table to be more clear:
For each auth type what properties are supported and what is required rather than using and and or in the text.
| | **`1`** | **User Account (Interactive 3-Legged OAuth)** | `OAuthClientId`, `OAuthClientSecret` | | ||
| | **`2`** | **Pre-generated Access Token or Refresh Token** | `OAuthAccessToken` or `OAuthRefreshToken` (`OAuthClientId`, `OAuthClientSecret`) | | ||
| | **`3`** | **Application Default Credentials (ADC)** / gcloud CLI | None (reads `GOOGLE_APPLICATION_CREDENTIALS` or `gcloud` context) | | ||
| | **`4`** | **Workload Identity Federation (BYOID)** | `BYOID_AudienceUri`, `BYOID_CredentialSource`, `BYOID_SubjectTokenType` | |
There was a problem hiding this comment.
It supports OAuthPvtKeyPath for the json file too.
|
|
||
| | Property Name | Default Value | Description | | ||
| | :--- | :---: | :--- | | ||
| | `OAuthType` | `-1` | Authentication mechanism: `0` (Service Account), `1` (User Account), `2` (Pre-generated Token / Refresh Token), `3` (ADC), `4` (Workload Identity Federation / BYOID). | |
There was a problem hiding this comment.
Need to mark that this is Required property rather than using -1 as default.
| | `ServiceAccountImpersonationChain` | `null` | Comma-separated list of service account emails representing the impersonation chain. | | ||
| | `ServiceAccountImpersonationScopes` | `https://www.googleapis.com/auth/bigquery` | Comma-separated OAuth 2.0 scopes for impersonated credentials. | | ||
| | `ServiceAccountImpersonationTokenLifetime` | `3600` | Lifetime (in seconds) for impersonated service account tokens. | | ||
| | `RequestGoogleDriveScope` | `0` | If `1` (or `true`), appends the Google Drive read-only scope (`https://www.googleapis.com/auth/drive.readonly`) to query external Drive tables. | |
There was a problem hiding this comment.
Better to use bool values where possible
|
|
||
| | Property Name | Default Value | Description | | ||
| | :--- | :---: | :--- | | ||
| | `ProjectId` | *Default Project* | Google Cloud Project ID for billing and query execution. | |
There was a problem hiding this comment.
iirc it is required, there is no default
| | **`BIGQUERY`** | BigQuery Core REST API | `https://www.googleapis.com/bigquery/v2/` | Regional endpoints (e.g. `us-east4`) or PSC private VIPs. | | ||
| | **`READ_API`** | Storage Read API (HTAPI) | `bigquerystorage.googleapis.com:443` | Private gRPC endpoints for high-throughput extractions. | | ||
| | **`OAUTH2`** | Google OAuth2 Token Server | `https://oauth2.googleapis.com/token` | Private OAuth token exchange gateways. | | ||
| | **`STS`** | Security Token Service | `https://sts.googleapis.com` | Workload Identity / BYOID token exchange. | |
There was a problem hiding this comment.
Workload might be using its own set of urls provided via credentials file,
| ;EndpointOverrides=BIGQUERY=https://us-east4-bigquery.googleapis.com | ||
| ``` | ||
|
|
||
| #### 2. Custom Endpoint Requirements per Authentication Type (`OAuthType`) |
There was a problem hiding this comment.
What's the value of this table? I'd just remove it.
It has a lot of mistakes too + I don't think there is "primary enterprise use case" idea. It is one example of when it can be used, but what makes it 'primary'?
|
|
||
| --- | ||
|
|
||
| #### 3. Endpoint Configuration Code Recipes by `OAuthType` |
There was a problem hiding this comment.
Private endpoints & auth are not related to each other, so I don't see any value in having examples of mixing them together.
We need user guide describing OAuthType behavior, user guide describing EndpointsOverride behavior, but we shouldn't generate all possible combinations of both.
| ```yaml | ||
| spring: | ||
| datasource: | ||
| url: "jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=my-gcp-project;DefaultDataset=my_dataset;OAuthType=0;OAuthServiceAcctEmail=sa@my-project.iam.gserviceaccount.com;OAuthPvtKeyPath=/path/to/key.json" |
There was a problem hiding this comment.
Can remove OAuthServiceAcctEmail as it is actually unused.
|
|
||
| --- | ||
|
|
||
| ## 10. Logging, Diagnostics & Troubleshooting |
There was a problem hiding this comment.
Please also add information about logging env vars available at https://docs.cloud.google.com/bigquery/docs/jdbc-for-bigquery#logging
b/538176465