feat(datastore): add support for request tags#13732
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces support for passing and merging RequestOptions (such as request tags) in Datastore queries and aggregation queries, allowing both instance-level and request-level tags to be propagated. The feedback highlights a critical issue where instance-level tags are ignored when query-level RequestOptions are not explicitly provided in QueryResultsImpl and AggregationQueryRequestProtoPreparer. Additionally, the reviewer suggests simplifying the varargs withRequestTags overload in Datastore by delegating to the List version, and adding a test case to verify that instance-level tags are correctly populated and sent with queries.
f124116 to
aab1fd3
Compare
|
ci / lint is failing. Could you please take a look? |
| return this; | ||
| } | ||
|
|
||
| public Builder setTags(ImmutableList<String> requestTags) { |
There was a problem hiding this comment.
Would it be better to use setReplaceTags?
There was a problem hiding this comment.
I think setXx inherently means that. Setting the existing with a new value which is effectively a replace.
There was a problem hiding this comment.
Sorry, I don't follow. Could you please explain a bit more?
I meant that because we are setting requestTags and since we have getRequestTags(), I'm thinking it'll be better to stay consistent. Having the builder methods match the property/getter name is standard convention in the SDK and helps users discover the setters via IDE auto-complete.
There was a problem hiding this comment.
I think in your original comment "Would it be better to use setReplaceTags?" you intended to say setRequestTags instead of replace, and hence the confusion. I have renamed the method to setRequestTags()
| return this; | ||
| } | ||
|
|
||
| public Builder setTags(String... requestTags) { |
There was a problem hiding this comment.
Would it be better to use setReplaceTags?
There was a problem hiding this comment.
I think setXx inherently means that. Setting the existing with a new value which is effectively a replace.
There was a problem hiding this comment.
Sorry, I don't follow. Could you please explain a bit more?
I meant that because we are setting requestTags and since we have getRequestTags(), I'm thinking it'll be better to stay consistent. Having the builder methods match the property/getter name is standard convention in the SDK and helps users discover the setters via IDE auto-complete.
There was a problem hiding this comment.
Replied in the previous comment.
| <T> QueryResults<T> run(Query<T> query, RequestOptions requestOptions); | ||
|
|
||
| /** | ||
| * Submits a {@link Query} with specified {@link com.google.datastore.v1.RequestOptions} and | ||
| * returns its result. {@link ReadOption}s can be specified if desired. | ||
| */ | ||
| <T> QueryResults<T> run(Query<T> query, RequestOptions requestOptions, ReadOption... options); | ||
|
|
||
| /** | ||
| * Submits a {@link Query} with specified {@link com.google.cloud.datastore.models.ExplainOptions} | ||
| * and {@link com.google.datastore.v1.RequestOptions} and returns its result. {@link ReadOption}s | ||
| * can be specified if desired. | ||
| */ | ||
| @BetaApi | ||
| <T> QueryResults<T> run( | ||
| Query<T> query, | ||
| ExplainOptions explainOptions, | ||
| RequestOptions requestOptions, | ||
| ReadOption... options); |
There was a problem hiding this comment.
This looks like we have to overload run and runAggregation every single time we add a variation of ...Option in here.
Would it be possible to just create a wrapping Option class (e.g. DatastoreExecutionOption or w/e name that wraps these three options)?
That way any new request option doesn't require an overload of new public methods for the Datastore interface
There was a problem hiding this comment.
We actually dont foresee any additional options coming in near future. Shall we park this refactoring for later?
There was a problem hiding this comment.
I know some of the changes above from older implementations aren't related to your feature. However, even if we don't ever add in any new option variant, this PR will still add in 3 new public methods that can't be removed without breaking changes.
If we are to add anything new to the public surface, I would suggest that go for the variant that only adds in one (unless there is a valid reason to not do this):
e.g. run(Query, DatastoreExecutionOption) that manages to wrap all current options.
I think we can just delegate all the existing variants to create this new root options wrapper
Same below with runAggregation
| default Datastore withRequestTags(String... requestTags) { | ||
| return withRequestTags(java.util.Arrays.asList(requestTags)); | ||
| } |
There was a problem hiding this comment.
do we need a varargs variant? Why can't the list overload below work for all use cases?
There was a problem hiding this comment.
Makes sense, removed it.
There was a problem hiding this comment.
I think the method still exists upsteam. Might have forgot to push to remote
Fixed |
| return this; | ||
| } | ||
|
|
||
| public Builder setTags(ImmutableList<String> requestTags) { |
There was a problem hiding this comment.
Sorry, I don't follow. Could you please explain a bit more?
I meant that because we are setting requestTags and since we have getRequestTags(), I'm thinking it'll be better to stay consistent. Having the builder methods match the property/getter name is standard convention in the SDK and helps users discover the setters via IDE auto-complete.
| return this; | ||
| } | ||
|
|
||
| public Builder setTags(String... requestTags) { |
There was a problem hiding this comment.
Sorry, I don't follow. Could you please explain a bit more?
I meant that because we are setting requestTags and since we have getRequestTags(), I'm thinking it'll be better to stay consistent. Having the builder methods match the property/getter name is standard convention in the SDK and helps users discover the setters via IDE auto-complete.
| * @param requestTags the request tags to append to existing ones | ||
| */ | ||
| default Datastore withRequestTags(List<String> requestTags) { | ||
| ImmutableList.Builder<String> builder = ImmutableList.builder(); |
There was a problem hiding this comment.
Can the requestTags be null here? And if so, would it make sense to just return the current client rather than build a new one?
e.g.
if (requestTags == null || requestTags.isEmpty()) {
return this;
}
| CommitRequest.Builder requestPb = | ||
| CommitRequest.newBuilder() | ||
| .setMode(CommitRequest.Mode.NON_TRANSACTIONAL) | ||
| .setProjectId(getOptions().getProjectId()) | ||
| .setDatabaseId(getOptions().getDatabaseId()) | ||
| .addAllMutations(mutationsPb); |
There was a problem hiding this comment.
Do we need to support instance-level tags here? e.g.
if (!getOptions().getRequestTags().isEmpty()) {
requestPb.setRequestOptions(RequestOptionsHelper.createRequestOptions(getOptions(), null));
}
There was a problem hiding this comment.
Do we need to support instance-level tags here? e.g.
if (!getOptions().getRequestTags().isEmpty()) {
requestPb.setRequestOptions(RequestOptionsHelper.createRequestOptions(getOptions(), null));
}
| @@ -425,23 +508,21 @@ public List<Key> allocateId(IncompleteKey... keys) { | |||
| if (keys.length == 0) { | |||
| return Collections.emptyList(); | |||
| } | |||
| com.google.datastore.v1.AllocateIdsRequest.Builder requestPb = | |||
| com.google.datastore.v1.AllocateIdsRequest.newBuilder(); | |||
| AllocateIdsRequest.Builder requestPb = AllocateIdsRequest.newBuilder(); | |||
| for (IncompleteKey key : keys) { | |||
| requestPb.addKeys(trimNameOrId(key).toPb()); | |||
| } | |||
| requestPb.setProjectId(getOptions().getProjectId()); | |||
| requestPb.setDatabaseId(getOptions().getDatabaseId()); | |||
There was a problem hiding this comment.
Do we need to support instance-level tags here? e.g.
if (!getOptions().getRequestTags().isEmpty()) {
requestPb.setRequestOptions(RequestOptionsHelper.createRequestOptions(getOptions(), null));
}
| /** | ||
| * Returns a new Datastore client with the specified request tags added. | ||
| * | ||
| * @param requestTags the request tags to append to existing ones | ||
| */ | ||
| default Datastore withRequestTags(List<String> requestTags) { | ||
| ImmutableList.Builder<String> builder = ImmutableList.builder(); | ||
| builder.addAll(getOptions().getRequestTags()); | ||
| builder.addAll(requestTags); | ||
| return getOptions().toBuilder().setTags(builder.build()).build().getService(); | ||
| } |
There was a problem hiding this comment.
qq, what is the user need to re-build a Datastore client with new request tags via withRequestTags(...)? I'm not entirely sure I follow why we need a new default method on the Datastore interface.
Why can't the user just set the tag in the DatastoreOption builder and then manually create the client again
e.g. something like
DatastoreOptions.toBuilder().setTags(...).build.getService()
6c23afc to
e5396c0
Compare
run(...)andrunAggregation(...)client methods, that passRequestOptionsproto as the last parameter.RequestOptionsHelperutility and sent inside the query requests.