feat: integrate proxy support for REST/gRPC/OIDC and add new OIDC authentication grant#539
Conversation
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Infrastructure as Code | View in Orca | ||
| SAST | View in Orca | ||
| Secrets | View in Orca | ||
| Vulnerabilities | View in Orca |
|
To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge. |
|
Nice addition, thank you @fer-marino |
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Infrastructure as Code | View in Orca | ||
| SAST | View in Orca | ||
| Secrets | View in Orca | ||
| Vulnerabilities | View in Orca |
🛡️ The following SAST misconfigurations have been detected
| NAME | FILE | ||
|---|---|---|---|
| Prevent Server-Side Request Forgery (SSRF) via User Input in URLs | ...ltGrpcTransport.java | View in code |
Note: The scan should have failed if no policies were configured in warn-only mode.
|
oh sorry. I've added a lot more stuff. Using an http proxy proved much more difficult than I thought, so here is the implementation. I've also added an extra method for the OIDC authentication to use resourceOwnerPassword providing also a client secret id (needed if anon login is disabled) |
|
@fer-marino it's been a busy week, but I'm going to get back to this PR first thing Monday. Appreciate you taking time to contribute!
Could you please elaborate on the difficulties you ran into with configuring http proxy via system properties? I guess one limitation of that approach would be that it doesn't allow using a different proxy per client. Was there something else you encountered? |
bevzzz
left a comment
There was a problem hiding this comment.
I suggest focusing this PR on adding proxy support and introduce ROPC authentication in a separate PR.
src/main/java/io/weaviate/client6/v1/internal/oidc/OidcUtils.java
Outdated
Show resolved
Hide resolved
src/main/java/io/weaviate/client6/v1/internal/oidc/nimbus/NimbusTokenProvider.java
Outdated
Show resolved
Hide resolved
src/main/java/io/weaviate/client6/v1/internal/oidc/nimbus/NimbusTokenProvider.java
Show resolved
Hide resolved
|
hi, I've implemented most of your suggestions. About my difficulties in implementing the proxy, is the different http libraries used by the different interfaces. My initial solution worked only for the rest transport, leaving out gRPC and OIDC authentication. |
|
Thanks! Could you please rebase your branch on the latest |
bevzzz
left a comment
There was a problem hiding this comment.
This is shaping up really well. Here are a couple more things I noticed.
Is there any simple test we could write for the HTTP / gRPC proxy configs? We're already using MockServer for some tests, can we extend the test suite to also verify request are being proxied correctly? e.g. using MockServer's proxying utility
src/main/java/io/weaviate/client6/v1/internal/oidc/nimbus/Flow.java
Outdated
Show resolved
Hide resolved
src/main/java/io/weaviate/client6/v1/internal/oidc/nimbus/NimbusTokenProvider.java
Outdated
Show resolved
Hide resolved
src/main/java/io/weaviate/client6/v1/internal/oidc/nimbus/Flow.java
Outdated
Show resolved
Hide resolved
src/main/java/io/weaviate/client6/v1/internal/oidc/OidcConfig.java
Outdated
Show resolved
Hide resolved
src/main/java/io/weaviate/client6/v1/internal/TokenProvider.java
Outdated
Show resolved
Hide resolved
src/main/java/io/weaviate/client6/v1/internal/oidc/nimbus/NimbusTokenProvider.java
Outdated
Show resolved
Hide resolved
Makes sense, yes. Thanks a lot for taking these two aspects into consideration. |
|
added some more test units, including one for the proxy. I've also modified the pom, as in my setup the lombok annotation processor was not getting invoked during unit tests execution. |
src/test/java/io/weaviate/client6/v1/internal/rest/ProxyTest.java
Outdated
Show resolved
Hide resolved
src/test/java/io/weaviate/client6/v1/internal/rest/ProxyTest.java
Outdated
Show resolved
Hide resolved
src/test/java/io/weaviate/client6/v1/internal/rest/ProxyTest.java
Outdated
Show resolved
Hide resolved
src/main/java/io/weaviate/client6/v1/internal/oidc/OidcConfig.java
Outdated
Show resolved
Hide resolved
src/main/java/io/weaviate/client6/v1/internal/oidc/OidcConfig.java
Outdated
Show resolved
Hide resolved
src/main/java/io/weaviate/client6/v1/internal/oidc/nimbus/NimbusTokenProvider.java
Show resolved
Hide resolved
src/test/java/io/weaviate/testutil/transport/MockRestTransport.java
Outdated
Show resolved
Hide resolved
src/test/java/io/weaviate/testutil/transport/MockRestTransport.java
Outdated
Show resolved
Hide resolved
Thank you 👍 |
|
pushed another commit with your suggestions |
|
@fer-marino please update your branch to the latest +3 pending comments ⬆️ |
|
rebased |
|
@fer-marino looks like the commit hashes got changed in the rebase and the history/diff in this PR now includes changes already present in the main branch. Could you please squash your changes in to a single commit and force-push the branch? Something like: git reset --soft origin/main
git commit -m "feat: add proxy support and ROPC authorization grant"
git push --force-with-leaseIt'll make it easier for me to wrap up the review. |
35a8d82 to
907f71b
Compare
|
hi, have a look now |
src/main/java/io/weaviate/client6/v1/internal/oidc/OidcConfig.java
Outdated
Show resolved
Hide resolved
src/main/java/io/weaviate/client6/v1/internal/grpc/DefaultGrpcTransport.java
Outdated
Show resolved
Hide resolved
|
hey @fer-marino! There's one unit test that's prone to deadlocks (it's causing the pipeline in this PR to hang rn), which was fixed in the #549. That PR just got merged to |
|
@fer-marino the commit hashes got changed after your latest rebase and the history/diff in this PR now includes changes already present in the main branch. Could you please squash your changes in to a single commit and force-push the branch? Something like: git reset --soft origin/main
git commit -m "feat: add proxy support and ROPC authorization grant"
git push --force-with-leaseIt'll make it easier for me to wrap up the review. Alternatively, you can always merge the main branch into your feature branch instead of rebasing them -- this should keep the original hashes intact and the diff focused. |
514f742 to
311dc69
Compare
|
Some unit tests are failing seemingly after the merge, I'll fix them to unblock the PR. |
bevzzz
left a comment
There was a problem hiding this comment.
Congratulations on your first PR, @fer-marino!
This PR introduces comprehensive proxy support for both REST and gRPC communications, as well as for OIDC authentication.
Changes
Features
Proxyrecord to hold proxy configuration (host, port, scheme, and optional credentials).proxy(Proxy)method to theConfigbuilder, allowing users to configure a proxy globally for all client operations.resourceOwnerPasswordCredentialsto theAuthenticationinterface, supporting the Resource Owner Password Credentials authorization grant.DefaultRestTransportto use the configured proxy with Apache HttpClient 5.DefaultGrpcTransport.NimbusTokenProviderandOidcUtils, ensuring that OIDC metadata and token requests (fetching and refreshing) correctly respect the proxy settings.Refactoring
Testing
ProxyTestand updatedAuthenticationTestto verify proxy integration for both REST requests and OIDC authentication flows (including Resource Owner Password grant).Verification