Skip to content

Should be possible to use GCS without authentication #25810

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
patricklucas opened this issue May 16, 2025 · 0 comments · May be fixed by #25811
Open

Should be possible to use GCS without authentication #25810

patricklucas opened this issue May 16, 2025 · 0 comments · May be fixed by #25811

Comments

@patricklucas
Copy link

patricklucas commented May 16, 2025 •
edited
Loading

My use-case is local development and testing using fake-gcs-server. Auth is not required to use its GCS-compatible API.

However, Trino does not appear to have a way to not attempt authentication with Google's servers, even when setting the gcs.endpoint property.

If no auth-related properties (like gcs.json-key) are set, then GcsStorageFactory calls GoogleCredentials.getApplicationDefault().

However, this method throws an exception if no credentials are found:

Caused by: java.io.IOException: Your default credentials were not found. To set up Application Default Credentials for your environment, see https://cloud.google.com/docs/authentication/external/set-up-adc.

In contrast, if StorageOptions.Builder#setCredentials is simply never called, the underlying GCS library populates it by calling GoogleCredentials.getApplicationDefault() but ignoring any exceptions it throws.

I have a temporary workaround to just not call StorageOptions.Builder#setCredentials in GcsStorageFactory if gcs.endpoint is set:

diff --git a/lib/trino-filesystem-gcs/src/main/java/io/trino/filesystem/gcs/GcsStorageFactory.java b/lib/trino-filesystem-gcs/src/main/java/io/trino/filesystem/gcs/GcsStorageFactory.java
index f8ea12c452..b6b20537fd 100644
--- a/lib/trino-filesystem-gcs/src/main/java/io/trino/filesystem/gcs/GcsStorageFactory.java
+++ b/lib/trino-filesystem-gcs/src/main/java/io/trino/filesystem/gcs/GcsStorageFactory.java
@@ -93,6 +93,9 @@ public class GcsStorageFactory
                     credentials = GoogleCredentials.fromStream(inputStream).createScoped(DEFAULT_SCOPES);
                 }
             }
+            else if (endpoint.isPresent()) {
+                credentials = null;
+            }
             else {
                 credentials = jsonGoogleCredential.orElseGet(() -> {
                     try {
@@ -110,10 +113,13 @@ public class GcsStorageFactory

             endpoint.ifPresent(storageOptionsBuilder::setHost);

+            if (credentials != null) {
+                storageOptionsBuilder.setCredentials(credentials);
+            }
+
             // Note: without uniform strategy we cannot retry idempotent operations.
             // The trino-filesystem api does not violate the conditions for idempotency, see https://cloud.google.com/storage/docs/retry-strategy#java for details.
             return storageOptionsBuilder
-                    .setCredentials(credentials)
                     .setStorageRetryStrategy(getUniformStorageRetryStrategy())
                     .setRetrySettings(RetrySettings.newBuilder()
                             .setMaxAttempts(maxRetries + 1)

The three solutions I see are, in order of preference:

  • where GcsStorageFactory currently calls getApplicationDefault(), just set credentials to null instead, and if it is indeed null, do not call StorageOptions.Builder#setCredentials, allowing the underlying library's behavior to apply
  • suppress exceptions thrown by GoogleCredentials.getApplicationDefault() like the GCS client library itself does by default
  • add an additional config property to disable auth for GCS altogether, perhaps resulting in passing NoCredentials.getInstance() as the credentials

I'm happy to open a PR for this.
@patricklucas patricklucas linked a pull request May 16, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Allow using GCS without providing credentials patricklucas/trino
1 participant
@patricklucas