407 Proxy Authentication Required Error for GCP Java SDK calls

Question

I'm trying to build an application which uses GCP's buckets in the backend. I'm using Java SDK to build the application. I'm using Access Credential JSON file to authenticate the request.

GCP calls made from the code throws the error com.google.cloud.storage.StorageException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authentication Required"

Exception Trace:

com.google.cloud.storage.StorageException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authentication Required"
 at com.google.cloud.storage.StorageException.translate(StorageException.java:170)
 at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:313)
 at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:504)
 at com.google.cloud.storage.StorageImpl.lambda$internalBucketGet65ドル(StorageImpl.java:1589)
 at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103)
 at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
 at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
 at com.google.cloud.storage.Retrying.run(Retrying.java:65)
 at com.google.cloud.storage.StorageImpl.run(StorageImpl.java:1514)
 at com.google.cloud.storage.StorageImpl.internalBucketGet(StorageImpl.java:1587)
 at com.google.cloud.storage.StorageImpl.get(StorageImpl.java:316)
 at com.application.DataHandler.getData(DataHandler.java:310)

But I have set the correct proxy credentials via HttpTransportFactory, in the GoogleCredentials object.

Code snippet:

import java.io.InputStream;
import java.net.Authenticator;
import java.net.PasswordAuthentication;
import java.nio.channels.Channels;
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.HttpClient;
import org.apache.http.conn.routing.HttpRoutePlanner;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.ProxyAuthenticationStrategy;
import org.apache.http.impl.conn.DefaultProxyRoutePlanner;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.http.apache.v2.ApacheHttpTransport;
import com.google.auth.http.HttpTransportFactory;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.cloud.storage.Bucket;
import com.google.cloud.storage.Storage;
private static void queryBucket(InputStream accessCredentialJson) {
 GoogleCredentials googleCredentials = GoogleCredentials.fromStream(accessCredentialJson, getHttpTransportFactory());
 Storage storage = StorageOptions.newBuilder().setCredentials(googleCredentials).build().getService();
 Bucket gcpBucketObj = storage.get("gcpBucket"); // --> Throws 407 Proxy Auth Reqd. Error
}
public HttpTransportFactory getHttpTransportFactory() {
 HttpHost proxyHostDetails = new HttpHost(PROXY_HOST, PROXY_PORT);
 HttpRoutePlanner httpRoutePlanner = new DefaultProxyRoutePlanner(proxyHostDetails);
 CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
 credentialsProvider.setCredentials(
 new AuthScope(proxyHostDetails.getHostName(), proxyHostDetails.getPort()),
 new UsernamePasswordCredentials(PROXY_USERNAME, PROXY_PASSWORD)
 );
 HttpClient httpClient = ApacheHttpTransport.newDefaultHttpClientBuilder()
 .setRoutePlanner(httpRoutePlanner)
 .setProxyAuthenticationStrategy(ProxyAuthenticationStrategy.INSTANCE)
 .setDefaultCredentialsProvider(credentialsProvider)
 .build();
 final HttpTransport httpTransport = new ApacheHttpTransport(httpClient);
 return new HttpTransportFactory() {
 @Override
 public HttpTransport create() {
 return httpTransport;
 }
 };
 }

I have verified that my proxy credentials are correct.

When I tried connecting google servers from my machine directly using curl request with proxy credentials, the connection was established, meaning proxy credentials are correct. Only when request is made via Google Cloud SDK, i'm facing this error.

I have even tried setting the Proxy in System environment variables, like

System.setProperty("https.proxyHost", PROXY_HOST);
System.setProperty("https.proxyPort", PROXY_PORT);

It didn't help too.

Please let me know what I'm doing wrong and help me resolve this issue.

Dependency Jars used: (with version)

google-cloud-storage-2.29.1.jar
google-cloud-core-2.21.0.jar
google-cloud-core-http-2.21.0.jar
google-auth-library-oauth2-http-1.19.0.jar
google-auth-library-credentials-1.19.0.jar
gax-2.31.0.jar
guava-30.1-jre.jar
gson-2.10.jar
threetenbp-1.6.8.jar
api-common-2.14.1.jar
google-http-client-gson-1.42.3.jar
google-http-client-1.42.3.jar
google-http-client-apache-v2-1.43.3.jar
google-http-client-appengine-1.43.3.jar
google-http-client-jackson2-1.42.3.jar
google-api-client-1.34.0.jar
opencensus-api-0.31.1.jar
opencensus-contrib-http-util-0.31.1.jar
google-api-services-storage-v1-rev20231028-2.0.0.jar
gax-httpjson-2.31.0.jar
httpclient-4.5.14.jar

• Raniz: I've run your code with a local Squid proxy, and everything works. However, if I set the password to something invalid, I get 407 proxy authentication errors. Could you turn on logging for org.apache.http and post the logs? You may need to add the jcl-over-slf4j package to your project if you're using an SLF4j logging implementation. Don't forget to remove the base64 encoded credentials from the logs :)
• Kumaran: Thanks @Raniz for the help. I have solved the issue by adding the TransportOptions to the Storage object. I have included the answer below.

Answer 1

Have found the fix.

Adding TransportOptions to the Storage object solved the issue.

Storage storage = StorageOptions.newBuilder().setTransportOptions(HttpTransportOptions.newBuilder().setHttpTransportFactory(getHttpTransportFactory()).build()).setCredentials(googleCredentials).build().getService();