Kubernetes ingress custom JWT authentication cache key

Question

We are leveraging Kubernetes ingress with external service JWT authentication using auth-url as a part of the ingress.

Now we want to use the auth-cache-key annotation to control the caching of JWT token. At current our external auth service just respond with 200/401 by looking at the token. All our components are backend micro-services with rest api. Incoming request may not be the UI request. How do we fill in the `auth-cache-key' for a JWT token coming in.

 annotations:
 nginx.ingress.kubernetes.io/auth-url: http://auth-service/validate
 nginx.ingress.kubernetes.io/auth-response-headers: "authorization"
 nginx.ingress.kubernetes.io/auth-cache-key: '$remote_user$http_authorization'
 nginx.ingress.kubernetes.io/auth-cache-duration: '1m'
 kubernetes.io/ingress.class: "nginx"

Looking at the example, $remote_user$http_authorization is specified as an example in K8s documentation. However not sure if $remote_user will be set in our case. Because this is not external basic auth. How do we decide on the auth cache key in case of this?

Not enough example/documentations exists around this.

• Mikolaj S.: Which Kubernetes and NGINX Ingress controller versions are you using?
• Mikolaj S.: Could you explain what do you mean by "Incoming request may not be the UI request. and "How do we decide on the auth cache key in case of this?" What exactly is your issue? What do you want to achieve?
• Mikolaj S.: Could you provide a minimal reproducible example of your current setup?

Answer 1

Posting general answer as no further details and explanation provided.

It's true that there is not so much documentation around, so I decided to dig into NGINX Ingress source code.

The value set in annotation nginx.ingress.kubernetes.io/auth-cache-key is a variable $externalAuth.AuthCacheKey in code:

{{ if $externalAuth.AuthCacheKey }}
set $tmp_cache_key '{{ $server.Hostname }}{{ $authPath }}{{ $externalAuth.AuthCacheKey }}';
set $cache_key '';

As can see, $externalAuth.AuthCacheKey is used by variable $tmp_cache_key, which is encoded to base64 format and set as variable $cache_key using lua NGINX module:

rewrite_by_lua_block {
 ngx.var.cache_key = ngx.encode_base64(ngx.sha1_bin(ngx.var.tmp_cache_key))
}

Then $cache_key is used to set variable $proxy_cache_key which defines a key for caching:

proxy_cache_key "$cache_key";

Based on the above code, we can assume that we can use any NGINX variable to set nginx.ingress.kubernetes.io/auth-cache-key annotation. Please note that some variables are only available if the corresponding module is loaded.

Example - I set following auth-cache-key annotation:

nginx.ingress.kubernetes.io/auth-cache-key: '$proxy_host$request_uri'

Then, on the NGINX Ingress controller pod, in the file /etc/nginx/nginx.conf there is following line:

set $tmp_cache_key '{my-host}/_external-auth-Lw-Prefix$proxy_host$request_uri';

If you will set auth-cache-key annotation to nonexistent NGINX variable, the NGINX will throw following error:

nginx: [emerg] unknown "nonexistent_variable" variable

It's up to you which variables you need.

Please check also following articles and topics:

• A Guide to Caching with NGINX and NGINX Plus
• external auth provider results in a lot of external auth requests