Levels of caching

• Service level - all responses to all operations in this service are cached
• Operation level - all responses to this operation are cached
• Response level - this response is cached

The levels are for developer convince. On the wire, all cache control is handled at the response level.

Server side

• Declararative
  • Services or operations marked with an IdempotencyLevel in their proto will get caching automatically
  • Services or operations can be marked with a protobuf option communicating their desired cache eviction strategy
• Imperative
  • Service responses programmatically tagged with cache-control metadata during service execution.

On the wire

Cache control policy is communicated to clients using standard HTTP headers on a per-message basis.

• A gRPC ServerInterceptor converts cache control policy into response HTTP headers.
• Cacheable service responses are marked with the following HTTP headers
  • Cache-Control:
    • max-age=xxx - the max number of seconds the response can be cached, before requiring a refresh
  • ETag: "ID" - a hash or version of the response used for cache re-validation

Client Side

• Client-side caching handled by a client interceptor
• The client specifies the maximum size of the cache in megabytes using some kind of configuration
• If a cache line is expired but not yet evicted, the next request for the cached request will include an If-None-Match: xxx header, including the etag of the expired cached response
• If the server's response has still not changed after cache expiration, a HTTP 304 Not Modified response with a fresh Cache-Control header will be sent in lieu of the response message, instructing the client to continue relying on its cache.