Redis strategy for cache item dependency

Question

I'm implementing cache structure using Redis, my domain is like this:

• There are multiple categories (3000+ categories). Each category contains multiple questions (around 20-30 questions per category). Each question has several properties like whether it's published, what's the type, etc
• When loading the questions, I'll need to load questions from several categories and with some question filers specified by user that matches question properties: is published, type, etc
• I implement this in Redis using string and set:
  • Each question is stored as string in Redis (serialized JSON) with key: "question:[QuestionId]"
  • Each category has list of question ids belonging to it (Redis set) with key: "category:[CategoryId]:questions"
  • For each category, each property filterable by user has its own set of question id (Redis set) e.g. published questions under category 1 has key: "category:1:questions:ispublished
  • When loading the questions, I just need to based on the filters provided by user, do the intersect of set in Redis to find relevant question ids and load those questions
  • So each category will have several cache items associated with it:
    1. category:[CategoryId]:questions
    2. category:[CategoryId]:questions:ispublished
    3. category:[CategoryId]:questions:notpublished
    4. category:[CategoryId]:questions:type:1
    5. category:[CategoryId]:questions:type:2 etc

This is working fine. But the challenging is to maintain consistency of cache items when there's eviction. Let's say if Redis cache decides to evict "category:[CategoryId]:questions:notpublished" cache item, how do i make sure all other cache items related to that category id also invalidated? I can do the checking of all the filters cache item to make sure they exist in cache before loading the questions, but that will require a lot of requests to Redis. Or any other strategy/way to solve my problem? I'm new to Redis and this is the first time I implement this type of cache structure in Redis

At the moment what I do is I just check for "category:[CategoryId]:questions" key. If it doesn't exist then I load data from database and set all cache items for that category. But it cannot detect if any other key is missing

Answer 1

You can attack this issue from a couple of angles:

1. You can choose an eviction policy that will not evict those keys, e.g, eviction policy volatile-lru and set those keys without expiration. (more on eviction policies)
   The issue here is that you might get OOM if there are no keys that can evict.
2. The second approach is to set an eviction listener, you can use the notify-keyspace-events to make redis publish keyspace and keyevent notifications. (more on key notifications)
   The following command will make redis publish evictions on it keyspace notification channel redis-cli config set notify-keyspace-events Ke
   And the following command will establish a connection subscribed to all redis notifications: redis-cli --csv psubscribe '__key*__:*'
   This is clearly just an example of getting it done with redis-cli, you should implement it as a separate thread in your application that deletes all evicted key related data from your dataset.

• Phuong Nguyen: interesting. I didn't know about the keyspace events before. But I'm using Azure redis which seems not supporting keyspace events at the moment. So that leaves me with option 1 only. I guess I'll need to make sure the cache is not full
• readyornot: @PhuongNguyen, what did you end up doing? I'm in a similar situation and am considering implementing my own application level eviction strategy, whereby I manage the deletion of older dependent keys when newer ones get added.
• Phuong Nguyen: @readyornot I ended up with option 1, set keys with no expiration and make sure the cache has more than enough memory to store all data I need. I also have a worker to periodically check the cache and update from database to make sure the cache up-to-date with database. When I delete the parent key, I also make sure to delete all the dependency keys