Problems flushing Magento Redis Cache on an installation with a separate backend server

Question

My problem is that I do not think I am able to refresh the magento redis cache from the admin page. I realize that the problem could come from many sources, but my gut tells me it has something to do with the backend being on a separate server. My magento installation is as follows:

• Backend server and NFS(media) on an Amazon AWS EC2 at http://admin.example.com
• Database on AWS RDS MySQL 2 app servers (scalable to more) on AWS Elastic Beanstalk at http://www.example.com (route53)
• regular backend cache(database 0), Lesti-FPC(database 0), and redis_session (database 1) on AWS elasticache redis

I originally had my Lesti-FPC configured to use database 2 on the redis cache. I thought it worked pretty well as far as I could tell, until I realized that I couldn't flush the cache at all from the admin System>Cache Management page. "Flush Magento Cache," "Flush Cache Storage," "disable", and "refresh" did nothing. I could only flush it by rebooting the redis node or going in with redis-cli and using redis commands.

I then tried configuring Lesti-FPC to use database 0 as described above. It worked better. Now, I could flush the FPC with "Flush Cache Storage," although the other options still didn't work. At the time, I assumed it was an issue specifically with Lesti-FPC. But anyway, using "Flush Cache Storage" was good enough for me at the time, especially once I discovered that I could flush the cache through code using

Mage::app()->getCacheInstance()->flush();

I just recently found out that the problem may not be specific to Lesti-FPC. While trying to fix the Lesti issue, I tried monitoring redis. I know nothing about redis or caching, but when I would try to refresh the FPC, I would see commands like:

"del" "zc:ti:403_FPC"
"srem" "zc:tags" "403_FPC"

But those tags never existed. Doing:

keys *FPC*

in redis would give me

"zc:ti:109_FPC"

but nothing with 403. SO this means that my fpc caches do not get invalidated like they're supposed to after product/category changes and reindexing. I got around this by manually flushing the cache after changes and running cron jobs to flush and prime the fpc every few hours.

But it made me suspicious. I tried refreshing the other caches from the admin, and I found that magento would always try to delete and read the 403 keys(some of which existed and some of which did not) but never any 109 keys (of which there are many).

My guess is that the 403 keys are specific to the admin server, and the 109 keys are specific to the app servers. The admin server, maybe because it is on a different subdomain, is not touching the app servers' cached stuff. But the app servers are able to find its own keys fine, as demonstrated by the fact that the FPC is working very well.

Does this make sense? Is there something I could do to fix this? Did I configure something incorrectly or is this a magento bug?

• medmek: can someone point me in the equivalent of flushing all redis in M2 (ie. without using redis-cli flushall) ? thank you.

Answer 1

In case anyone sees this. I've posted this info elsewhere, but the 109 and 403 prefixes come from an MD5 hash of the path to the path/etc folder. If your folder is in different places on different servers (or the same place on different servers that are sharing a cache) you can run into problems.

• Shapeshifter: That is good to know
• Fabian Schmengler: Thank you so much for pointing it out! Magento 2.2 still has the same issue, unless you add id_prefix to the configuration, to set an explicit prefix

Answer 2

we should always use cache:clean instead of flush for redis cache