-
-
Notifications
You must be signed in to change notification settings - Fork 92
Redis downgrade migration #65
Unanswered
leonchen83
asked this question in
Q&A
-
Redis Downgrade Migration
1. Source of the Issue
Related issues and discussions in Redis:
- Downgrade support in Redis
- How to restore redis 3.0.6 with redis 4.0's dump.rdb
- Sync data between KeyDB v6 - Redis v7
- How to migrate RDB to new format supported by Redis 7?
2. Common Solutions for Downgrade Migration
Solution 1: Conversion to AOF Migration, Pseudocode as Follows
let kvs = parse(RDB)
for (key, val) in kvs
if val is map
for (field, value) in val
// execute migration
redis.HSET(key, field, vlaue)
if val is string
// execute migration
redis.SET(key, val)
...
...
Advantages of this solution:
- Simple implementation
- Can handle large key downgrade migration
Disadvantages of this solution:
- Uses non-compact AOF protocol for migration, consuming bandwidth
- Cannot guarantee atomicity of migration, needs to ensure either successful or failed migration
Solution 2: Conversion to Lower Version DUMP Format, Pseudocode as Follows
let kvs = parse(RDB)
for (key, val) in kvs
let dump-val = convertToDump(val)
let lower-dump-val = convertToLowerDump(dump-val)
// execute migration
redis.RESTORE(key, lower-dump-val)
Advantages of this solution:
- Achieves atomic migration (critical in most migration tools)
- Compact migration format, reducing bandwidth usage
Disadvantages of this solution:
- Complex implementation
- Requires ensuring sufficient memory in migration tool for large key migration and adjusting parameters of source and target Redis instances
Note: Both Redis-replicator and Redis-rdb-CLI adopt Solution 2 for migration.
3. How Redis-replicator Performs Downgrade Migration
Code Example:
// Downgrade migration from redis-7.0.0 to redis-6.0.0 Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); replicator.setRdbVisitor(new DumpRdbVisitor(replicator, 9)); replicator.addEventListener(new EventListener() { @Override public void onEvent(Replicator replicator, Event event) { if (event instanceof DumpKeyValuePair) { DumpKeyValuePair dkv = (DumpKeyValuePair) event; byte[] serialized = dkv.getValue(); // Use redis RESTORE command to migrate serialized data to target redis. } } }); replicator.open();
Explanation:
// Create a DumpRdbVisitor Rdb parser to parse rdbv10 format into rdbv9 format replicator.setRdbVisitor(new DumpRdbVisitor(replicator, 9));
// Register event listener, where serialized is the byte array downgraded to rdbv9 // It can be directly migrated to target database using the RESTORE command replicator.addEventListener(new EventListener() { @Override public void onEvent(Replicator replicator, Event event) { if (event instanceof DumpKeyValuePair) { DumpKeyValuePair dkv = (DumpKeyValuePair) event; byte[] serialized = dkv.getValue(); } } });
4. How Redis-rdb-CLI Performs Downgrade Migration
# Redis-rdb-CLI internally relies on Redis-replicator # The first step of migration requires changing the configuration file /path/to/redis-rdb-cli/conf/redis-rdb-cli.conf # Change dump_rdb_version from -1 to 9 $ sed -i 's/dump_rdb_version=-1/dump_rdb_version=9/g' /redis-rdb-cli/conf/redis-rdb-cli.conf # Execute migration command, data will be automatically downgraded to lower version DUMP format $ rmt -s redis://com.redis7:6379 -m redis://com.redis6:6379 -r
5. Known Issues and Limitations
Cannot downgrade migrate data types not supported by lower versions of Redis. For example, cannot migrate MODULE format to versions below Redis 4.0 or STREAM format to versions below Redis 5.0. No other limitations known.
Beta Was this translation helpful? Give feedback.
All reactions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment