This project is abandoned.
A Simple MySQL to MongoDB Migration Tool
NOTE: not "montgolfier" but "mongolfier". ;)
$ npm install mongolfier
$ node ./node_modules/.bin/mongolfier --help
$ npm install -g mongolfier
$ mongolfier --help
To copy all rows from users table on test mysql database
into users collection on test mongo db.
$ mongolfier -s mysql://root@localhost/test -o mongodb://localhost/test -b users
NOTE use -d flag to test WITHOUT real insert operations:
$ mongolfier -s mysql://root@localhost/test -o mongodb://localhost/test -b -d users
NOTE use -e flag to remove all existing documents in the collection:
$ mongolfier -s mysql://root@localhost/test -o mongodb://localhost/test -b -e users
For complex migration,
you need to write a config file and template files and use -c option:
$ mongolfier -c config.json
$ mongolfier -help
Usage: mongolfier [OPTIONS] [COLLECTION_NAME ...]
Options:
-h, --help output usage information
-V, --version output the version number
-c, --config [FILE] path to config file(js or json)
-s, --mysql [URL] mysql connection url
-o, --mongo [URL] mongodb connection url
-e, --empty-collection make empty collection before migration
-b, --bulk-copy do bulk copy if no template available
-d, --dry-run do not insert into collection
- NOTE: no comment allowed here. this file should conforms to strict json format.
- NOTE:
{{and}}is just a marker. replace it with yours.
{ "mysql": "mysql://user:password@host:port/database", "mongo": "mongodb://user:password@host:port/db", "context": { "{{custom_attr_key}}": "{{custom_attr_value}}", ... }, "before": [ "{{before_script.js}}", ... ], "after": [ "{{after_script.js}}", ... ] "collections": [ { "collection": "{{mongo_collection_name}}", "template": "{{path_to_collection_template}}", "query": "{{mysql_select_query}}" }, { "collection": "{{mongo_collection_name}}", "template": "{{path_to_collection_template}}", "query": [ "{{mysql_select_query_part1}}", "{{mysql_select_query_part2}}", ... ] }, ... ] }
contextis optional. this could be accessed via$variable across processing all collections.beforeis optional. these scripts are evalulated before processing the first collection.afteris optional. these scripts are evalulated after processing the last collection.templateis optional(default:{{collection name}}.json).queryis string or array. array will be joined to a query.
NOTE: comment allowed here. this file is javascript or something. ;)
NOTE:
{{and}}is just a marker. replace it with yours.
({ // use mongodb ObjectID "_id": new mongo.ObjectID(), // use mysql column values "{{mongo_field_name}}": ${{mysql_column_name}}, ... })
var id = new mongo.ObjectID(); ... ({ "_id": id, "foo": ($bar).toUpperCase(), "sum": $num1 + $num2, "now": new Date(), ... })
- NOTE on the enclosing braces the result. do not use
returnkeyword.
var d = q.defer(); var id = new mongo.ObjectID(); setTimeout(function () { ... d.resolve({ "_id": id, "foo": ($bar).toUpperCase(), "sum": $num1 + $num2, "now": new Date(), ... }) }, 100); d.promise;
- NOTE on the last line
d.promise;. do not usereturnkeyword.
$ROW- the current mysql row as object.$COLLECTION- the current mongo collection as object.$MYSQL- the active mysql connection.$MONGO- the active mongo connection.$CONTEXT- a shared object across all mappings.mongo- mongo modulemysql- mysql moduleconsole- console objectutil- util modulefs- fs modulepath- path moduleQ- q module_- lodash module.- and so on...
TBD... ;)
That's all folks.
May the SOURCE be with you...