How to handle db password change in a live service

Question

Suppose I have a REST application running on a remote server which connects to a mysql database. Now let's say the password of the db has to be changed. What are the right methods and practices that has to be followed to avoid failure of requests.

One solution is to create a new user with same level of permissions as old and deploy. Once we ensure all the instances are connecting with the same username, we can delete the old username/password combo.

Is there a better solution than this?

Answer 1

When you are changing the password, with the main concern being maintaining access to the database, here are a few methods:

1. Operate on another database temporarily, if you are only doing reading then you can have a read fail-over
2. Use another user temporarily
3. Similar-- use a MySQL proxy user to have different credentials, but the same permissions
4. If you are using a Secrets Manager (Hashipcorp Vault, or AWS Secrets Manager), the vault will automatically, periodically rotate the password over time, and your program will automatically retrieve the new password each time. This would be a best practice and nice long-term, but takes a bit of work initially.

Happy coding!

• Philip Kendall: Aside: if you're already in AWS-land, you can use IAM to access your MySQL database which also takes away this problem as you can overlap keys when rotating them.