-
Notifications
You must be signed in to change notification settings - Fork 1.1k
.couch files becomes .couch.locked and unable to restore databases #4045
-
all the files become .locked and unable to restore the database. Got the following errors:
This is a single node server
[notice] 2022年06月01日T00:00:56.405884Z couchdb@127.0.0.1 <0.421.0> -------- chttpd_auth_cache changes listener died because the _users database does not exist. Create the database to silence this notice. [error] 2022年06月01日T00:00:56.405935Z couchdb@127.0.0.1 emulator -------- Error in process <0.5498.2815> on node 'couchdb@127.0.0.1' with exit value: {database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,400}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,375}]},{mem3_shards,load_s>
Beta Was this translation helpful? Give feedback.
All reactions
If you're not aware of any application you run which creates those files, there is a good chance it's from a ransomware virus https://fileinfo.com/extension/locked
You can check if the criminals left a note, like readme txt file of some sort, in the home or root directory, which would confirm that theory.
Replies: 11 comments 15 replies
-
CouchDB doesn't create .locked files. I am not sure why that is happening. Since you mentioned "restore", since CouchDB doesn't have a built-in backup/restore feature, perhaps it's the software used to restore database files?
Beta Was this translation helpful? Give feedback.
All reactions
-
Beta Was this translation helpful? Give feedback.
All reactions
-
If you're not aware of any application you run which creates those files, there is a good chance it's from a ransomware virus https://fileinfo.com/extension/locked
You can check if the criminals left a note, like readme txt file of some sort, in the home or root directory, which would confirm that theory.
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
-
Beta Was this translation helpful? Give feedback.
All reactions
-
my couchdb server also got affected by cerber ransomware last week. Do you actually able to download the decrypter after make the payment? Did you manage to restore your db?
Beta Was this translation helpful? Give feedback.
All reactions
-
Yes, I did.
Beta Was this translation helpful? Give feedback.
All reactions
-
@ozgursel you can make sure that the shard files in the _dbs db match up with the shard map. https://docs.couchdb.com/en/3.2.2/cluster/index.html
You can make a new database named differently, (say newdb1) with the same Q sharding factor and see what the _dbs shard doc looks like. Then re-create the clustered db doc for your db to match.
Beta Was this translation helpful? Give feedback.
All reactions
-
I saw but didnt try yet. Is there any way the get docs from single .couch file? @nickva
Beta Was this translation helpful? Give feedback.
All reactions
-
Are you using cluster setup?
Beta Was this translation helpful? Give feedback.
All reactions
-
It doesn't matter actually.
I figured out to restore DB, thanks to @nickva.
@paragasu but you have to decrypt your files first then I will guide you to restore if you want.
and desperately I can say decryptor comes with a unique key I guess, anyway I send you from mail
Beta Was this translation helpful? Give feedback.
All reactions
-
@ozgursel you can read the docs from either the :5986 localhost port (a node-local port) or from the :5984/_node/_local/$shardname paths
Beta Was this translation helpful? Give feedback.
All reactions
-
🎉 1
-
@nickva Is it a possible 0-day vulnerability on CouchDB?
Beta Was this translation helpful? Give feedback.
All reactions
-
@ozgursel It's possible. There was a recent vulnerability fix for CouchDB https://docs.couchdb.org/en/3.2.2/cve/2022-24706.html make sure to upgrade your instances. It could also be another vulnerability which is used for the attack so it's hard say definitely.
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
-
As a general (and obvious) note you should not pay ransomware authors but restore from a backup on to a clean, secured replacement server. obviously that is not always an option.
If you have recovered your shard files (shards/000-FFF/blah123.couch, etc) you can recreate the _dbs database manually. The format is not documented but it's fairly simple, an example is below. shard_suffix is an erlang string of the .12345 part of the shard filenames (a list of integers where the ints are ascii values).
If you uncover any evidence to suggest your served was compromised via couchdb we are very interested to hear any and all details.
{
"_id": "dbname",
"shard_suffix": [
46,
49,
54,
53,
52,
54,
51,
53,
48,
54,
56
],
"by_node": {
"node1@127.0.0.1": [
"00000000-7fffffff",
"80000000-ffffffff"
],
"node2@127.0.0.1": [
"00000000-7fffffff",
"80000000-ffffffff"
],
"node3@127.0.0.1": [
"00000000-7fffffff",
"80000000-ffffffff"
]
},
"by_range": {
"00000000-7fffffff": [
"node1@127.0.0.1",
"node2@127.0.0.1",
"node3@127.0.0.1"
],
"80000000-ffffffff": [
"node1@127.0.0.1",
"node2@127.0.0.1",
"node3@127.0.0.1"
]
}
}
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
-
then put this json using
curl -X PUT http://username:password@yourhost:yourport/_node/_local/_dbs/yourdbname -d '{theonelinejson}'
command
Beta Was this translation helpful? Give feedback.
All reactions
-
Hi @nickva can explain more on how to come up with the shard_suffix value?
Beta Was this translation helpful? Give feedback.
All reactions
-
It would be great if @ahmedsaifreza, @paragasu and @ozgursel could tell us which version of CouchDB they are each running, so we can eliminate previously patched security vulnerabilities from consideration.
Beta Was this translation helpful? Give feedback.
All reactions
-
I am running couchdb 2.3.1
Beta Was this translation helpful? Give feedback.
All reactions
-
thank you.
Beta Was this translation helpful? Give feedback.
All reactions
-
I am actually running couchdb 3.1.1
Beta Was this translation helpful? Give feedback.
All reactions
-
oh and Operating System and version would help too.
Beta Was this translation helpful? Give feedback.
All reactions
-
Debian 10
Beta Was this translation helpful? Give feedback.
All reactions
-
thanks
Beta Was this translation helpful? Give feedback.
All reactions
-
Beta Was this translation helpful? Give feedback.
All reactions
-
Beta Was this translation helpful? Give feedback.
All reactions
-
sounds like it, yes. You'll need to rebuild the server from trusted sources, it has clearly been compromised and you can no longer trust anything on it to be what it appears.
In addition to using the latest version of CouchDB's installer, you should install a firewall (iptables based is easiest) and only allow inbound connections to port 5984 (or 6984 if you are using TLS, which we recommend).
Beta Was this translation helpful? Give feedback.