I have developed NoSQL database.
At the moment it has no threads (async).
When the database needs to save data to disk, it blocks connections until it saves.
This method works OK, because save is fast - 3-4 sec, but I still can improve by background save.
Here is the prototype of the background save facility.
I have the following questions:
Do I need
atomic
? I have only one thread, can I use justthread::joinable()
instead?Anything else I am missing?
Option 1, use atomic_bool
:
#include <atomic>
#include <thread>
#include <chrono>
#include <vector>
#include <iostream>
struct X{
std::vector<int> rw;
std::vector<int> ro;
mutable
std::atomic<bool> saveInProgress{ false };
std::thread saveThread;
~X(){
if (saveThread.joinable()){
std::cout << "d-tor wait for join()" << '\n';
saveThread.join();
}
std::cout << "d-tor save in foreground..." << '\n';
if (!std::empty(rw)){
using std::swap;
swap(rw, ro);
saveProcess_();
}
}
// this is not multi threaded code
void save(){
if (saveInProgress){
std::cout << "save wait for join()" << '\n';
saveThread.join();
}
using std::swap;
swap(rw, ro);
saveInProgress = true;
saveThread = std::thread{ [this](){ saveProcess_(); } };
// this runs parallel with the thread now...
rw.clear();
}
private:
void saveProcess_() const{
std::cout << "Save process started " << std::this_thread::get_id() << '\n';
// saving vector...
for(auto &x : ro){
using namespace std::chrono_literals;
std::this_thread::sleep_for(100ms);
std::cout << "Save " << std::this_thread::get_id() << " " << x << '\n';
}
std::cout << "Save process done " << std::this_thread::get_id() << '\n';
saveInProgress = false;
}
};
int main(){
X x;
x.rw = { 10, 11, 12, 13, 14, 15, 16, 17, 18, 19 };
x.save();
x.rw = { 20, 21, 22, 23, 24, 25, 26, 27, 28, 29 };
x.save();
x.rw = { 30, 31, 32, 33, 34, 35, 36, 37, 38, 39 };
// d-tor will kick now...
}
Option 2, no atomic, just joinable()
:
#include <thread>
#include <chrono>
#include <vector>
#include <iostream>
struct X{
using Container = std::vector<int>;
Container rw;
Container ro;
std::thread saveThread;
~X(){
std::cout << "d-tor save in foreground..." << '\n';
saveProcess__(rw);
joinSaveProcess_("d-tor");
}
// this is not multi threaded code
void save(){
joinSaveProcess_("save");
using std::swap;
swap(rw, ro);
saveThread = std::thread{ [this](){ saveProcess__(ro); } };
// this runs parallel with the thread now...
rw.clear();
}
private:
void joinSaveProcess_(const char *who){
if (saveThread.joinable()){
std::cout << who << " wait for join()" << '\n';
saveThread.join();
}
}
static void saveProcess__(Container const &data){
auto id = std::this_thread::get_id();
std::cout << "Save process started " << id << '\n';
// saving vector...
for(auto &x : data){
using namespace std::chrono_literals;
std::this_thread::sleep_for(100ms);
std::cout << " - " << id << " " << x << '\n';
}
std::cout << "Save process done " << id << '\n';
}
};
int main(){
if(1){
X x;
// d-tor will kick now...
}
std::cout << "------------------------------------" << '\n';
if(1){
X x;
x.rw = { 10, 11, 12, 13, 14, 15, 16, 17, 18, 19 };
// d-tor will kick now...
}
std::cout << "------------------------------------" << '\n';
if(1){
X x;
x.rw = { 10, 11, 12, 13, 14, 15, 16, 17, 18, 19 };
x.save();
// d-tor will kick now...
}
std::cout << "------------------------------------" << '\n';
if(1){
X x;
x.rw = { 10, 11, 12, 13, 14, 15, 16, 17, 18, 19 };
x.save();
x.rw = { 20, 21, 22, 23, 24, 25, 26, 27, 28, 29 };
x.save();
x.rw = { 30, 31, 32, 33, 34, 35, 36, 37, 38, 39 };
// d-tor will kick now...
}
}
1 Answer 1
Do I need atomic? I have only one thread, can I use just thread::joinable() instead?
If you only use an instance of struct X
from one thread, then you could indeed use thread::joinable()
to check whether the save thread is running. However, as soon as you have multiple threads accessing an X
, then you need atomics, otherwise two threads can simultaneously check thread::joinable()
, both get false
, and then both will start a new saveThread
simultaneously.
The atomic flag doesn't help you with thread safety in any way. It only ensures atomic modification of the flag itself.
Anything else I am missing?
I can't tell, because you've just shown a prototype, and you haven't said what properties (like ACID) your database has. Most serious databases use some form of a transaction log, which is appended to for every operation on the database, they don't just rely on periodic saving of the data.