Custom serialiser with object factory

Question

A couple of years ago I wrote a serialisation library for my project. By the time it worked well, but recently I thought it could be modernised. Back then I needed something that looks similar to Boost's serialiser, but also support polymorphism. I need to serialise and deserialise different types of objects from the same inheriting tree. However I didn't wanted to include Boost library just for this. Also I needed to support different sources besides FILE* (I only included just one here.)

I came across with this solution, tweaked a bit, wrote a couple of tests around it, and been using. It's spread across multiple files in my repo, and I do have a couple more tests for it using GTest.

My overall questions are

• How can I use std some more to make it more compatible with its containers? I think about supporting std::map first
• What other possible way are there to improve the way I define and store factories of the different objects?
• The most important thing is for me is support more efficiently the serialisation of the parents of the class instances. At this point I only have a terrible solution that is I define a non-virtual _Serialize() method that takes care of it via override. Is there a solution to collect all the parents serialiser method more seamlessly and invoke them?

Here's the source code, that I had concatenated together and modified a bit to work single handed. I used clang on MacOS via Cmake to test it.

/*
 Dynamics API
*/
#ifndef __DYNAMICS_H__
#define __DYNAMICS_H__
#include <string>
#include <map>
namespace Grafkit
{
class Clonables;
/**
 */
class Clonable
{
 friend class Clonables;
public:
 virtual ~Clonable() = default;
 virtual Clonable *CreateObj() const = 0;
};
/**
 */
class Clonables
{
 std::map<std::string, const Clonable *> m_clonables;
 Clonables() {}
 Clonables(const Clonables &) = delete;
 Clonables &operator=(const Clonables &) = delete;
 virtual ~Clonables()
 {
 for (auto it = m_clonables.begin(); it != m_clonables.end(); ++it)
 {
 const Clonable *clone = it->second;
 delete clone;
 }
 m_clonables.clear();
 }
public:
 static Clonables &Instance()
 {
 static Clonables instance;
 return instance;
 }
 void AddClonable(const char *CLASS_NAME, Clonable *clone)
 {
 std::string name = CLASS_NAME;
 auto it = m_clonables.find(name);
 if (it == m_clonables.end())
 {
 m_clonables[name] = clone;
 }
 }
 const Clonable *Find(const char *CLASS_NAME)
 {
 std::string name = CLASS_NAME;
 auto it = m_clonables.find(name);
 if (it == m_clonables.end())
 return nullptr;
 const Clonable *clone = it->second;
 return clone;
 }
 Clonable *Create(const char *CLASS_NAME)
 {
 const Clonable *clone = Find(CLASS_NAME);
 if (clone)
 return clone->CreateObj();
 return nullptr;
 }
};
class AddClonable
{
public:
 AddClonable(const char *CLASS_NAME, Clonable *clone)
 {
 Clonables::Instance().AddClonable(CLASS_NAME, clone);
 }
};
} // namespace Grafkit
#define CLONEABLE_DECL(CLASS_NAME) \
public: \
 virtual Grafkit::Clonable *CreateObj() const override \
 { \
 return new CLASS_NAME(); \
 }
#define CLONEABLE_FACTORY_DECL(CLASS_NAME) \
 CLONEABLE_DECL(CLASS_NAME) \
public: \
 class Factory : public Grafkit::Clonable \
 { \
 public: \
 virtual Grafkit::Clonable *CreateObj() const \
 { \
 return new CLASS_NAME(); \
 } \
 }; \
 \
private: \
 static Grafkit::AddClonable _addClonableFactory;
#define CLONEABLE_FACTORY_IMPL(CLASS_NAME) \
 Grafkit::AddClonable CLASS_NAME::_addClonableFactory(#CLASS_NAME, new CLASS_NAME::Factory());
#define CLONEABLE_FACTORY_LOCAL_IMPL(CLASS_NAME) \
 Grafkit::AddClonable CLASS_NAME##_addClonableFactory(#CLASS_NAME, new CLASS_NAME::Factory());
#endif //__DYNAMICS_H__
/*
Persistence API
*/
#ifndef __PERSISTENCE_H__
#define __PERSISTENCE_H__
#include <cstdlib>
#include <vector>
// Alrady included above
//#include "dynamics.h"
namespace Grafkit
{
class Archive;
class Persistent : public Clonable
{
public:
 Persistent() {}
 virtual ~Persistent() {}
 static Persistent *Load(Archive &ar);
 template <class C>
 static C *LoadT(Archive &ar) { return dynamic_cast<C *>(Load(ar)); }
 void Store(Archive &ar);
protected:
 virtual void Serialize(Archive &ar) = 0;
 virtual std::string GetClazzName() const = 0;
 virtual int GetVersion() const { return 0; }
};
class Archive
{
public:
 explicit Archive(int IsStoring = 0);
 virtual ~Archive();
 /* IO API */
protected:
 virtual void Write(const void *buffer, size_t length) = 0;
 virtual void Read(void *buffer, size_t length) = 0;
 size_t WriteString(const char *input);
 size_t ReadString(char *&output);
public:
 template <typename T>
 void PersistField(T &t)
 {
 if (m_isStoring)
 {
 Write(&t, sizeof(T));
 }
 else
 {
 Read(&t, sizeof(T));
 }
 }
 template <typename T>
 void PersistVector(T *&v, uint32_t &count)
 {
 if (m_isStoring)
 {
 Write(&count, sizeof(count));
 Write(v, sizeof(T) * count);
 }
 else
 {
 uint32_t readCount = 0;
 Read(&readCount, sizeof(readCount));
 void *p = malloc(sizeof(T) * readCount);
 Read(p, sizeof(T) * readCount);
 v = reinterpret_cast<T *>(p);
 count = readCount;
 }
 }
 template <typename T>
 void PersistStdVector(std::vector<T> &v)
 {
 if (m_isStoring)
 {
 uint32_t u32count = v.size(); //clamp it down to 32 bit
 Write(&u32count, sizeof(u32count));
 void *p = v.data();
 Write(p, sizeof(T) * u32count);
 }
 else
 {
 uint32_t count = 0;
 Read(&count, sizeof(count));
 void *p = malloc(sizeof(T) * count);
 Read(p, sizeof(T) * count);
 v.clear();
 v.assign(static_cast<T *>(p), static_cast<T *>(p) + count);
 }
 }
 void PersistString(const char *&str);
 void PersistString(std::string &str);
 void StoreObject(Persistent *object);
 Persistent *LoadObject();
 int IsStoring() const { return m_isStoring; }
 void SetDirection(bool IsStoring) { m_isStoring = IsStoring; }
private:
 int m_isStoring;
};
} // namespace Grafkit
// ------------------
#define PERSISTENT_DECL(CLASS_NAME, VERSION_NO) \
public: \
 CLONEABLE_FACTORY_DECL(CLASS_NAME) \
public: \
 std::string GetClazzName() const override \
 { \
 return #CLASS_NAME; \
 } \
 int GetVersion() const override \
 { \
 return VERSION_NO; \
 }
#define PERSISTENT_IMPL(CLASS_NAME) \
 CLONEABLE_FACTORY_IMPL(CLASS_NAME)
// ---
#define PERSIST_FIELD(AR, FIELD) (AR.PersistField<decltype(FIELD)>((FIELD)))
#define PERSIST_VECTOR(AR, VECTOR, COUNT) (AR.PersistVector<std::remove_pointer<decltype(VECTOR)>::type>(VECTOR, COUNT))
#define PERSIST_STD_VECTOR(AR, VECTOR) (AR.PersistStdVector<decltype(VECTOR)::value_type>(VECTOR))
#define PERSIST_STRING(AR, FIELD) (AR.PersistString((FIELD)))
#define _PERSIST_OBJECT(AR, TYPE, IN_FIELD, OUT_FIELD) \
 { \
 if (AR.IsStoring()) \
 { \
 AR.StoreObject(dynamic_cast<Persistent *>((IN_FIELD))); \
 } \
 else \
 { \
 (OUT_FIELD) = dynamic_cast<TYPE>(AR.LoadObject()); \
 } \
 }
#define PERSIST_OBJECT(AR, OBJECT) _PERSIST_OBJECT(AR, decltype(OBJECT), OBJECT, OBJECT)
#define PERSIST_REFOBJECT(AR, REF) _PERSIST_OBJECT(AR, decltype(REF.Get()), REF.Get(), REF)
#endif //__PERSISTENCE_H__
/*
 Archive IO layer
*/
#ifndef __ARCHIVE_H__
#define __ARCHIVE_H__
// #include "persistence.h"
namespace Grafkit
{
class ArchiveFile : public Archive
{
public:
 ArchiveFile(FILE *stream, bool IsStoring = false);
 virtual ~ArchiveFile();
 void Write(const void *buffer, size_t length) override;
 void Read(void *buffer, size_t length) override;
private:
 FILE *_stream;
};
}; // namespace Grafkit
#endif //__ARCHIVE_H__
/***************************************************
 Impl
****************************************************/
/*
 Persistent layer impl: persistent.cpp
*/
#include <vector>
#include <cassert>
// Already pasted above
//#include "persistence.h"
//#include "dynamics.h"
using namespace Grafkit;
using namespace std;
void Persistent::Store(Archive &ar)
{
 string CLASS_NAME = this->GetClazzName();
 uint8_t ver = this->GetVersion();
 PERSIST_STRING(ar, CLASS_NAME);
 PERSIST_FIELD(ar, ver);
 this->Serialize(ar);
}
Persistent *Persistent::Load(Archive &ar)
{
 string CLASS_NAME;
 uint8_t ver = 0;
 PERSIST_STRING(ar, CLASS_NAME);
 Clonable *clone = Clonables::Instance().Create(CLASS_NAME.c_str());
 assert(clone);
 Persistent *obj = dynamic_cast<Persistent *>(clone);
 assert(obj);
 PERSIST_FIELD(ar, ver);
 assert(ver == obj->GetVersion());
 obj->Serialize(ar);
 return obj;
}
/**
 Archive
*/
Archive::Archive(int IsStoring) : m_isStoring(IsStoring)
{
}
Archive::~Archive()
{
}
size_t Archive::WriteString(const char *input)
{
 uint16_t slen = strlen(input);
 this->Write(&slen, sizeof(slen));
 this->Write(input, slen + 1);
 return slen;
}
size_t Archive::ReadString(char *&output)
{
 uint16_t slen = 0;
 this->Read(&slen, sizeof(slen));
 output = new char[slen + 1];
 this->Read(output, slen + 1);
 return slen;
}
void Archive::PersistString(const char *&str)
{
 if (m_isStoring)
 {
 WriteString(str);
 }
 else
 {
 char *in = nullptr;
 ReadString(in);
 str = in;
 }
}
void Archive::PersistString(string &str)
{
 if (m_isStoring)
 {
 WriteString(str.c_str());
 }
 else
 {
 char *in = nullptr;
 ReadString(in);
 str = in;
 delete[] in;
 }
}
void Archive::StoreObject(Persistent *object)
{
 uint8_t isNotNull = object != nullptr;
 PersistField(isNotNull);
 if (isNotNull)
 object->Store(*this);
}
Persistent *Archive::LoadObject()
{
 uint8_t isNotNull = 0;
 PersistField(isNotNull);
 if (isNotNull)
 return Persistent::Load(*this);
 return nullptr;
}
/*
 Archive IO impl: archive.cpp
*/
// Already included above
// #include "archive.h"
#include <cassert>
using namespace Grafkit;
ArchiveFile::ArchiveFile(FILE *stream, bool IsStoring) : Archive(IsStoring),
 _stream(stream)
{
 assert(_stream);
}
ArchiveFile::~ArchiveFile()
{
}
void ArchiveFile::Write(const void *buffer, size_t length)
{
 assert(_stream);
 fwrite(buffer, length, 1, this->_stream);
}
void ArchiveFile::Read(void *buffer, size_t length)
{
 assert(_stream);
 fread(buffer, length, 1, this->_stream);
}
/**
 Test classes, Main
*/
class SimpleClass : public Grafkit::Persistent
{
public:
 SimpleClass() : Persistent(), m_i(0)
 {
 }
 void SetParams(const int i, const std::string str)
 {
 m_i = i;
 m_str = str;
 }
 int GetI() const { return m_i; }
 std::string GetStr() const { return m_str; }
 PERSISTENT_DECL(SimpleClass, 1);
protected:
 void Serialize(Archive &ar) override
 {
 PERSIST_FIELD(ar, m_i);
 PERSIST_STRING(ar, m_str);
 }
private:
 int m_i;
 std::string m_str;
};
class SimpleBaseClass : public Grafkit::Persistent
{
public:
 SimpleBaseClass() : Persistent(), m_i(0) {}
 void SetParams(const int i, const std::string str)
 {
 m_i = i;
 m_str = str;
 }
 int GetI() const { return m_i; }
 std::string GetStr() const { return m_str; }
 virtual std::string GetSomeIntern() const = 0;
protected:
 void _Serialize(Archive &ar)
 {
 PERSIST_FIELD(ar, m_i);
 PERSIST_STRING(ar, m_str);
 }
private:
 int m_i;
 std::string m_str;
};
class DerivedClassA : public SimpleBaseClass
{
public:
 DerivedClassA() : SimpleBaseClass(), m_str1("This is derived class A") {}
 virtual std::string GetSomeIntern() const override { return m_str1; }
 PERSISTENT_DECL(DerivedClassA, 1);
protected:
 void Serialize(Archive &ar) override
 {
 SimpleBaseClass::_Serialize(ar);
 PERSIST_STRING(ar, m_str1);
 }
private:
 std::string m_str1;
};
class DerivedClassB : public SimpleBaseClass
{
public:
 DerivedClassB() : SimpleBaseClass(), m_str2("This is derived class B") {}
 virtual std::string GetSomeIntern() const override { return m_str2; }
 PERSISTENT_DECL(DerivedClassB, 1);
protected:
 void Serialize(Archive &ar) override
 {
 SimpleBaseClass::_Serialize(ar);
 PERSIST_STRING(ar, m_str2);
 }
private:
 std::string m_str2;
};
#include <cstdio>
PERSISTENT_IMPL(SimpleClass);
PERSISTENT_IMPL(DerivedClassA);
PERSISTENT_IMPL(DerivedClassB);
int main()
{
 FILE *fp;
 // -- 1. simple class
 // given
 SimpleClass *simpleObj = new SimpleClass();
 simpleObj->SetParams(42, "Hello Serializer");
 SimpleBaseClass *inheritedA = new DerivedClassA();
 SimpleBaseClass *inheritedB = new DerivedClassB();
 inheritedA->SetParams(314, "Hello Serializer w/ inherited classes");
 inheritedA->SetParams(216, "Hello Serializer GLEJD");
 // when
 fp = fopen("archive.bin", "wb'");
 assert(fp);
 ArchiveFile arWrite(fp, true);
 simpleObj->Store(arWrite);
 inheritedA->Store(arWrite);
 inheritedB->Store(arWrite);
 fflush(fp);
 fclose(fp);
 fp = nullptr;
 // then
 fp = fopen("archive.bin", "rb");
 assert(fp);
 ArchiveFile arRead(fp, false);
 SimpleClass *loadedObj = Persistent::LoadT<SimpleClass>(arRead);
 SimpleBaseClass *loadedA = Persistent::LoadT<SimpleBaseClass>(arRead);
 SimpleBaseClass *loadedB = Persistent::LoadT<SimpleBaseClass>(arRead);
 assert(loadedObj);
 assert(simpleObj->GetI() == loadedObj->GetI());
 assert(simpleObj->GetStr().compare(loadedObj->GetStr()) == 0);
 assert(loadedA);
 assert(dynamic_cast<DerivedClassA *>(loadedA));
 assert(inheritedA->GetI() == loadedA->GetI());
 assert(inheritedA->GetStr().compare(loadedA->GetStr()) == 0);
 assert(inheritedA->GetSomeIntern().compare(loadedA->GetSomeIntern()) == 0); 
 assert(loadedB);
 assert(dynamic_cast<DerivedClassB *>(loadedB));
 assert(inheritedB->GetI() == loadedB->GetI());
 assert(inheritedB->GetStr().compare(loadedB->GetStr()) == 0);
 assert(inheritedB->GetSomeIntern().compare(loadedB->GetSomeIntern()) == 0); 
 fclose(fp);
 delete simpleObj;
 delete inheritedA;
 delete inheritedB;
 delete loadedObj;
 delete loadedA;
 delete loadedB;
 return 0;
}

• user52292: What is the purpose of Cloneables? It is singleton map wrapper, but creates objects through Clonable::CreateObj which does not appear to do cloning. It would make sense in Factory which I guess is there for de-serialization to create empty/default objects before filling them, but you should really describe how is it supposed to work.

Answer 1

Here are some suggestions.

• __DYNAMICS_H__ is a reserved identifier because it contains consecutive underscores. Use a non-reserved identifier like DYNAMICS_H_. Also, this name is too common and may introduce name clashes. One solution is to append a sequence of random characters.

• I see no reason to make Clonables a friend of Clonable. The latter doesn't have anything private.

• Your Clonables::m_clonables owns the objects, so it should use std::unique_ptr instead of plain pointers. This way, you don't need to care about the copy operations, move operations, or destructors. The default constructor is also redundant. Also, I don't see why you make the destructor of Cloneables private.

• Instance is effectively a global variable. Global variables are generally discouraged in C++. They should belong to the place where they are used.

• AddClonable should take a std::string_view or, in C++14, take a std::string by value and move it into the operation. ALL-CAPS names are usually used for macros. And clone should be const Clonable*. First performing a find and then do [] is wasteful. Use emplace:

  void AddClonable(std::string name, const Clonable* clone)
  {
   m_clonables.emplace(std::move(name), clone);
  }
  

• In Find, the variable clone isn't really necessary. The extra copy is avoidable in C++14 by using a template:

  template <typename T>
  const Clonable* Find(const T& name)
  {
   auto it = m_clonables.find(name);
   if (it == m_clonables.end())
   return nullptr;
   else
   return it->second;
  }
  

  Create is similar.

• The AddClonable class feels questionable from a design perspective.

These should be enough to get you started.

• Caiwan: Thanks, Sine then I've improved this one a lot. - I use #pragma once for guarding - Clonables had been merged into Persistence - Properties / behavior of Clonable had been merged into Serializable - Instead of storing prototpye objects, I use std::function as a virtual factory to reduce footprint and side effects - I also added this to implement the rest of the serialization - The complete single-header soluiton is here

Answer 2

I started reading.

IMO Find should be private and const.

Once you have done this you can replace the map with a map of std::string, unique_ptr.

Doing so you get rid of the destructor of Clonables.

Does this make sense?

• user52292: It would make sense if you elaborated on object ownership (and the lack of it in the code provided) and provide alternative solution using unique_ptr. Find could still be useful, but should not return raw pointer (describe why). Solution is up to you, but it should probably take inspiration in map::find as Clonables is essentialy singleton wrapper of a map.