Using ORMs in two separate programs which share a DB

Question

I have an application that is divided into two main parts:

1. A web crawler written in Python
2. A REST API written in Golang

They share a MySQL database, which is mainly populated/updated by the web crawler, and then read from by the REST API.

I wanted to use an ORM on each side to make interacting with the database easier, but now I am not sure if this is a good approach.

---

On the Python side, I was looking at using peewee, and for Golang I will use gorm.

---

It doesn't seem like a good idea to maintain two sets of ORM model files, where I would have to change each set whenever I want to change some attributes of the database. Additionally, I am worried that with two ORMs battling for structure of the database, that conflicts might arise, or the models might become out of sync with what the actual db structure is.

---

Is this a scenario where using an ORM will cause more trouble than its worth? Or maybe I should use a single ORM (for either Python or Golang) and write raw queries for the other side.

• Sjoerd Job Postmus: Just a quick question: can't you write two generators which generate the ORM model files from a shared description? That way you still have 1 representative definition of your model. For bonus points, let it also generate the SQL schema definition (or, if both your libraries support it, let both of them generate the definition, and assert that they generate the same definition)
• Jeroen Vannevel: If you change the DB schema you'll always have to change the clients, ORM or not.
• TylerAndFriends: @JeroenVannevel Usually I would change the DB schema via an ORM, but yes you're correct.
• gbjbaanb: write all your queries using stored procedures - ten you can change your DB schema and keep a fixed API.

Answer 1

I think it is okay to have two sets of ORM if you think it will make your life easier in terms of build the applications. Think of it this way without the ORM as well you will need to change the queries in both applications if you make some database schema changes etc. So I won't count that to be a huge issue. As long you as can gain productivity by using ORM on both applications it should be okay.

Answer 2

Use both, and write a Python script that'll import the peewee models and convert them to GORM models.

I've scratched something very basic. I've created models.py with the example in peewee's README.md:

from peewee import *
import datetime
db = SqliteDatabase('my_database.db', threadlocals=True)
class BaseModel(Model):
 class Meta:
 database = db
class User(BaseModel):
 username = CharField(unique=True)
class Tweet(BaseModel):
 user = ForeignKeyField(User, related_name='tweets')
 message = TextField()
 created_date = DateTimeField(default=datetime.datetime.now)
 is_published = BooleanField(default=True)

And here is a simple script that imports it and create Go code:

import peewee
def enumerate_models():
 import models
 for name in dir(models):
 item = getattr(models, name)
 if isinstance(item, type) and issubclass(item, models.BaseModel) and item != models.BaseModel:
 yield item
def model_fields(model):
 for name in dir(model):
 item = getattr(model, name)
 if isinstance(item, peewee.Field):
 yield name, item
def field_go_type(field_spec):
 return {
 peewee.PrimaryKeyField: 'uint',
 peewee.ForeignKeyField: 'uint',
 peewee.TextField: 'string',
 peewee.CharField: 'string',
 peewee.DateTimeField: 'time.Time',
 peewee.BooleanField: 'bool',
 }.get(type(field_spec), '?%s?' % type(field_spec))
for model in enumerate_models():
 print('type' ,model.__name__, 'struct {')
 for field_name, field_spec in model_fields(model):
 field_go_decleration = []
 field_go_decleration.append(field_name)
 field_go_decleration.append(field_go_type(field_spec))
 print('\t' + ' '.join(field_go_decleration))
 print('}')

It's output:

type Tweet struct {
 created_date time.Time
 id uint
 is_published bool
 message string
 user uint
}
type User struct {
 id uint
 username string
}

My script still needs more work to add the annotations, but you get the basic idea.