Using a thread to update a table automatically in Django

Question

For my portfolio app using Django I made a thread that pulls out the usd exchange rates for various currencies every 6 hours and updates the database. The program kicks in as soon as the module module_stock.py is being imported.

Other than to make the thread to start in a function call, should I be aware of any disadvantages if this runs in production environment for days, weeks on end? Or in other words is it ok to have this thread running at all time ?

Relevant pieces of code below.

module_stock.py:

import threading
UPDATE_INTERVAL = 21600
class WorldTradingData:
 ''' methods to handle trading data
 website: https://www.worldtradingdata.com
 '''
 @classmethod
 def setup(cls,):
 cls.api_token = config('API_token')
 cls.stock_url = 'https://api.worldtradingdata.com/api/v1/stock'
 cls.intraday_url = 'https://intraday.worldtradingdata.com/api/v1/intraday'
 cls.history_url = 'https://api.worldtradingdata.com/api/v1/history'
 cls.forex_url = 'https://api.worldtradingdata.com/api/v1/forex'
 @classmethod
 def update_currencies(cls):
 base_currency = 'USD'
 url = ''.join([cls.forex_url,
 '?base=' + base_currency,
 '&api_token=' + cls.api_token,
 ])
 try:
 res = requests.get(url)
 forex_dict = json.loads(res.content).get('data', {})
 except requests.exceptions.ConnectionError:
 forex_dict = {}
 logger.info(f'connection error: {url}')
 for cur in Currency.objects.all().order_by('currency'):
 currency_key = cur.currency
 currency_object = Currency.objects.get(currency=currency_key)
 usd_exchange_rate = forex_dict.get(currency_key, '')
 if usd_exchange_rate:
 currency_object.usd_exchange_rate = usd_exchange_rate
 currency_object.save()
def update_currencies_at_interval(interval=UPDATE_INTERVAL):
 ''' update the currency depending on interval in seconds
 default value is 6 hours (21600 seconds)
 '''
 assert isinstance(interval, int), f'check interval setting {interval} must be an integer'
 wtd = WorldTradingData()
 wtd.setup()
 start_time = time.time()
 current_time = start_time
 elapsed_time = int(current_time - start_time)
 while True:
 if elapsed_time % interval == 0:
 wtd.update_currencies()
 current_time = time.time()
 elapsed_time = int(current_time - start_time)
thread_update_currencies = threading.Thread(
 target=update_currencies_at_interval, kwargs={'interval': UPDATE_INTERVAL})
thread_update_currencies.start()

models.py:

from django.db import models
class Currency(models.Model):
 currency = models.CharField(max_length=3, unique=True)
 usd_exchange_rate = models.CharField(max_length=20, default='1.0')
 def __str__(self):
 return self.currency
```

Answer 1

Keep a base URL

'https://api.worldtradingdata.com/api/v1/' should be factored out of your URL variables. You can use the Python libraries in urllib to safely construct the URLs after, which is preferable to using join as you do.

Don't manually construct a URL

 url = ''.join([cls.forex_url,
 '?base=' + base_currency,
 '&api_token=' + cls.api_token,
 ])

This is more work than you have to do. Just pass a dict to requests.get's params kwarg.

Don't call json.loads()

 res = requests.get(url)
 forex_dict = json.loads(res.content).get('data', {})

Just call res.json().

Check your HTTP result

Call res.raise_for_status(), or at the absolute least check the status of the result. Currently, there are some failure modes that will not throw from your code at all.

• Bruno Vermeulen: Thanks for the suggestions, which I have now implemented. Any ideas if and how I should implement the thread to update the currencies every six hours...
• Reinderien: Probably the answer is to not do it in Python, and to have a cron job.
• Bruno Vermeulen: @Reiderien, I have changed the update now using a cron job, updating the Currency table directly with sql.
• Reinderien: @BrunoVermeulen Feel free to post that in a new question if you want another review :)
• Chris: You could use Celery and Beat. This way you could still use the Django ORM, reducing a bit of duplication. (You could also use sleep in the above loop, so that the CPU isn't spinning the whole time, but I wouldn't on anything more than a proof of concept.)