python: mock a module

Question

Is it possible to mock a module in python using unittest.mock? I have a module named config, while running tests I want to mock it by another module test_config. how can I do that ? Thanks.

config.py:

CONF_VAR1 = "VAR1"
CONF_VAR2 = "VAR2"

test_config.py:

CONF_VAR1 = "test_VAR1"
CONF_VAR2 = "test_VAR2" 

All other modules read config variables from the config module. While running tests I want them to read config variables from test_config module instead.

Answer 1

If you're always accessing the variables in config.py like this:

import config
...
config.VAR1

You can replace the config module imported by whatever module you're actually trying to test. So, if you're testing a module called foo, and it imports and uses config, you can say:

from mock import patch
import foo
import config_test
....
with patch('foo.config', new=config_test):
 foo.whatever()

But this isn't actually replacing the module globally, it's only replacing it within the foo module's namespace. So you would need to patch it everywhere it's imported. It also wouldn't work if foo does this instead of import config:

from config import VAR1

You can also mess with sys.modules to do this:

import config_test
import sys
sys.modules["config"] = config_test
# import modules that uses "import config" here, and they'll actually get config_test

But generally it's not a good idea to mess with sys.modules, and I don't think this case is any different. I would favor all of the other suggestions made over it.

• Peter Gibson: Very nice, although when I test this I can't get it to mock foo.config before actually importing foo (as I can with config.var1 say). This means that any code executed when foo is imported will use the original config module. Any suggestions?
• dano: @PeterGibson Right, that's another limitation of this approach. I'm not aware of any way around it.
• and: @PeterGibson I don't think there is a way to patch code executed when foo is imported without messing with sys.modules. I've described it in my answer. But unless this import patching is necessary, I would stick to dano's approach.

Answer 2

foo.py:

import config
VAR1 = config.CONF_VAR1
def bar():
 return VAR1

test.py:

import unittest
import unittest.mock as mock
import test_config
class Test(unittest.TestCase):
 def test_one(self):
 with mock.patch.dict('sys.modules', config=test_config):
 import foo
 self.assertEqual(foo.bar(), 'test_VAR1')

As you can see, the patch works even for code executed during import foo.

• n611x007: what if your module is not called config but config.foo? Wrinting mock.patch.dict('sys.modules', config.foo=test_config) would not do...
• and: @naxa, you can pass a dictionary to this function patch.dict('sys.modules', {'config.foo': test_config}) (docs)
• kev: How would you patch a module for all tests?
• and: @kev, if you mean all tests in a TestCase, you should create a patcher with patch.dict() and start() it in setUp(), then stop() it in tearDown(). If you mean all tests in a project, this sounds like a good Question on it own. I suppose you could have a custom test runner with separate config (django seems to do something like this). But first of all I would consider changing design to avoid patching altogether.
• Heather Sawatsky: This doesn't seem to work if the module is already imported in the test module itself. For example, in my unit tests, I'm importing pyspark (a third-party library) to create objects for some unit tests. But then in another module, I'm optionally importing pyspark. I'd like to mock the pyspark import from the other module to raise ImportError, but can't because it's already imported at the top of my unit tests. Ideas?

Answer 3

If you want to mock an entire module just mock the import where the module is used.

myfile.py

import urllib

test_myfile.py

import mock
import unittest
class MyTest(unittest.TestCase):
 @mock.patch('myfile.urllib')
 def test_thing(self, urllib):
 urllib.whatever.return_value = 4

Answer 4

I recently ran into this same problem too. I need to test some magic code that modifies a modules __dir__ and __getattr__ function, so mocking using a class is not an option for me.

I then realized Django uses modules to configure settings, so in their tests, they must be "mocking" modules too. Here's how they did it:

from types import ModuleType
m = ModuleType("mymodule")
print(m) # <module 'mymodule'>
m.CONF_VAR1 = "conf1"
m.CONF_VAR2 = "conf2"

Django source code: https://github.com/django/django/blob/4.2.4/tests/settings_tests/tests.py#L366

Answer 5

Consider this following setup

configuration.py:

import os
class Config(object):
 CONF_VAR1 = "VAR1"
 CONF_VAR2 = "VAR2"
class TestConfig(object):
 CONF_VAR1 = "test_VAR1"
 CONF_VAR2 = "test_VAR2"
if os.getenv("TEST"):
 config = TestConfig
else:
 config = Config

now everywhere else in your code you can use:

from configuration import config
print config.CONF_VAR1, config.CONF_VAR2

And when you want to mock your coniguration file just set the environment variable "TEST".

Extra credit: If you have lots of configuration variables that are shared between your testing and non-testing code, then you can derive TestConfig from Config and simply overwrite the variables that need changing:

class Config(object):
 CONF_VAR1 = "VAR1"
 CONF_VAR2 = "VAR2"
 CONF_VAR3 = "VAR3"
class TestConfig(Config):
 CONF_VAR2 = "test_VAR2"
 # CONF_VAR1, CONF_VAR3 remain unchanged

Answer 6

If your application ("app.py" say) looks like

import config
print config.var1, config.var2

And gives the output:

$ python app.py
VAR1 VAR2

You can use mock.patch to patch the individual config variables:

from mock import patch
with patch('config.var1', 'test_VAR1'):
 import app

This results in:

$ python mockimport.py
test_VAR1 VAR2

Though I'm not sure if this is possible at the module level.