Accessing the contents of a project's root directory in Python

Question

This morning I was trying to find a good way of using os.path to load the content of a text file into memory, which exists in the root directory of my current project.

This approach strikes me as a bit hackneyed, but after some thought it was exactly what I was about to do, except with os.path.normpath(os.path.join(__file__, '..', '..'))

These relative operations on the path to navigate to a static root directory are brittle.

Now, if my project layout changes, these associations to the (former) root directory change with it. I wish I could assign the path to find a target, or a destination, instead of following a sequence of navigation operations.

I was thinking of making a special __root__.py file. Does this look familiar to anyone, or know of a better implementation?

my_project
|
| __init__.py
| README.md
| license.txt
| __root__.py
| special_resource.txt
| ./module
 |-- load.py

Here is how it is implemented:

"""__root__.py"""
import os
def path():
 return os.path.dirname(__file__)

And here is how it could be used:

"""load.py"""
import __root__
def resource():
 return open(os.path.join(__root__.path(), 'special_resource.txt'))

Answer 1

I never heard of __root__.py and don't think that is a good idea.

Instead create a files.py in a utils module:

MAIN_DIRECTORY = dirname(dirname(__file__))
def get_full_path(*path):
 return join(MAIN_DIRECTORY, *path)

You can then import this function from your main.py:

from utils.files import get_full_path
path_to_map = get_full_path('res', 'map.png')

So my project directory tree looks like this:

project_main_folder/
 utils/
 __init__.py (empty)
 files.py
 main.py

When you import a module the Python interpreter searches if there's a built-in module with that name (that's why your module name should be different or you should use relative imports). If it hasn't found a module with that name it will (among others in sys.path) search in the directory containing your script. You can find further information in the documentation.

• raylu: But how do you import main without knowing where the main directory is already?
• Joschua: I modified my answer to include the import line and also put the function in the module utils.files.
• raylu: My question remains the same: how do you import utils.files without knowing where utils is?
• Joschua: When you put the code that imports utils.files in main.py Python will automatically figure it out. I added further information to my answer.
• raylu: Oh. What if I'm in dir1/module1.py and I want to import dir2.module2?

Answer 2

your approach seems ok, but as it is not a common problem/solution, you should document what your __root__.py file is doing.

One thing: you should not use double leading and trailing underscores, see pep8

double_leading_and_trailing_underscore: "magic" objects or attributes that live in user-controlled namespaces. E.g. __init__, __import__ or __file__. Never invent such names; only use them as documented.

Answer 3

A standard way to achieve this would be to use the pkg_resources module which is part of the setuptools module. setuptools is used to create an installable python package.

You can use pkg_resources to return the contents of your desired file as a string and you can use pkg_resources to get the actual path of the desired file on your system.

Let's say that you have a module called stackoverflow.

stackoverflow/
|-- app
| `-- __init__.py
`-- resources
 |-- bands
 | |-- Dream\ Theater
 | |-- __init__.py
 | |-- King's\ X
 | |-- Megadeth
 | `-- Rush
 `-- __init__.py
3 directories, 7 files

Now let's say that you want to access the file Rush. Use pkg_resources.resouces_filename to get the path to Rush and pkg_resources.resource_string to get the contents of Rush, thusly:

import pkg_resources
if __name__ == "__main__":
 print pkg_resources.resource_filename('resources.bands', 'Rush')
 print pkg_resources.resource_string('resources.bands', 'Rush')

The output:

/home/sri/workspace/stackoverflow/resources/bands/Rush
Base: Geddy Lee
Vocals: Geddy Lee
Guitar: Alex Lifeson
Drums: Neil Peart

This works for all packages in your python path. So if you want to know where lxml.etree exists on your system:

import pkg_resources
if __name__ == "__main__":
 print pkg_resources.resource_filename('lxml', 'etree')

output:

/usr/lib64/python2.7/site-packages/lxml/etree

The point is that you can use this standard method to access files that are installed on your system (e.g pip install xxx) and files that are within the module that you're currently working on.