Relative imports for the billionth time

Question

I've been here:

• PEP 328 – Imports: Multi-Line and Absolute/Relative
• Modules, Packages
• Python packages: relative imports
• Python relative import example code does not work
• Relative imports in Python 2.5
• Relative imports in Python
• Python: Disabling relative import

and plenty of URLs that I did not copy, some on SO, some on other sites, back when I thought I'd have the solution quickly.

The forever-recurring question is this: how do I solve this "Attempted relative import in non-package" message?

ImportError: attempted relative import with no known parent package

I built an exact replica of the package on pep-0328:

package/
 __init__.py
 subpackage1/
 __init__.py
 moduleX.py
 moduleY.py
 subpackage2/
 __init__.py
 moduleZ.py
 moduleA.py

The imports were done from the console.

I did make functions named spam and eggs in their appropriate modules. Naturally, it didn't work. The answer is apparently in the 4th URL I listed, but it's all alumni to me. There was this response on one of the URLs I visited:

Relative imports use a module's name attribute to determine that module's position in the package hierarchy. If the module's name does not contain any package information (e.g. it is set to 'main') then relative imports are resolved as if the module were a top level module, regardless of where the module is actually located on the file system.

The above response looks promising, but it's all hieroglyphs to me. How do I make Python not return to me "Attempted relative import in non-package"? It has an answer that involves -m, supposedly.

Why does Python give that error message? What does by "non-package" mean? Why and how do you define a 'package'?

• BrenBarn: See my answer. You still haven't fully clarified what you're doing, but if you're trying to do from .something import something in the interactive interpreter, that won't work. Relative imports can only be used within modules, not interactively.
• Ani Menon: Related question: Python3 correct way to import relative or absolute?
• adam.hendry: @CodeJockey see stackoverflow.com/questions/63145047/… for context.
• Joe Flack: There's more to this than meets the eye. Same project, same setup on my team. Python 3.8 anaconda one one; relative imports work. Python 3.9 on mine; doesn't work. I've been on several projects now where relative imports worked on one person's machine but not the other. Running same command from the same directory.
• KiriSakow: See this workaround for a relative local import of a Python file from another project: stackoverflow.com/questions/44070953/…

Answer 1

Script vs. Module

Here's an explanation. The short version is that there is a big difference between directly running a Python file, and importing that file from somewhere else. Just knowing what directory a file is in does not determine what package Python thinks it is in. That depends, additionally, on how you load the file into Python (by running or by importing).

There are two ways to load a Python file: as the top-level script, or as a module. A file is loaded as the top-level script if you execute it directly, for instance by typing python myfile.py on the command line. It is loaded as a module when an import statement is encountered inside some other file. There can only be one top-level script at a time; the top-level script is the Python file you ran to start things off.

Naming

When a file is loaded, it is given a name (which is stored in its __name__ attribute). If it was loaded as the top-level script, its name is __main__. If it was loaded as a module, its name is the filename, preceded by the names of any packages/subpackages of which it is a part, separated by dots.

So for instance in your example:

package/
 __init__.py
 subpackage1/
 __init__.py
 moduleX.py
 moduleA.py

if you imported moduleX (note: imported, not directly executed), its name would be package.subpackage1.moduleX. If you imported moduleA, its name would be package.moduleA. However, if you directly run moduleX from the command line, its name will instead be __main__, and if you directly run moduleA from the command line, its name will be __main__. When a module is run as the top-level script, it loses its normal name and its name is instead __main__.

Accessing a module NOT through its containing package

There is an additional wrinkle: the module's name depends on whether it was imported "directly" from the directory it is in or imported via a package. This only makes a difference if you run Python in a directory, and try to import a file in that same directory (or a subdirectory of it). For instance, if you start the Python interpreter in the directory package/subpackage1 and then do import moduleX, the name of moduleX will just be moduleX, and not package.subpackage1.moduleX. This is because Python adds the current directory to its search path when the interpreter is entered interactively; if it finds the to-be-imported module in the current directory, it will not know that that directory is part of a package, and the package information will not become part of the module's name.

A special case is if you run the interpreter interactively (e.g., just type python and start entering Python code on the fly). In this case, the name of that interactive session is __main__.

Now here is the crucial thing for your error message: if a module's name has no dots, it is not considered to be part of a package. It doesn't matter where the file actually is on disk. All that matters is what its name is, and its name depends on how you loaded it.

Now look at the quote you included in your question:

Relative imports use a module's name attribute to determine that module's position in the package hierarchy. If the module's name does not contain any package information (e.g. it is set to 'main') then relative imports are resolved as if the module were a top-level module, regardless of where the module is actually located on the file system.

Relative imports...

Relative imports use the module's name to determine where it is in a package. When you use a relative import like from .. import foo, the dots indicate to step up some number of levels in the package hierarchy. For instance, if your current module's name is package.subpackage1.moduleX, then ..moduleA would mean package.moduleA. For a from .. import to work, the module's name must have at least as many dots as there are in the import statement.

... are only relative in a package

However, if your module's name is __main__, it is not considered to be in a package. Its name has no dots, and therefore you cannot use from .. import statements inside it. If you try to do so, you will get the "relative-import in non-package" error.

Scripts can't import relative

What you probably did is you tried to run moduleX or the like from the command line. When you did this, its name was set to __main__, which means that relative imports within it will fail, because its name does not reveal that it is in a package. Note that this will also happen if you run Python from the same directory where a module is, and then try to import that module, because, as described above, Python will find the module in the current directory "too early" without realizing it is part of a package.

Also remember that when you run the interactive interpreter, the "name" of that interactive session is always __main__. Thus you cannot do relative imports directly from an interactive session. Relative imports are only for use within module files.

Two solutions:

1. If you really do want to run moduleX directly, but you still want it to be considered part of a package, you can do python -m package.subpackage1.moduleX. The -m tells Python to load it as a module, not as the top-level script.

2. Or perhaps you don't actually want to run moduleX, you just want to run some other script, say myfile.py, that uses functions inside moduleX. If that is the case, put myfile.py somewhere else – not inside the package directory – and run it. If inside myfile.py you do things like from package.moduleA import spam, it will work fine.

Notes

• For either of these solutions, the package directory (package in your example) must be accessible from the Python module search path (sys.path). If it is not, you will not be able to use anything in the package reliably at all.

• Since Python 2.6, the module's "name" for package-resolution purposes is determined not just by its __name__ attributes but also by the __package__ attribute. That's why I'm avoiding using the explicit symbol __name__ to refer to the module's "name". Since Python 2.6 a module's "name" is effectively __package__ + '.' + __name__, or just __name__ if __package__ is None.)

• Michael Scott Asato Cuthbert: See python.org/dev/peps/pep-0366 -- "Note that this boilerplate is sufficient only if the top level package is already accessible via sys.path . Additional code that manipulates sys.path would be needed in order for direct execution to work without the top level package already being importable." -- this is the most disturbing bit to me since this "additional code" is actually quite long and can't be stored elsewhere in package to run easily.
• balu: I keep coming back to this post despite being a Python veteran. The main message for me is: Either fiddle around with sys.path and __package__ (which is rather ugly, see the other answers) or simply create a "main script" main.py in the root directory of your project and put all modules to be imported in subdirectories. main.py can then access all modules directly through their package names (= the names of the respective folders they're in).
• user2357112: This answer is currently off on a few important details regarding __name__ and sys.path. Specifically, with python -m pkg.mod, __name__ is set to __main__, not pkg.mod; relative imports are resolved using __package__ rather than __name__ in this case. Also, Python adds the script's directory rather than the current directory to sys.path when running python path/to/script.py; it adds the current directory to sys.path when running most other ways, including python -m pkg.mod.
• adamF: Finally understand after hours of reading... Worth noting that code under if __name__ == '__main__' will still run when using -m. See the comment from @user2357112
• C.J.: A feature is broken if you have to spend an hour reading this, and if you have to google it time after time, and still only the gurus get it. The python designers dropped the ball on this one, and it should just work, the first time, like any other programming language.

Answer 2

This is really a problem within python. The origin of confusion is that people mistakenly take the relative import as path relative which is not.

For example when you write in faa.py:

from .. import foo

This has a meaning only if faa.py was identified and loaded by python, during execution, as a part of a package. In that case, the module's name for faa.py would be for example some_packagename.faa. If the file was loaded just because it is in the current directory, when python is run, then its name would not refer to any package and eventually relative import would fail.

A simple solution to refer modules in the current directory, is to use this:

if __package__ is None or __package__ == '':
 # uses current directory visibility
 import foo
else:
 # uses current package visibility
 from . import foo

• Giacomo Alzetta: The correct solution is from __future__ import absolute_import and force the user to use your code correctly... so that you can always do from . import foo
• Smit Johnth: If you use IDE (and the only IDE for python is Pycharm), it manages imports by itself, and you have to manually copy them every time. Not good solution. And if foo ist an installed package, it will be used instead of your local file.
• Joooeey: @SmitJohnth Another IDE for Python is Spyder. I bet there are more.
• Smit Johnth: @Joooeey no, it's "IDE". The only IDE without quotes I know is Pycharm.
• resurrected user: This adds an import line for every context the code might be run from.

Answer 3

There are too many long answers in a foreign language. So, I'll try to make it short.

If you write from . import module, opposite to what you think, module will not be imported from current directory, but from the top level of your package! If you run .py file as a script, it simply doesn't know where the top level is and thus refuses to work.

If you start it like this py -m package.module from the directory above package, then Python knows where the top level is. That's very similar to Java: java -cp bin_directory package.class

• Levi Lesches: This is in @BrenBarn's answer, but it's the TL;DR of it. OP and anyone else looking for answers, this is it. Took me forever to find this elsewhere.
• Spartan: What is even more confusing is that when you install a package, absolute imports don't work for me. i need to use from .submodule import module. When i use import submodule.module or from submodule import module, it can't be found, even when the folder is right in the package folder.
• Smit Johnth: @LeviLesches I think we need TL;DRs. English is not my native language, and I don't have time to read walls of text which can be lossless compressed to a paragraph. Are people doing this paid for it? Is it chinese answers, like chinese code when they were paid for code lines?
• Blckknght: Part of this answer is wrong. from . import module doesn't import from "the top level of the package", it imports from the same package as the module the import statement is written in, which may be some deep subpackage in a complicated package hierarchy.
• user2189731: This is the short version. The key is the ".", ".." is nothing to do with directory of the .py file that contains it, while it's relative to the script.py in "python script.py". That's too confusing a feature.

Answer 4

So after carping about this along with many others, I came across a note posted by Dorian B in this article that solved the specific problem I was having where I would develop modules and classes for use with a web service, but I also want to be able to test them as I'm coding, using the debugger facilities in PyCharm. To run tests in a self-contained class, I would include the following at the end of my class file:

if __name__ == '__main__':
 # run test code here...

but if I wanted to import other classes or modules in the same folder, I would then have to change all my import statements from relative notation to local references (i.e. remove the dot (.)) But after reading Dorian's suggestion, I tried his 'one-liner' and it worked! I can now test in PyCharm and leave my test code in place when I use the class in another class under test, or when I use it in my web service!

# import any site-lib modules first, then...
import sys
parent_module = sys.modules['.'.join(__name__.split('.')[:-1]) or '__main__']
if __name__ == '__main__' or parent_module.__name__ == '__main__':
 from codex import Codex # these are in same folder as module under test!
 from dblogger import DbLogger
else:
 from .codex import Codex
 from .dblogger import DbLogger

The if statement checks to see if we're running this module as main or if it's being used in another module that's being tested as main. Perhaps this is obvious, but I offer this note here in case anyone else frustrated by the relative import issues above can make use of it.

• Brownbat: I have a similar issue -- tools that have to be packaged into the same folder to work as an add-on to another larger program. The main add-on interfaces with the larger program and works only when that larger program is running. For testing, I want to run the smaller utilities and let them call each other. It's a nightmare. I've started just using chained try/except ImportError blocks and adding every possible way of importing something in there. It works, it's short, but is so incredibly unpythonic it hurts every time.
• Brian Wylie: This is my exact use case, testing/debugging within PyCharm. The solution, for PyCharm users, is to setup one or more 'Source Roots'. From the PyCharm docs "PyCharm uses the source roots as the starting point for resolving imports." - jetbrains.com/help/pycharm/configuring-project-structure.html
• Smit Johnth: You said you use Pycharm. It manages imports by itself, and you have to manually copy them every time. Not good solution.
• WestCoastProjects: "Perhaps this is obvious" .. umm that code is obvious? I'm going to stash it away somewhere in any case - given I live and die by JetBrains tools..

Answer 5

Here is one solution that I would not recommend, but might be useful in some situations where modules were simply not generated:

import os
import sys
parent_dir_name = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
sys.path.append(parent_dir_name + "/your_dir")
import your_script
your_script.a_function()

• WurzelseppQX: Also check the answer of Lars who created a clean version of this answer that you can just copy/paste as boilerplate into all of your modules.
• ElpieKay: After reading so many answers, I think I'll just stick to this solution. One suggestion is to use os.path.join or other path libs instead of string concatenation when generating a path.

Answer 6

Here's a general recipe, modified to fit as an example, that I am using right now for dealing with Python libraries written as packages, that contain interdependent files, where I want to be able to test parts of them piecemeal. Let's call this lib.foo and say that it needs access to lib.fileA for functions f1 and f2, and lib.fileB for class Class3.

I have included a few print calls to help illustrate how this works. In practice you would want to remove them (and maybe also the from __future__ import print_function line).

This particular example is too simple to show when we really need to insert an entry into sys.path. (See Lars' answer for a case where we do need it, when we have two or more levels of package directories, and then we use os.path.dirname(os.path.dirname(__file__))—but it doesn't really hurt here either.) It's also safe enough to do this without the if _i in sys.path test. However, if each imported file inserts the same path—for instance, if both fileA and fileB want to import utilities from the package—this clutters up sys.path with the same path many times, so it's nice to have the if _i not in sys.path in the boilerplate.

from __future__ import print_function # only when showing how this works
if __package__:
 print('Package named {!r}; __name__ is {!r}'.format(__package__, __name__))
 from .fileA import f1, f2
 from .fileB import Class3
else:
 print('Not a package; __name__ is {!r}'.format(__name__))
 # these next steps should be used only with care and if needed
 # (remove the sys.path manipulation for simple cases!)
 import os, sys
 _i = os.path.dirname(os.path.abspath(__file__))
 if _i not in sys.path:
 print('inserting {!r} into sys.path'.format(_i))
 sys.path.insert(0, _i)
 else:
 print('{!r} is already in sys.path'.format(_i))
 del _i # clean up global name space
 from fileA import f1, f2
 from fileB import Class3
... all the code as usual ...
if __name__ == '__main__':
 import doctest, sys
 ret = doctest.testmod()
 sys.exit(0 if ret.failed == 0 else 1)

The idea here is this (and note that these all function the same across Python 2.7 and Python 3.x):

1. If run as import lib or from lib import foo as a regular package import from ordinary code, __package is lib and __name__ is lib.foo. We take the first code path, importing from .fileA, etc.

2. If run as python lib/foo.py, __package__ will be None and __name__ will be __main__.

   We take the second code path. The lib directory will already be in sys.path so there is no need to add it. We import from fileA, etc.

3. If run within the lib directory as python foo.py, the behavior is the same as for case 2.

4. If run within the lib directory as python -m foo, the behavior is similar to cases 2 and 3. However, the path to the lib directory is not in sys.path, so we add it before importing. The same applies if we run Python and then import foo.

   (Since . is in sys.path, we don't really need to add the absolute version of the path here. This is where a deeper package nesting structure, where we want to do from ..otherlib.fileC import ..., makes a difference. If you're not doing this, you can omit all the sys.path manipulation entirely.)

Notes

There is still a quirk. If you run this whole thing from outside:

python2 lib.foo

or:

python3 lib.foo

the behavior depends on the contents of lib/__init__.py. If that exists and is empty, all is well:

Package named 'lib'; __name__ is '__main__'

But if lib/__init__.py itself imports routine so that it can export routine.name directly as lib.name, you get:

python2 lib.foo

Output:

Package named 'lib'; __name__ is 'lib.foo'
Package named 'lib'; __name__ is '__main__'

That is, the module gets imported twice, once via the package and then again as __main__ so that it runs your main code. Python 3.6 and later warn about this:

python3 lib.routine

Output:

Package named 'lib'; __name__ is 'lib.foo'
[...]/runpy.py:125: RuntimeWarning: 'lib.foo' found in sys.modules
after import of package 'lib', but prior to execution of 'lib.foo';
this may result in unpredictable behaviour
 warn(RuntimeWarning(msg))
Package named 'lib'; __name__ is '__main__'

The warning is new, but the warned-about behavior is not. It is part of what some call the double import trap. (For additional details see issue 27487.) Nick Coghlan says:

This next trap exists in all current versions of Python, including 3.3, and can be summed up in the following general guideline: "Never add a package directory, or any directory inside a package, directly to the Python path".

Note that while we violate that rule here, we do it only when the file being loaded is not being loaded as part of a package, and our modification is specifically designed to allow us to access other files in that package. (And, as I noted, we probably shouldn't do this at all for single level packages.) If we wanted to be extra-clean, we might rewrite this as, e.g.:

 import os, sys
 _i = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 if _i not in sys.path:
 sys.path.insert(0, _i)
 else:
 _i = None
 from sub.fileA import f1, f2
 from sub.fileB import Class3
 if _i:
 sys.path.remove(_i)
 del _i

That is, we modify sys.path long enough to achieve our imports, then put it back the way it was (deleting one copy of _i if and only if we added one copy of _i).

Answer 7

Let me start this answer by validating your annoyance. Yes, it defies your initial intuitions and you are valid in feeling annoyed. It's confusing and you're not sure what to do. BrenBarn's excellent answer goes into more technical detail, but I got lost the first few times I tried to read it. This is my attempt to answer more directly and simply.

The solutions

There are two correct solutions to these top-level relative import errors. Both will work, but you should choose the one that fits your situation. This answer is to help you understand when to use each one.

1. If the folder really should be a package, then you should use -m from above it to force python to treat the whole folder as a "package".
2. If the folder should not be a package (e.g. also holds dev tools, configuration and tests), then put your source code in a single inner folder, swap out the relative imports for standard imports and use PYTHONPATH='.' or python -m from inside the folder.

Don't use sys.path.append at the top of your scripts. This makes them effectively broken if anything else ever imports that script file. Not super likely, but better to not risk it.

The reasons

The unbreakable rule is: Relative imports are only allowed within a package.

There is a good reason for this. Packages could be located anywhere. We don't want code to depend on how separate packages are arranged on disk. e.g. We should not accept from ..os import path in a file in math.

Most people don't have a problem with this, in concept. Instead, the problems I see people having is:

1. Knowing what a package is, exactly.
2. Knowing how to apply that knowledge to find the correct resolution.

Defining a package

Let's look at a few perspectives on what a "package" is. I promise they are all relevant.

1. Running python: When you start a script, the directory containing the script is added to sys.path (docs). Then, from the perspective of running python, a package is just any folder that lives inside something on sys.path. Importantly, this means that the status of "package" is unrelated to the contents of a folder. It is entirely decided at run-time by the relationship between the folder and the contents of sys.path.
2. Python ecosystem: When we think of what a "package" should be, we think of it as a set of distinct, installable code. In particular, we don't want to include dev tools or test folders in the distributed code. So, python projects on github will typically have an outer folder with those dev tools and test, and an inner source folder which contains the actual working code. This inner source code will either be named "src" or have the same name as the repo. That part is the actual "package" part of the repo. Examples: numpy, requests, pytest
3. Historically: Prior to Python 3.3, a "package" needed to have an __init__.py to be considered a valid package. Importantly, this was not the defining feature of a package, but at the time was a necessary part of it. These days, however, an empty __init__.py is usually unnecessary. See more.

Understanding the problem

Let's consider two scenarios you might encounter.

ImportError: attempted relative import with no known parent package

package/
 subpackage1/
 moduleX.py
 moduleA.py # from .subpackage1 import moduleX

With package/ as the cwd, we run python moduleA.py. This adds package/ to sys.path. A script is not a package, so relative imports aren't allowed here (see BrenBarn's answer).

ImportError: attempted relative import beyond top-level package

package/
 subpackage1/
 moduleX.py
 subpackage2/
 moduleZ.py # from ..subpackage1 import moduleX
 moduleA.py # import subpackage2.moduleZ

With package/ as the cwd, we run python moduleA.py. This adds package/ to sys.path. Again package/ is not a "package" according to python. This time, python will recognise "subpackage2" as a package.

But, since "subpackage2" is the package, trying to do a relative import with ".." is trying to jump up above the top-level of a package. i.e. not allowed.

Understanding the solutions

Finally, to understand the solutions presented at the top, let's see why you would use each.

1. Use cd .. && python -m package.moduleA.
   • You are saying "actually, I meant that the whole package/ was a package, not that subpackage1 and subpackage2 were separate packages".
   • Your "working" folder is above package/.
   • You are tying moduleA and subpackage1 together. They won't work if they are separated, and you're completely ok with that.
   • You are implying that the entire package/ folder should be distributed in order to use your code.
2. Modify subpackage2/moduleZ.py to be from subpackage1 import moduleX and run with PYTHONPATH='.' python subpackage2/moduleZ.py or python -m subpackage2.moduleZ.
   • Usually comes up when trying to run tests. e.g. subpackage1 is your source, and subpackage2 is your tests folder.
   • You are saying "I just wanted to import subpackage1.moduleX from something that doesn't live directly in package/, but I didn't want to do sys.path hacks because they're brittle."
   • You are simulating the case that subpackage1 is a package installed to someone else's site-packages.
   • You are telling python to consider the subpackage1 and subpackage2 folders to be entirely independent.

Why doesn't it just work anyway?

Couldn't python just treat the running directory as a package and allow relative imports anyway? Sure, it could, but this would mean that you hadn't properly decided what you wanted to do. So, rather than mask that ambiguity from you, it forces you to decide what you actually meant.

Is it a package or not? Hopefully now you know enough to answer that question.

• resurrected user: "...this would mean that you hadn't properly decided what you wanted to do." This ambiguity is an essential part of the development process. It may not be desired for a final library, but you cannot just brush it under the carpet stating it should not exist.
• Multihunter: I'm not sure I follow the complaint. Anything python does to enable that ambiguity would persist through to a final library, so it shouldn't do that. But, anyway, there's nothing stopping you from using either method while you are still unsure which applies and then changing it later. It would only be in the very early stages of project conception that such an ambiguity could exist, so changing it later should never be particularly onorous.

Answer 8

@BrenBarn's answer says it all, but if you're like me it might take a while to understand. Here's my case and how @BrenBarn's answer applies to it, perhaps it will help you.

The case

package/
 __init__.py
 subpackage1/
 __init__.py
 moduleX.py
 moduleA.py

Using our familiar example, and add to it that moduleX.py has a relative import to ..moduleA. Given that I tried writing a test script in the subpackage1 directory that imported moduleX, but then got the dreaded error described by the OP.

Solution

Move test script to the same level as package and import package.subpackage1.moduleX

Explanation

As explained, relative imports are made relative to the current name. When my test script imports moduleX from the same directory, then module name inside moduleX is moduleX. When it encounters a relative import the interpreter can't back up the package hierarchy because it's already at the top

When I import moduleX from above, then name inside moduleX is package.subpackage1.moduleX and the relative import can be found

• variable: Hoping you can guide me on this. In the following link, if you go to Case 3, it says solution 1 is not possible. Please can you check this and let me know. It will help me immensely. chrisyeh96.github.io/2017/08/08/…
• Brad Dre: @variable there's a typo in the link and I'm not allowed to edit. Looked at case 3 and didn't follow exactly what you're getting at. When I tried that example in python 2 there were no issues which makes me think I missed something. Maybe you should post a new question but need to provide a clearer example. Case 4 touches on what I'm talking about in my answer here: you can't go up a directory for relative import UNLESS the interpreter starts in a parent directory
• variable: Thanks I'm referring to python 3 and here the question stackoverflow.com/questions/58577767/…
• PythoNic: @BradDre typo fixed: chrisyeh96.github.io/2017/08/08/…

Answer 9

I had a similar problem where I didn't want to change the Python module search path and needed to load a module relatively from a script (in spite of "scripts can't import relative with all" as BrenBarn explained nicely).

So I used the following hack. Unfortunately, it relies on the imp module that became deprecated since version 3.4 to be dropped in favour of importlib. (Is this possible with importlib, too? I don't know.) Still, the hack works for now.

Example for accessing members of moduleX in subpackage1 from a script residing in the subpackage2 folder:

#!/usr/bin/env python3
import inspect
import imp
import os
def get_script_dir(follow_symlinks=True):
 """
 Return directory of code defining this very function.
 Should work from a module as well as from a script.
 """
 script_path = inspect.getabsfile(get_script_dir)
 if follow_symlinks:
 script_path = os.path.realpath(script_path)
 return os.path.dirname(script_path)
# loading the module (hack, relying on deprecated imp-module)
PARENT_PATH = os.path.dirname(get_script_dir())
(x_file, x_path, x_desc) = imp.find_module('moduleX', [PARENT_PATH+'/'+'subpackage1'])
module_x = imp.load_module('subpackage1.moduleX', x_file, x_path, x_desc)
# importing a function and a value
function = module_x.my_function
VALUE = module_x.MY_CONST

A cleaner approach seems to be to modify the sys.path used for loading modules as mentioned by Federico.

#!/usr/bin/env python3
if __name__ == '__main__' and __package__ is None:
 from os import sys, path
 # __file__ should be defined in this case
 PARENT_DIR = path.dirname(path.dirname(path.abspath(__file__)))
 sys.path.append(PARENT_DIR)
from subpackage1.moduleX import *

• Andrew Wagner: That looks better... too bad it still requires you to embed the name of the parent directory in the file... maybe that can be improved with importlib. Maybe importlib can even be monkeypatched to make relative import "just work" for simple use cases. I'll take a crack at it.
• ChumbiChubaGo: I'm using python 2.7.14 though. Would something like this still work ?
• Lars: I just tested both approaches on python 2.7.10 and they worked fine for me. If fact, you do not have the problem of a deprecated imp module in 2.7, so all the better.
• WurzelseppQX: Came back to this after years and just wanted to mention, that this answer, that last code snippet with the cleaner version, I am using in all my code for quite some time now. It is messy and you have this ugly boiler plate, but it is working like I would expect it to work in the first place. Currently I am using Python 3.11 and still this is not part of the normal behavior. Very sad. But this is really helping. Thank you @Lars.

Answer 10

Here is a dead simple solution in case you don't want to do any of these:

• add __init__.py files
• run with python -m mymodule
• edit __package__
• add if check in __main__
• edit sys.path by hand
• edit PYTHONPATH
• restructure the project

pip install importmonkey

This is a robust wrapper around sys.path hacks to keep everything simple and neat.
[github] [pip] [docs]

├─ src
│ └─ project
│ └─ mymodule.py
└─ test
 └─ test.py

# In test.py
from importmonkey import add_path
add_path("../src/project") # relative to current __file__
import mymodule
# add as many paths as needed, absolute or relative
# unix path conventions work so you can use '..' and '.'
# add_path validates the paths and returns added path as string

The module does not need to have __init__.py associated with it for this to work.

Disclosure of affiliation: I made importmonkey.

• Solomon Ucko: "if you mention your product, website, etc. in your question or answer (or any other contribution to the site), you must disclose your affiliation in your post" stackoverflow.com/help/promotion
• Markus Hirsimäki: @SolomonUcko thank you for the reminder. I will try to remember that. In this case it is my own creation.
• alelom: Seconding this solution. It works great, and it allows to avoid all the faff of Python's bad design when it comes to relative imports.

Answer 11

Following up on what Lars has suggested I've wrapped this approach in an experimental, new import library: ultraimport

It gives the programmer more control over imports and it allows file system based imports. Therefore, you can do relative imports from scripts. Parent package not necessary. ultraimports will always work, no matter how you run your code or what is your current working directory because ultraimport makes imports unambiguous. You don't need to change sys.path and also you don't need a try/except block to sometimes do relative imports and sometimes absolute.

You would then write in somefile.py something like:

import ultraimport
foo = ultraimport('__dir__/foo.py')

__dir__ is the directory of somefile.py, the caller of ultraimport(). foo.py would live in the same directory as somefile.py.

One caveat when importing scripts like this is if they contain further relative imports. ultraimport has a builtin preprocessor to rewrite subsequent relative imports to ultraimports so they continue to work. Though, this is currently somewhat limited as original Python imports are ambiguous and there's only so much you can do about it.

Answer 12

__name__ changes depending on whether the code in question is run in the global namespace or as part of an imported module.

If the code is not running in the global space, __name__ will be the name of the module. If it is running in global namespace -- for example, if you type it into a console, or run the module as a script using python.exe yourscriptnamehere.py then __name__ becomes "__main__".

You'll see a lot of python code with if __name__ == '__main__' is used to test whether the code is being run from the global namespace – that allows you to have a module that doubles as a script.

Did you try to do these imports from the console?

• theodox: Ah, so you mention -m. That makes your module execute as a script - if you stick an if __name__ == '__main__' in there you should see that it is '__main__' because of the -m. Try just importing your module into another module so it is not the top level... that should allow you to do the relative import
• user1881400: I tried to do these imports from the console, with the active file being the correct module.
• BrenBarn: @Stopforgettingmyaccounts...: What do you mean the "active file"?
• user1881400: I use Pyscripter. I was in moduleX.py when I ran these imports: from .moduleY import spam and from . import ModuleY.

Answer 13

Relative imports use a module's name attribute to determine that module's position in the package hierarchy. If the module's name does not contain any package information (e.g. it is set to 'main') then relative imports are resolved as if the module were a top level module, regardless of where the module is actually located on the file system.

I wrote a small Python package to PyPI that might help viewers of this question. The package acts as workaround if one wishes to be able to run Python files containing imports containing upper level packages from within a package / project without being directly in the importing file's directory.

Answer 14

For the poor souls like me which cannot make anything of this works, here is the solution posted by FEMista

Add an intermediate parent folder, to act as common branch for both siblings:

package/ 
 __init__.py
 SUBPACKAGES/
 __init__.py
 subpackage1/
 __init__.py
 moduleX.py
 moduleY.py
 subpackage2/
 __init__.py
 moduleZ.py
 moduleA.py

Remember to add the folder SUBPACKAGES to the import paths.

• Junglemath: How does this help? All you did was replace package with SUBPACKAGES.

Answer 15

I posted a solution on the other question but someone pointed me to this. So let me give an answer to the "how" part of the OP's question "Why and how do you define a 'package'?" However, to use my solution as it is at least Python 3.8 (released 14 Oct 2019) is required for the walrus operator.

The simple answer is that you can edit __package__ and add the folder containing the root package to sys.path. But how do you do this cleanly and not totally clutter up the top of the Python script? (As a demonstration that this dynamic editing works, I'm using import * but it's not recommended in general.)

Using a known top-level package

Suppose your code some_script.py resides somewhere within a directory structure which looks like:

project_folder
|-- subdir
 |-- ab
 |-- cd
 |-- some_script.py
 |-- script1.py
 |-- script2.py
 |-- script3.py
|-- script4.py
|-- other_folder
 |-- script5.py

And you need your package to be subdir.ab.cd without knowing ab or cd or even the number of nested levels (as long as none of the intermediate levels are called "subdir" as well). Then you could use the following:

import os
import sys
if not __package__:
 __package__ = ((lambda p, d:
 (".".join(p[-(n := p[::-1].index(d) + 1):]),
 sys.path.insert(0, os.sep.join(p[:-n])))[0])(
 os.path.realpath(__file__).split(os.sep)[:-1], "subdir"))
from .script1 import *
from ..script2 import *
from ...script3 import *
from subdir.script3 import *
from script4 import *
from other_folder.script5 import *

Using a known number of sub-levels

Suppose your code some_script.py resides somewhere within a directory structure which looks like:

project_folder
|-- ab
 |-- cd
 |-- some_script.py
 |-- script1.py
 |-- script2.py
|-- script3.py
|-- other_folder
 |-- script4.py

And you need your package to be ab.cd without knowing ab or cd but the depth of the package is guaranteed to be 2. Then you could use the following:

import os
import sys
if not __package__:
 __package__ = (
 (lambda p, n:
 (".".join(p[-n:]), sys.path.insert(0, os.sep.join(p[:-n])))[0])(
 os.path.realpath(__file__).split(os.sep)[:-1], 2))
from .script1 import *
from ..script2 import *
from script3 import *
from other_folder.script4 import *

Relative imports and absolute imports all work

With the sys.path including the project folder, you also of course do any absolute imports from there. With __package__ correctly computed, one can now do relative imports as well. A relative import of .other_script will look for other_script.py in the same folder as some_script.py. It is important to have one additional level in the package hierarchy as compared to the highest ancestor reached by the relative path, because all the packages traversed by the ".."/"..."/etc will need to be a Python package with a proper name.

Still works if further included in a bigger package

Even if it is included in a bigger package, as long as the original root folder remains a subfolder of the new root folder, this script and the functions in it can even be imported in the same way as a module into other modules in the bigger package. This is because the dynamic edits are only applied if __package__ is empty or None. Other scripts in the bigger package can of course have this piece of code to make them callable directly by Python.

P.S.: All code is done in a lambda just so that it would not pollute the global namespace with variables.

Answer 16

I will show off a folder structure that can do relative importing from the same folder, from a sub-folder and the parent folder. The file structure looks as follows:

.
├── main.py
└── bardir/
 ├── __init__.py
 ├── bar.py
 └── foodir/
 ├── __init__.py
 ├── foo.py
 ├── foo2.py
 └── bazdir/
 ├── __init__.py
 └── baz.py

This folder structure is chosen so foo.py is called first and displays foo, foo2, bar and baz in order. All while showing the three ways of relative importing.

Contents of main.py:

print('__name__:',__name__)
print('__package__:', __package__)
print('main')
from bardir.foodir import foo

Contents of foo.py:

print('\t\t__name__:',__name__)
print('\t\t__package__:', __package__)
print('\t\tfoo')
from . import foo2
from .. import bar
from . bazdir import baz

Every __init__ file is empty and the other files contain similar printing statements. Upon running main.py you get the following output:

__name__: __main__
__package__: None
main
 __name__: bardir.foodir.foo
 __package__: bardir.foodir
 foo
 __name__: bardir.foodir.foo2
 __package__: bardir.foodir
 foo2
 __name__: bardir.bar
 __package__: bardir
 bar
 __name__: bardir.foodir.bazdir.baz
 __package__: bardir.foodir.bazdir
 baz

You can use the following rules to reason about such file structures:

1. Relative importing is done relative to the package variable. A single dot means to look relative to __package__. Two dots means to look relative to the package before the first dot (from the right). Three dots means to look to the package before the second dot. The __package__ contains no dots? You can't use .. and you will get an error. Is your __package__ equal to None? No relative importing for you!
2. A folder becomes package as soon as there is an __init__.py file present. The init file may be empty.
3. If you run a script directly, i.e. press the run button in your IDE, the __name__ variable will be set to __main__ and the __package__ variable will be set to None. This means relative importing using dots will not work.

Answer 17

In most cases when I see the ValueError: attempted relative import beyond top-level package and pull my hair out, the solution is as follows:

You need to step one level higher in the file hierarchy!

#dir/package/module1/foo.py
#dir/package/module2/bar.py
from ..module1 import foo

Importing bar.py when interpreter is started in dir/package/ will result in error despite the import process never going beyond your current directory.

Importing bar.py when interpreter is started in dir/ will succeed.

Similarly for unit tests: python3 -m unittest discover --start-directory=. successfully works from dir/, but not from dir/package/.