homepage

backtrace (relevant outer frames only):
 File "/path/to/myfile", line 1198, in get_hook_name
 for frame_record in inspect.stack():
 File "/usr/mbench2.2/lib/python2.4/inspect.py", line 819, in stack
 return getouterframes(sys._getframe(1), context)
 File "/usr/mbench2.2/lib/python2.4/inspect.py", line 800, in getouterframes
 framelist.append((frame,) + getframeinfo(frame, context))
 File "/usr/mbench2.2/lib/python2.4/inspect.py", line 775, in getframeinfo
 lines, lnum = findsource(frame)
 File "/usr/mbench2.2/lib/python2.4/inspect.py", line 437, in findsource
 if pat.match(lines[lnum]): break
IndexError: list index out of range
Based on a quick look at the inspect code, I think
this happens when you:
 - Start python and load a module
 - While it's running, edit the source file for the
 module (before inspect tries to look into it).
 - Call a routine in the edited module that will
 lead to a call to inspect.stack().
During an inspect.stack() call, inspect will open
source files to get the source code for the
routines on the stack. If the source file that is
opened doesn't match the byte compiled code that's
being run, there are problems. Inspect caches the
files it reads (using the linecache module), so if
the file gets cached before it is edited, nothing
should go wrong. But if the source file is edited
after the module is loaded and before inspect has
a chance to cache the source, you're out of luck.
Of course, this shouldn't be a problem in production
code, but it has bit us more than once in a
development environment.
Seems like it would be easy to avoid by just comparing
the timestamps on the source/object files. If the
source file is newer, just behave the same as if it
wasn't there.
Attached is a stupid little python script that
reproduces the problem.

Could you possibly work up a patch demonstrating exactly what you're proposing with the "comparing the timestamps" solution? That seems like a lot of complication for such a rare issue, but I'd be interested in seeing a patch all the same.

What I had in mind was something like changing:
 def getsourcefile(object):
 """Return the Python source file an object was defined in, if it exists."""
 filename = getfile(object)
 if string.lower(filename[-4:]) in ['.pyc', '.pyo']:
 filename = filename[:-4] + '.py'
 # ...the rest of the function...
to
 def getsourcefile(object):
 """Return the Python source file an object was defined in, if it exists."""
 filename = getfile(object)
 if string.lower(filename[-4:]) in ['.pyc', '.pyo']:
 src_filename = filename[:-4] + '.py'
 try:
 # if src file has been edited since we loaded the obj file, ignore it
 if os.stat(filename).st_mtime > os.stat(src_filename).st_mtime:
 filename = src_filename
 except OSError:
 pass
 # ...the rest of the function...
But now that I've tried to implement it, and thought about it a bit more, I see some
of the issues with what I thought would be a simple fix:
- I originally thought that, if a module is loaded from source (either because the
objfile did not exist, or because objfile was older), and the compiled src was successfully
written to a new objfile, that inspect.getfile() would return the path to the objfile.
I see now that that is not the case; it returns the srcfile. That makes my fix
a bit more difficult -- now you have to find out if there is an obj file, and if
so, what it's called.
And even if you had a handle on both the srcfile and the objfile:
- Even if the srcfile's timestamp is newer than the objfile's, it doesn't necessarily
mean that the srcfile has been edited since it was loaded. It could be that the srcfile
was already newer than the objfile before the module was loaded, so the module was
loaded from the srcfile, and the objfile was not updated because it was not writable by
the user.
- Even if the srcfile's timestamp is older than the objfile's, it doesn't necessarily
mean that the srcfile represents what you have loaded in memory. It could be that
after you loaded the module, your colleague down the hall edited the source, loaded
the module, and in the process was successful in updating the objfile. (Not a likely
scenario, but...)
- If the module was loaded from source, there was no objfile, and an objfile could not
be created because the directory was not writable, the fix doesn't help at all.
To be able to fix this right, you'd have to know what the timestamp was on whatever file
was loaded at the time that import loaded it. I guess import itself would have to stash
this away somewhere.

"""
To be able to fix this right, you'd have to know what the timestamp was on
whatever file
was loaded at the time that import loaded it. I guess import itself would
have to stash
this away somewhere.
"""
While I don't see this change being made, you might talk to Brett Cannon about it. He's working on a rewrite of the import machinery in Python and probably knows that system better than most by this point. Unless he/you can come up with something, though, I'm inclined to close the bug because of its provocation difficulty.

This issue appears implicated in https://bitbucket.org/pypa/setuptools/issue/201