homepage

Please run attached zip archive as a python script. Note that the problem is not specific for using __main__.py - any module that comes from a zip archive or loaded by a custom loader would show the same bug. 
$ unzip -l test.zip 
Archive: test.zip
 Length Date Time Name
 -------- ---- ---- ----
 79 10-24-08 18:26 __main__.py
 -------- -------
 79 1 file
$ $ cat __main__.py
from trace import Trace
def traced(flag):
 if flag:
 return 1
 else:
 return 2
tracer = Trace()
tracer.runfunc(traced, False)
results = tracer.results()
results.write_results(coverdir='.')
$ python testtrace.zip 
 --- modulename: __main__, funcname: traced
Not printing coverage data for 'testtrace.zip/__main__.py': [Errno 20] Not a directory: 'testtrace.zip/__main__.py'
__main__.py(4): __main__.py(7): $

I commited Alexander's fix for #10329: r86303. The trace module now uses the input Python script encoding, instead of the locale encoding.

I am attaching a "proof of concept" patch. The trace code is in a dire need of restructuring to eliminate repeated reading of source files.

+ try:
+ with open(filename, 'rb') as fp:
+ encoding, _ = tokenize.detect_encoding(fp.readline)
+ except IOError:
+ encoding = None
You should use 'utf-8' instead of None (which will fall back to the locale encoding) here.

On Sun, Nov 7, 2010 at 8:47 PM, STINNER Victor <report@bugs.python.org> wrote:
>
> STINNER Victor <victor.stinner@haypocalc.com> added the comment:
>
> +      try:
> +        with open(filename, 'rb') as fp:
> +          encoding, _ = tokenize.detect_encoding(fp.readline)
> +      except IOError:
> +        encoding = None
>
> You should use 'utf-8' instead of None (which will fall back to the locale encoding) here.
>
I know. I was too lazy to look up the correct spelling for the proof
of concept. I am really posting this patch to show how this bug can
be fixed in theory and discuss how much of refactoring is acceptable.
For example, do we need to preserve trace.find_strings() function?
What is the best way to pass around source code? - file-like objects,
line iterators, readline-like function? Also compile(prog, filename,
"exec") to find module's bytecode is a hack. There must be a standard
way to achieve that which would use a .pyc file if available.

The handling of __loader__ looks fine to me, although I don't quite see the point of the get_source call as it isn't returned or used.

On Sun, Nov 7, 2010 at 8:59 PM, Brett Cannon <report@bugs.python.org> wrote:
> .. I don't quite see the point of the get_source call as it isn't returned or used.
It is passed to find_docstrings() to produce the "strs" dictionary.

>What is the best way to pass around source code?
> - file-like objects, line iterators, readline-like function?
Line iterator (list of lines) as returned by open().readlines.
Memory should not be an issue. Read disk once and close.
with open('file', ...) as f:
 src = f.readlines()

On Fri, Nov 12, 2010 at 12:54 PM, Terry J. Reedy <report@bugs.python.org> wrote:
..
>>What is the best way to pass around source code?
>> - file-like objects, line iterators, readline-like function?
>
> Line iterator (list of lines) as returned by open().readlines.
> Memory should not be an issue. Read disk once and close.
>
> with open('file', ...) as f:
> src = f.readlines()
Line iterator does not carry the information about source encoding
which may be important for annotating the source code.

>Line iterator does not carry the information about source encoding
which may be important for annotating the source code.
I would pass around both encoding and lines, possibly as a tuple.
A person heavily into OO might define a _Source class and turn the private functions into methods. But that would not play well with the deprecation process until it is done with.

I updated the patch for 3.5.
[Brett]
> I don't quite see the point of the get_source call as it isn't returned or used.
It *is* used: it is passed to _find_strings_stream:
+ source = loader.get_source(modulename)
+ strs = _find_strings_stream(io.StringIO(source))

Hi, I left a couple of comments on Rietveld.