This issue tracker has been migrated to GitHub ,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2012年05月03日 15:42 by vinay.sajip, last changed 2022年04月11日 14:57 by admin. This issue is now closed.
| Messages (2) | |||
|---|---|---|---|
| msg159864 - (view) | Author: Vinay Sajip (vinay.sajip) * (Python committer) | Date: 2012年05月03日 15:42 | |
I'm not sure if I've done something wrong, but I get an AssertionError when trying to run the tokenizer on a Python file from the Django source. The gist at https://gist.github.com/1977558 has the files concerned: 1. test_tokenize.py - the script which fails 2. tokenize_example.py - the file being tokenized and untokenized. This is from the Django source: django/extras/csrf_migration_helper.py 3. tokenizer.py - your tokenize module, I renamed it because I was working in /tmp and didn't want to import the Python 3.2 stdlib's tokenize.py 4. The test output shows that the tokenize_example module imports OK in Python 2.7.2, but running the test_tokenize script on it with Python3.2 fails with an AssertionError. I did some more testing, there are 131 failures in the Django source tree (all look like the same AssertionError). N.B. I posted this to your GitHub repo where you published the hook. |
|||
| msg174720 - (view) | Author: A.M. Kuchling (akuchling) * (Python committer) | Date: 2012年11月04日 00:03 | |
Duplicate of #9974. The tokenizer is choking on lines that end in '\'. So even this can't be parsed: a = 1 + \ 2 |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022年04月11日 14:57:29 | admin | set | github: 58918 |
| 2016年04月23日 20:47:04 | berker.peksag | set | status: open -> closed resolution: duplicate stage: resolved |
| 2014年12月31日 16:20:19 | akuchling | set | nosy:
- akuchling |
| 2012年11月04日 00:03:15 | akuchling | set | superseder: tokenizer.untokenize not invariant with line continuations messages: + msg174720 nosy: + akuchling |
| 2012年08月01日 22:25:44 | vinay.sajip | set | versions: + Python 3.3 |
| 2012年05月03日 15:44:07 | vinay.sajip | set | type: behavior |
| 2012年05月03日 15:42:20 | vinay.sajip | create | |