This issue tracker has been migrated to GitHub ,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2005年06月30日 17:35 by aguiar, last changed 2022年04月11日 14:56 by admin. This issue is now closed.
| Messages (3) | |||
|---|---|---|---|
| msg25692 - (view) | Author: Eduardo Aguiar (aguiar) | Date: 2005年06月30日 17:35 | |
hi, I have found a bug in 'tokenize' module: it is merging a COMMENT and a NL token for lines that start with a comment. I have made a fell changes and it seems to be working fine. Follows a patch: *** /usr/lib/python2.4/tokenize.py 2005年01月02日 03:34:20.000000000 -0200 --- tokenize.py 2005年06月30日 14:31:19.000000000 -0300 *************** *** 216,223 **** pos = pos + 1 if pos == max: break ! if line[pos] in '#\r\n': # skip comments or blank lines ! yield ((NL, COMMENT)[line[pos] == '#'], line[pos:], (lnum, pos), (lnum, len(line)), line) continue --- 216,235 ---- pos = pos + 1 if pos == max: break ! if line[pos] == '#': # skip comments ! end = len(line) - 1 ! while end > pos and line[end] in '\r\n': ! end = end - 1 ! end = end + 1 ! ! yield (COMMENT, line[pos:end], ! (lnum, pos), (lnum, end), line) ! yield (NL, line[end:], ! (lnum, end), (lnum, len(line)), line) ! continue ! ! if line[pos] in '\r\n': # skip blank lines ! yield (NL, line[pos:], (lnum, pos), (lnum, len(line)), line) continue Best regards, Eduardo Aguiar (eaguiar@programmer.net) |
|||
| msg25693 - (view) | Author: Raymond Hettinger (rhettinger) * (Python committer) | Date: 2005年07月15日 09:00 | |
Logged In: YES user_id=80475 Tim, this affects a line you checked-in, 1.23 on 6/18/2001. |
|||
| msg114517 - (view) | Author: Mark Lawrence (BreamoreBoy) * | Date: 2010年08月21日 17:06 | |
Fixed in r51526. |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022年04月11日 14:56:12 | admin | set | github: 42147 |
| 2010年08月21日 17:06:34 | BreamoreBoy | set | status: open -> closed nosy: + BreamoreBoy messages: + msg114517 dependencies: - tokenize: mishandles line joining resolution: fixed |
| 2009年02月16日 02:26:11 | ajaksu2 | set | keywords:
+ patch versions: + Python 2.6, - Python 2.4 dependencies: + tokenize: mishandles line joining type: behavior stage: test needed |
| 2005年06月30日 17:35:21 | aguiar | create | |