[Python-checkins] [python/cpython] fc354f: bpo-25324: copy tok_name before changing it (#1608...

Wed May 31 10:00:23 EDT 2017

 Branch: refs/heads/master
 Home: https://github.com/python/cpython
 Commit: fc354f07855a9197e71f851ad930cbf5652f9160
 https://github.com/python/cpython/commit/fc354f07855a9197e71f851ad930cbf5652f9160
 Author: Albert-Jan Nijburg <albertjan at trinket.io>
 Date: 2017年05月31日 (2017年5月31日)
 Changed paths:
 M Doc/library/token.rst
 M Include/token.h
 M Lib/test/test_tokenize.py
 M Lib/token.py
 M Lib/tokenize.py
 M Misc/NEWS
 M Parser/tokenizer.c
 Log Message:
 -----------
 bpo-25324: copy tok_name before changing it (#1608)
* add test to check if were modifying token
* copy list so import tokenize doesnt have side effects on token
* shorten line
* add tokenize tokens to token.h to get them to show up in token
* move ERRORTOKEN back to its previous location, and fix nitpick
* copy comments from token.h automatically
* fix whitespace and make more pythonic
* change to fix comments from @haypo
* update token.rst and Misc/NEWS
* change wording
* some more wording changes