This issue tracker has been migrated to GitHub ,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
| Author | loewis |
|---|---|
| Recipients | barry, loewis, mark.dickinson |
| Date | 2008年03月17日.04:39:05 |
| SpamBayes Score | 0.07301519 |
| Marked as misclassified | No |
| Message-id | <1205728746.74.0.894937501179.issue719888@psf.upfronthosting.co.za> |
| In-reply-to |
| Content | |
|---|---|
In 3k, the tokenize module should definitely return strings, and, in doing so, it should definitely consider the encoding declaration (and also the default encoding in absence of the encoding declaration). For 2.6, I wouldn't mind if it were changed incompatibly so that it returns Unicode strings, or else that it parses in Unicode, and then encodes back to the source encoding before returning anything. |
|
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2008年03月17日 04:39:07 | loewis | set | spambayes_score: 0.0730152 -> 0.07301519 recipients: + loewis, barry, mark.dickinson |
| 2008年03月17日 04:39:06 | loewis | set | spambayes_score: 0.0730152 -> 0.0730152 messageid: <1205728746.74.0.894937501179.issue719888@psf.upfronthosting.co.za> |
| 2008年03月17日 04:39:05 | loewis | link | issue719888 messages |
| 2008年03月17日 04:39:05 | loewis | create | |