Linked Questions
43 questions linked to/from Working with UTF-8 encoding in Python source
326
votes
7
answers
470k
views
"SyntaxError: Non-ASCII character ..." or "SyntaxError: Non-UTF-8 code starting with ..." trying to use non-ASCII text in a Python script
I tried this code in Python 2:
def NewFunction():
return '£'
But I get an error message that says:
SyntaxError: Non-ASCII character '\xa3' in file '...' but no encoding declared;
see http://www....
0
votes
2
answers
159
views
vs code encoding conflict between Output and matplotlib for french accents [duplicate]
when I prefix my .py file using utf-8 or Latin-1 I always get bad charcacters in OUTPUT or in matplotlib figure text.
Post 'Working with UTF-8 encoding in Python source' does not answer my question ...
181
votes
6
answers
99k
views
Correct way to define Python source code encoding
PEP 263 defines how to declare Python source code encoding. Normally, the first 2 lines of a Python file should start with:
#!/usr/bin/python
# -*- coding: <encoding name> -*-
But I have seen a ...
Oli's user avatar
- 16.1k
165
votes
4
answers
110k
views
Python: Using .format() on a Unicode-escaped string
I am using Python 2.6.5. My code requires the use of the "more than or equal to" sign. Here it goes:
>>> s = u'\u2265'
>>> print s
>>> ≥
>>> print "{0}".format(s)...
5
votes
3
answers
33k
views
brackets around print in python [duplicate]
I have this line of code in python
print 'hello world'
against
print ('hello world')
can someone tell me the difference between the two?
I used it in a a simple code
var = 3
if var > 2:
...
6
votes
3
answers
28k
views
Reading non-ASCII characters from a text file
I'm using python 2.7. I've tried many things like codecs but didn't work. How can I fix this.
myfile.txt
wörd
My code
f = open('myfile.txt','r')
for line in f:
print line
f.close()
Output
s\...
6
votes
3
answers
19k
views
"SyntaxError: Non-ASCII character" in running Python code
Im starting with python and when i use the interpreter and run this code:
>>>peliculas = ["movie1", "movie2", "movie3", "movie4"]
>>>print(peliculas[1])
when i use Pycharm IDE it ...
7
votes
3
answers
8k
views
Non-ASCII character '\x90' executing pserve on windows inside virtualenv
Question: How can i solve no-ascii character error executing pserve on virtualenv in windows?
Description: I'm trying to execute pserve (pyllons/pyramid development web server) inside a virtualenv on ...
4
votes
2
answers
2k
views
Defining unicode variables in Python
Recently, I have been reading about the Python source code encoding, especially PEP 263 and PEP 3120.
I have the following code:
# coding:utf-8
s = 'abc∂ ́ƒ©'
ƒ = 'My name is'
ß = ' ̊ß ̇ˆ†ˆ∆ ßå®åø©ˆ'
...
0
votes
1
answer
14k
views
utf-8 encoding and greek characters [duplicate]
While I managed to get all the data that I need as well as save it on a cv file, the output I get is in UTF-8 format, which is normal(correct me If I'm wrong)
TBH I've already "played" with the ....
1
vote
2
answers
5k
views
Python 3 character encoding issue
i am selecting values from a MySQL // Maria DB that contains latin1 charset with latin1_swedish_ci collation. There are possible characters from different European language as Spanish ñ, German ä or ...
2
votes
2
answers
5k
views
Writing CSV file with umlauts causing "UnicodeEncodeError: 'ascii' codec can't encode character"
I am trying to write characters with double dots (umlauts) such as ä, ö and Ö. I am able to write it to the file with data.encode("utf-8") but the result b'\xc3\xa4\xc3\xa4\xc3\x96' is not nice (UTF-8 ...
3
votes
1
answer
2k
views
Python: 'ascii' codec can't encode characters
I am using the following code to scrape a webpage that contains Japanese characters:
import urllib2
import bs4
import time
url = 'http://www.city.sapporo.jp/eisei/tiiki/toban.html'
pagecontent = ...
2
votes
2
answers
3k
views
Pymongo $regexMatch invalid flag in regex options: u
Using PyMongo 3.10.1, MongoDB 4.2 the aggregation below, using $group with $regexMatch works OK on command line:
db.accounts.aggregate([
{'$lookup': {'from': 'users', 'localField': '_id', '...
0
votes
2
answers
4k
views
Python Syntax error: non-ASCII [duplicate]
I keep getting an error and I'm not sure on how to fix it.
The Code line:
if not len(lines) or lines[-1] == '' or lines[-1] == '▁':
lines = list(filter(lambda line: False if line == '' or line ==...
1
vote
1
answer
2k
views
UnicodeEncodeError: 'ascii' codec can't encode character error
I am reading some files from google cloud storage using python
spark = SparkSession.builder.appName('aggs').getOrCreate()
df = spark.read.option("sep","\t").option("encoding", "UTF-8").csv('gs://...
0
votes
1
answer
4k
views
SyntaxError: Non-UTF-8 code starting with '\xe7'
I made a program which contains chinese and russian words, but when I ran it, I had a problem with the encoding
In the code that I shared, a complete sentence with some Russian and Chinese characters ...
1
vote
3
answers
3k
views
encode hash in utf-8
I want to substitude a substring with a hash - said substring contains non-ascii caracters, so I tried to encode it to UTF-8.
result = re.sub(r'(Start:\s*)([^:]+)(:\s*)([^:]+)', lambda m: m.group(1) ...
0
votes
2
answers
1k
views
How do I split a multi-languages line in Python and get the Unicode hex value?
I try to split this kind of lines in Python:
aiburenshi 爱不忍释 "לא מסוגל להינתק, לא יכול להיפרד מדבר מרוב חיבתו אליו"
This line contains Hebrew, simplified Chinese and English.
If I have a ...
0
votes
2
answers
2k
views
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf2 in position 6: ordinal not in range(128)
i'm trying to work with a file .pkl.
Here is my code
import pickle
with open('C:\\Users\\Utilizador\\Desktop\\teste\\teste.pkl', 'rb') as f:
data = pickle.load(f)
It gives me this following ...
3
votes
1
answer
1k
views
Unicode error when printing from Python to Heroku logs
I have a python script that's running periodically on Heroku using their Scheduler add-on. It prints some debug info, but when there's a non-ASCII character in the text, I get an error in the logs ...
0
votes
1
answer
1k
views
Unicode errors for '\xe4' for writing foreign characters into csv
Here is my code
The following works but writes the wrong string
import csv
import codecs
if __name__ == "__main__":
# This works for writing unico but writes wrong string
with codecs.open("....
4
votes
0
answers
1k
views
Non UTF-8 code Error when I run a flask application with a debug mode
I am running the simple flask application. The python file is like the followings.
import os
from flask import Flask, jsonify
app = Flask(__name__)
@app.route('/')
def Welcome():
return app....
0
votes
2
answers
1k
views
Correct way to hash password with Non-ASCII characters in Python 2.7
What is the correct and/or standard way of creating password hashes from user input when this user input does include also non ascii characters ?
I have noticed that when I generate MD5 hashes using ...
0
votes
1
answer
964
views
file encoding issues with eclipse pydev
I have been building a calculator with a GUI in python and finished, and I went back to update stuff and simplify the coding and know I am running into text encoding issues. I had them fixed and had ...
1
vote
0
answers
916
views
signal only works in main thread, no encoding declared
I get the following error for encoding when I activate DEBUG mode:
set FLASK_DEBUG=1
For some reason, the debug mode does not get activated by app.debug=True. Here is the error:
Non-UTF-8 code ...
1
vote
0
answers
561
views
JSON encoding/decoding issues in Python
I'm trying to read in a response from a REST API, parse it as JSON and write the properties to a CSV file.
It appears some of the characters are in an unknown encoding and can't be converted to ...
1
vote
1
answer
551
views
Getting a Unicode error when printing prettified BeautifulSoup
I am currently taking a course on Python and during our unit on Beautiful Soup the instructor uses the following code:
import requests, pprint
from bs4 import BeautifulSoup
url = 'https://www....
0
votes
1
answer
396
views
How to convert a script to UTF-8
I have a Sikuli/Python Script that works very good.
Since I am Dutch we have the same letters in our alphabet as the English alphabet.
Now there is however one letter that we have that does not ...
3
votes
0
answers
432
views
How can we write Sanskrit grammar rules for parsing?
How can we write Sanskrit grammar rules for parsing in NLTK Python? Is there any tagged corpus available in Python NLTK?
I tried to write a grammar as usual like this:
grammar = CFG.fromstring("""
S ...
0
votes
1
answer
185
views
Check if a string contains "pålogget" - unicode error
I am playing around with screenscraping with BeautifulSoup on a Norwegian site. I need to check if a string contains the word "Pålogget" (meaning logged on).
if "Pålogget" in status:
I get the ...
3
votes
1
answer
364
views
Problem while retriving tamil character from mysql
I have a table in mysql with tamil characters. Using python i attempted to fetch data from the table, and tried to print the response in json format.
def train_masters_live():
questions = ...
1
vote
0
answers
352
views
How to display Japanese characters from Google Cloud Storage correctly using Java?
Problem Statement:
I have a file containing Japanese characters located in my Google Cloud Storage Bucket. While downloading and printing the contents of the same, I find that the Japanese characters ...
1
vote
3
answers
297
views
How do I declare an encoding for Python source code?
How do I declare an encoding for this code?
I know how to declare encoding in other situations but how would I do it with this?
if used_prefix and cmd=="shoot" and user.name in whitelist:
name = ...
0
votes
2
answers
231
views
reading special characters from web in python
I am scraping an xml webpage for names of people via RE searching, however if the names contain special characters python is not reading them correctly. For Example:
Güngüneş A
comes out as:
G\xc3\...
0
votes
1
answer
210
views
Get unicode error when attempting to save file
I am trying to save a python-docx document in Ubuntu, but I get this error: 'ascii' codec can't encode character '\xed' in position 65: ordinal not in range(128). I tried to apply this solution, but I ...
0
votes
0
answers
199
views
jpype - diactric signs cause UnicodeDecodeError
I've recently started using jpype and it worked nice so far but when I tried to pass a string with polish diacritic sings (ą,ć,ó - stuff like that) as a parameter it throws UnicodeDecodeError.
Even ...
1
vote
1
answer
112
views
How to change the default charset for in MySQL?
I have a python code which adds rows to the MySQL database which runs on the same machine. The code is as follows:
import mysql.connector
config = {
'user': 'root',
'password': 'a$$enjoyer',
'...
0
votes
1
answer
93
views
How do I fix an unusual SyntaxError in the python executable itself when running line_profiler?
Here's a strange one.
First of all: This is NOT a problem with the encoding of a python script file that I have written myself or am importing (see e.g. Working with UTF-8 encoding in Python source ...
0
votes
1
answer
85
views
python encoding issue Raspberry
I'm having a problem using sys.argv[] to pass some values to a Python script.
My Python script is like that:
#Obtención de los valores provenientes del servidor
referencia = sys.argv[1] #referencia ...
2
votes
2
answers
54
views
Error thrown even if a line is commented
I have this line:
#str = u'Harsha: This has unicode character ♭.\n'
This line causes SyntaxError: Non-ASCII character '\xe2' even if it's commented.
If I remove this line the error is gone. Can ...
0
votes
0
answers
33
views
conversion of html code into respective diacritic symbol/alphabet
I have a HTML file containing words like <i>rūpa</i>.
How to convert it into rūpa(rūpa)?
Is there any way to convert it?
Also i get to know that these are the html ...
0
votes
1
answer
31
views
how to solve below pivoting table scenario in python? rows are converting into columns with respect to field names
1st table is my input table present in 1 data frame and 2nd table is my result table. can some one please help how to solve this using python
[1]: https://i.sstatic.net/CJzgW.jpg