I am processing a file on windows OS in Python. I am getting errors like Unicode error surrogates not allowed.
Sample Text from Document:
Ten states led by Texas Attorney General Ken Paxton (R) filed an antitrust lawsuit against
Google on Wednesday, alleging the tech giant illegally sought to suppress competition and
reap massive profits from targeted advertisements placed across the Web.
The lawsuit — filed in a Texas federal court and backed exclusively by Republicans — strikes
at the heart of Google’s lucrative business in connecting those who seek to buy online ads
with the websites that sell them. Paxton and his GOP allies contend that Google relied on a
mix
of improper tactics to force its ad tools on publishers and solidify its pole position as a
"middleman" in the invisible transactions that power much of the Web.
Online advertising is expected to generate 42ドル billion in revenue this year for Google,
which captures a third of all digital ad spending, according to an October projection from
the firm eMarketer. Google’s vast reach led Texas and other state attorneys general to c
onclude in their lawsuit that the tech giant essentially had built the "largest electronic
trading market in existence," operating ad systems that are not unlike trades on a stock
exchange.
Code1:
return_doc.to_csv(path, index= False)
Error1:
UnicodeEncodeError: 'utf-8' codec can't encode character '\udc9d' in position 168: surrogates not allowed
Code2:
return_doc.to_csv(path, index= False, encoding='cp1252')
Error2:
UnicodeEncodeError: 'charmap' codec can't encode character '\udc9d' in position 168: character maps to
Code3:
return_doc.to_csv(path, index= False, encoding='ISO 8859-15')
Error3:
UnicodeEncodeError: 'charmap' codec can't encode character '\u201d' in position 14: character maps to
I have used
Code4:
return_doc.to_csv(path, index= False, encoding='cp1252', errors='replace)
The text from
"The actions harm every person in America," Paxton said in a video statement preceding the
case, which asked a judge to consider "structural" remedies that could theoretically include
forcing a breakup of the company.
converted into
“The actions harm every person in America,ï¿1⁄2? Paxton said in a video statement preceding
the case, which asked a judge to consider “structuralï¿1⁄2? remedies that could
theoretically include forcing a breakup of the company.
Which I don't want to happen.
PLease suggest me a solution where I don't get any error and don't get text changed.
stdinand ` stdout` on Windows is UTF-8. And the Windows-1252 encoding would not give the error surrogates not allowed.