UnicodeDecodeError by python regex

Question

I am trying to replace all tabs with a space so I can put my Coma seperated text in one line in another file. Now my code look like this:

from __future__ import print_function
import re
import ast
f = open('sample_test.txt', 'r')
g = open('sample_test1.txt', 'w')
for line in f:
 c = re.sub(r'\R', r' ', line.rstrip())
 print (c, file = g)
f.close()

The problem now is that I get this error:

UnicodeDecodeError: 'charmap' codec can't decode byte 0x98 in position 1944: character maps to <undefined>

• Tadhg McDonald-Jensen: what line does the error point to?

Answer 1

Open the files as utf-8, you also don't need a regex if you just want to replace tabs:

import io
with io.open('sample_test.txt', encoding="utf-8") as f, io.open('sample_test1.txt', 'w', encoding="utf-8") as g: 
 for line in f:
 g.write(line.replace("\t"," "))

• Vedad: Thx that solves the Problem with the error, but for some reasson it don't replace the tabs with an empty space. The regex also don't work. Any Idea while it just copy the same content to g???
• Padraic Cunningham: Are you sure you have tabs and not just spaces? If you print(repr(line)) what do you see?
• Vedad: It works now i needet to change the \t to \n. Thx a lot now it totally works :D
• Vedad: is there a reasson why it don't work with the regex cause i just tested the same with regex and it does nothing.
• Padraic Cunningham: What are you expecting \R to change?