Python Nested List Comprehension with If Else

Question

I was trying to use a list comprehension to replace multiple possible string values in a list of values.

I have a list of column names which are taken from a cursor.description;

['UNIX_Time', 'col1_MCA', 'col2_MCA', 'col3_MCA', 'col1_MCB', 'col2_MCB', 'col3_MCB']

I then have header_replace;

{'MCB': 'SourceA', 'MCA': 'SourceB'}

I would like to replace the string values for header_replace.keys() found within the column names with the values.

I have had to use the following loop;

headers = []
for header in cursor.description:
 replaced = False
 for key in header_replace.keys():
 if key in header[0]:
 headers.append(str.replace(header[0], key, header_replace[key]))
 replaced = True
 break
 if not replaced:
 headers.append(header[0])

Which gives me the correct output;

['UNIX_Time', 'col1_SourceA', 'col2_SourceA', 'col3_SourceA', 'col1_SourceB', 'col2_SourceB', 'col3_SourceB']

I tried using this list comprehension;

[str.replace(i[0],k,header_replace[k]) if k in i[0] else i[0] for k in header_replace.keys() for i in cursor.description]

But it meant that items were duplicated for the unmatched keys and I would get;

['UNIX_Time', 'col1_MCA', 'col2_MCA', 'col3_MCA', 'col1_SourceA', 'col2_SourceA', 'col3_SourceA', 
'UNIX_Time', 'col1_SourceB', 'col2_SourceB', 'col3_SourceB', 'col1_MCB', 'col2_MCB', 'col3_MCB']

But if instead I use;

[str.replace(i[0],k,header_replace[k]) for k in header_replace.keys() for i in cursor.description if k in i[0]]

@Bakuriu fixed syntax

I would get the correct replacement but then loose any items that didn't need to have an string replacement.

['col1_SourceA', 'col2_SourceA', 'col3_SourceA', 'col1_SourceB', 'col2_SourceB', 'col3_SourceB']

Is there a pythonesque way of doing this or am I over stretching list comprehensions? I certainly find them hard to read.

• Bach: What do you mean by output? The contents of headers?
• sneeu: I’d stick with the loops, and avoid the list comprehensions. My rule of thumb is that if you’re doing more than one thing in the comprehension, it’s probably best expanded into a loop with .append.
• Bach: cursor.description holds a list of strings. header[0] holds a single character. Is that on purpose?
• Dave Anderson: @HansZauber yes sorry I will use them to write the headers of a csv file using csv.writer(), normally I just do writer.writerow([i[0] for i in cursor.description])
• RemcoGerlich: @sneeu: or a generator that yields the things that should end up in the list, even simpler than creating an empty list and appending to it

Answer 1

[str.replace(i[0],k,header_replace[k]) if k in i[0] for k in header_replace.keys() for i in cursor.description]

this is a SyntaxError, because if expressions must contain the else part. You probably meant:

[i[0].replace(k, header_replace[k]) for k in header_replace for i in cursor.description if k in i[0]]

With the if at the end. However I must say that list-comprehension with nested loops aren't usually the way to go. I would use the expanded for loop. In fact I'd improve it removing the replaced flag:

headers = []
for header in cursor.description:
 for key, repl in header_replace.items():
 if key in header[0]:
 headers.append(header[0].replace(key, repl))
 break
 else:
 headers.append(header[0])

The else of the for loop is executed when no break is triggered during the iterations.

---

I don't understand why in your code you use str.replace(string, substring, replacement) instead of string.replace(substring, replacement). Strings have instance methods, so you them as such and not as if they were static methods of the class.

• Dave Anderson: Thanks, didn't know about the else in a for loop that can be really useful. What is the purpose of the _?
• Bakuriu: @DaveAnderson I use tuple unpacking to avoid using header[0] all the time. But I just changed it because that would work only if cursor.description was a pair, but since it can have any number of elements I decided to roll that back. The symbol _ doesn't have any particular meaning. It's just often used to mean "a variable that I'm not going to use, but is required". This convention was probably taken from Haskell where it is a special pattern that means "any value matches".

Answer 2

If your data is exactly as you described it, you don't need nested replacements and can boil it down to this line:

l = ['UNIX_Time', 'col1_MCA', 'col2_MCA', 'col3_MCA', 'col1_MCB', 'col2_MCB', 'col3_MCB']
[i.replace('_MC', '_Source') for i in l]
>>> ['UNIX_Time',
>>> 'col1_SourceA',
>>> 'col2_SourceA',
>>> 'col3_SourceA',
>>> 'col1_SourceB',
>>> 'col2_SourceB',
>>> 'col3_SourceB']

• Dave Anderson: Unfortunately the common MC and Source isn't guaranteed it could be anything from M00 through to MFF and the replacement values can be almost anything.
• aisbaa: in that case I would advice to write specialized string replace function M_replace(source_string)
• Michael: In that case I'd go with regular expressions instead of a huge dictionary of replacements.

Answer 3

I guess a function will be more readable:

def repl(key):
 for k, v in header_replace.items():
 if k in key:
 return key.replace(k, v)
 return key
print map(repl, names)

Another (less readable) option:

import re
rx = '|'.join(header_replace)
print [re.sub(rx, lambda m: header_replace[m.group(0)], name) for name in names]