Replace unicode character in the string

Question

A function which replaces an unicode character in a string:

void replaceAllOccurences(std::string& source,
 const std::string& replaceFrom,
 const std::string& replaceTo)
{
 std::vector<std::uint8_t> data(source.begin(), source.end());
 std::vector<std::uint8_t> pattern(replaceFrom.begin(), replaceFrom.end());
 std::vector<std::uint8_t> replaceData(replaceTo.begin(), replaceTo.end());
 std::vector<std::uint8_t>::iterator itr;
 while((itr = std::search(data.begin(), data.end(), pattern.begin(), pattern.end())) != data.end())
 {
 data.erase(itr, itr + pattern.size());
 data.insert(itr, replaceData.begin(), replaceData.end());
 }
 source = std::string(data.begin(), data.end());
}

Usage:

std::string source = "123€AAA€BBB";
std::string replaceFrom = "€";
std::string replaceTo = "\x80";
replaceAllOccurences(source, replaceFrom, replaceTo);

replaceTo may be get from some external conversion library, for example: iconvpp. Normally I would convert the whole source using iconvpp library but I have a case where I need to convert only particular character, for example "€".

• Loki Astari: Does this have anything specific to do with unicode or is it simply a way of replacing one sequence of characters with another?

Answer 1

If you're not worried about performance, you can replace all the manual work by using the <regex> facilities, which results in a considerable reduction of code to test and maintain.

#include <regex>
source = std::regex_replace(source, std::regex("€"), "\x80");

I would still keep it in a separate function to make it easy to change the implementation afterwards.

Answer 2

You don't need the std::vector<std::uint8_t> objects at all. You can use the input std::string objects directly.

Also, the code in the while loop needs to be updated for the following issues:

1. Make sure to capture the return value opf source.erase. If you don't the iterator is invalid.

2. To avoid infinite loop, use itr as the first argument to std::search.

3. Update itr inside the loop appropriately to avoid an infinite loop.

void replaceAllOccurences(std::string& source,
 const std::string& replaceFrom,
 const std::string& replaceTo)
{
 std::string::iterator itr = source.begin();
 while((itr = std::search(itr, source.end(), replaceFrom.begin(), replaceFrom.end())) != source.end())
 {
 itr = source.erase(itr, itr + replaceFrom.size());
 // itr is going be invalid after insert. Keep track of its
 // distance from begin() so we can update itr after insert.
 auto dist = std::distance(source.begin(), itr);
 source.insert(itr, replaceTo.begin(), replaceTo.end());
 // Make itr point to the character 1 past what got replaced.
 // This will avoid infinite loop incase the first character of
 // replaceTo is the same as the character being replaced.
 itr = std::next(source.begin(), dist+1);
 }
}

• Toby Speight: Doesn't insert() invalidate iterators?
• Loki Astari: Might have an issue with infinite loops.
• R Sahu: @TobySpeight, yes, it does. Thanks for pointing it out.
• R Sahu: @MartinYork, Indeed. Updated to address that issue.