Skip to main content
Stack Overflow
  1. About
  3. For Teams

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

Required fields*

Required fields*

UTF-8 decoding library

I have to code in an application which is in Unicode UTF-8 in Windows, MSVC 10. I'm aware that the UTF-8 encoded strings would use either 1 or 2 bytes per character. So, my question is : Is std::string suitable for this? If yes, how do I decode the strings? As far as I understand std::string is just an array of bytes and it doesn't provide any decoding logic. How can I know the logical length of the string? How can I extract logical characters from a string? Are there any libraries which helps me to extract logical characters from the string?

e.g : If I have the string "olé" in std::string, I need to know that the length is 3, but not 4.

Answer*

Draft saved
Draft discarded
Cancel
7
  • "in most cases ISO-8859-1 is what people use these days". On the interwebs, I see CP1252 mislabelled as ISO-8859-1 fairly frequently. Not sure which one you'd say they were "using" in that case, but it pretty much doesn't matter what "most people" are using, what matters is the minority of people whose text breaks your code ;-) Commented Jun 25, 2012 at 10:17
  • That’s not what OP wants. Why would he want to convert UTF-16 losslessly to single-byte codepoints? The question doesn’t imply this anywhere. Mention of ISO-8859-1 is just misguided. "in most cases [it’s] what people use these days" is completely wrong. In fact, modern browsers actually use a different encoding even if you explicitly request this encoding because almost nobody ever means ISO-8859-1, even if they say so. Commented Jun 25, 2012 at 10:19
  • Or how about converting to utf-16 or utf-32 for internal processing. Commented Jun 25, 2012 at 10:20
  • First of all, I did not say UTF-16. On Windows they use UCS-2. They don't know what UTF-16 is. Second, the first plane of Unicode is ISO-8859-1, whatever you say, that's what it is. Third, CP1252 is specific to Windows and if you convert from UTF-8 you're not going to get CP1252 which is why I mention that you get ISO-8859-1. Then it's your problem to properly select the correct font to render the text later. If you know what encoding you have, you can do it. Commented Jun 25, 2012 at 10:26
  • Yes, the first 256 characters of UCS-2 are the same as UCS-4, UTF-16 and UTF-8 once converted. They're all ISO-8859-1. Converting to another encoding (such as CP1252) requires tables or a library such as iconv (which I recommend you avoid!) Commented Jun 25, 2012 at 10:42

lang-cpp

AltStyle によって変換されたページ (->オリジナル) /