Encoding Problems

Question

Having an issue with a java string used for emails in a java source file. The string contains "Protégé". Our server environment from what I have been able to determine uses UTF-8.

So I converted it to "Protégé" for UTF-8. It works great on our server, but when I run it locally it doesn't translate it properly. So I changed eclipse to use UTF-8 under preferences but it doesn't translate it locally. Still shows "Protégé". Any ideas?

From the comments:

I ran this locally and on our server:

OutputStreamWriter out = new OutputStreamWriter(new ByteArrayOutputStream());
System.out.println(out.getEncoding());

And it displays Cp1252 locally and UTF-8 on our JBoss server. We originally had the string with "Protégé" but on JBoss it only shows "Prot".

When I use "Prot\u00e9g\u00e9" it works fine locally but when ran on our server it shows "Protg".

• Tyler: Java always uses UTF-8 by default, IIRC. Did you try just typing Protégé? Or better yet, Prot\u00e9g\u00e9?
• Vineet Reynolds: @MatrixFrog, I think you meant UTF-16.
• gatkin: Please post code to explain what you're doing. Strings in Java are always utf-16 and you can't change that; encodings such as utf-8 refer to the relationship between strings and byte arrays.
• Brian: I ran this locally and on our server: OutputStreamWriter out = new OutputStreamWriter(new ByteArrayOutputStream()); System.out.println(out.getEncoding()); And it displays Cp1252 locally and UTF-8 on our JBoss server. We originally had the string with "Protégé" but on JBoss it only shows "Prot"
• SyntaxT3rr0r: @MatrixFrog:*"Java always uses UTF-8 by default, IIRC."* Nonsense. And, no, you don't remember correctly.

Answer 1

If the string contains "Prot\u00e9g\u00e9", this precludes a compiler encoding problem (like alluded by SyntaxT3rr0r), since it is now right in the Java String (unless there is a compiler bug, which I would not assume).

Thus we have an problem between output, transfer and display. How do you look at the output from your server? It could be that there somewhere is some recoding which destroys your strings. Or that somewhere some output is mis-declared.

If you are using a Terminal/command window to look at the output, consider setting it to UTF-8 before connecting to the server.

---

And yes, Java uses internally UTF-16 for the strings, but some system dependent encoding as both compiler default and default encoding of OutputStreamWriter/InputStreamReader and several other APIs which convert between strings and bytes. Looks like this is UTF-8 on the server and Windows-1252 on your client system. This should not really matter here.

• Brian: The output is viewed via email which changes depending if the application is ran locally or on our server environment.
• Paŭlo Ebermann: So, you have to look at the piece of software which creates your eMail. It has to make sure that the same encoding which is used is also declared in the mail header.
• Paŭlo Ebermann: So, show the code. (Add it to your question, it has an edit button).

Answer 2

Try this:

 MimeMessage msg = new MimeMessage(session);
 MimeBodyPart mbp1 = new MimeBodyPart();
 mbp1.setDataHandler(new DataHandler(new ByteArrayDataSource(message.toString, "text/html"))); 
 mbp1.setContent(new String(message.getBytes("UTF-8"),"ISO-8859-1"), "text/html");
 Multipart mp = new MimeMultipart();
 mp.addBodyPart(mbp1);
 msg.setContent(mp, "text/html");

put your language char set instead of "ISO-8859-1"