I'm using JNI to interface between a Java program and a C++ function. The C++ function deals with multi-byte strings (CP 1252). I use this C++ code to convert the Java String to a char*:
char *arg=(char*) jEnv->GetStringUTFChars(jArg2,0);
This works fine unless I have some high-order characters. For example, if my input is:
Àlan (UTF: c2 6c 61 6e 20 4a 6f 6e 65 7e)
I can see that the resultant arg is:
c3 82 6c 61 6e
But, I would expect to see:
c0 6c 61 6e
Seeing that GetStringUTFChars() is supposed to return UTF strings, I tried obtaining the Unicode string with GetStringChars() and converting it via WideCharToMultiByte():
const jchar *str=jEnv->GetStringChars(jArg2,0);
WideCharToMultiByte(CP_UTF8,0,(LPCWSTR) str,jEnv->GetStringLength(jArg2),str,szStr,0,0);
(you can assume that I've allocated str and set szStr properly). In this situation, I see this in the resultant str:
c3 82 6c 61 6e
I've tried other CP_ values for the first parameter to WideCharToMultiByte, none yield useful results (they either return the above or substitute a '?' for the 'À'.
I would expect that somehow I could get this resultant str:
c0 6c 61 6e
But so far, I've had no luck.
Java uses a modified version of UTF-8. Here is a quote from Java's documentation:
Modified UTF-8 is not new to the Java platform, but it's something that application developers need to be more aware of when converting text that might contain supplementary characters to and from UTF-8. The main thing to remember is that some J2SE interfaces use an encoding that's similar to UTF-8 but incompatible with it. This encoding has in the past sometimes been called "Java modified UTF-8" or (incorrectly) just "UTF-8". For J2SE 5.0, the documentation is being updated to uniformly call it "modified UTF-8."
The incompatibility between modified UTF-8 and standard UTF-8 stems from two differences. First, modified UTF-8 represents the character U+0000 as the two-byte sequence 0xC0 0x80, whereas standard UTF-8 uses the single byte value 0x0. Second, modified UTF-8 represents supplementary characters by separately encoding the two surrogate code units of their UTF-16 representation. Each of the surrogate code units is represented by three bytes, for a total of six bytes. Standard UTF-8, on the other hand, uses a single four byte sequence for the complete character.
Modified UTF-8 is used by the Java Virtual Machine and the interfaces attached to it (such as the Java Native Interface, the various tool interfaces, or Java class files), in the java.io.DataInput and DataOutput interfaces and classes implementing or using them, and for serialization. The Java Native Interface provides routines that convert to and from modified UTF-8. Standard UTF-8, on the other hand, is supported by the String class, by the java.io.InputStreamReader and OutputStreamWriter classes, the java.nio.charset facilities, and many APIs layered on top of them.
Since modified UTF-8 is incompatible with standard UTF-8, it is critical not to use one where the other is needed. Modified UTF-8 can only be used with the Java interfaces described above. In all other cases, in particular for data streams that may come from or may be interpreted by software that's not based on the Java platform, standard UTF-8 must be used. The Java Native Interface routines that convert to and from modified UTF-8 cannot be used when standard UTF-8 is required.
The byte sequence c2 6c 61 6e 20 4a 6f 6e 65 7e is not valid under standard UTF-8. In cp1252, that same byte sequence would be the string Âlan Jone~ (notice  instead of À).
Under standard UTF-8, the string Àlan Jone~ would be the byte sequence c3 80 6c 61 6e 20 4a 6f 6e 65 7e (notice c3 80 6c instead of c2 6c).
All Java strings are natively UTF-16, so you don't need to retreive the string as UTF-8. Use GetStringChars() to get a original UTF-16 encoded characters and pass them as-is to WideCharToMultiByte() specifying 1252 as the codepage (note, in your example you are using str for both the UTF-16 input buffer and the cp1252 output buffer - don't get your variables confused!), eg:
const jchar *str = jEnv->GetStringChars(jArg2,0);
char *cp1252 = NULL;
int len = WideCharToMultiByte(1252, 0, (LPCWSTR)str, jEnv->GetStringLength(jArg2), NULL, 0, 0, 0);
if (len > 0)
{
cp1252 = new char[len + 1];
WideCharToMultiByte(1252, 0, (LPCWSTR)str, jEnv->GetStringLength(jArg2), cp1252, len, 0, 0);
cp1252[len] = 0;
}