Unicode and ascii

[corto]

Hi there,
I'm programming for windows mobile 2003. The device is setup in unicode, but I often need to convert this unicode to ascii (to write in a log for example), or to read ascii path and convert it to unicode.

I'm coding in C++ so I was thinking on building a kind of convertible string (no inheritance to std:string). But because I'm not familiar with unicode, I may not know of some stuff that already exists to take care of exactly this...

Anyone can give me a few pointers that will help my conversions needs?

Thanks!

[hanst99]

Convert the character encoding to UTF8, ASCII characters have the same encoding in UTF8 as in ASCII. There is probably some lib available to do this, but you'll have to look for that yourself I guess.

[corto]

Hi Hanst99,
I am required to use unicode in WinCE 3.0. I haven't seen anywhere that Win32 functions would accept UTF8, and besides, using UTF8 or Unicode, my problem remains the same.

[hanst99]

UTF8 is Unicode, just a different encoding. By default windows uses UTF16, and I just told you if you find a library that lets you convert UTF16 to UTF8 (which is always legal, unlike a conversion from UTF16 to ASCII), which shouldn't be that hard to do (considering that UTF8 is used for almost anything web based), your problem is already solved as the UTF8 encoding for characters in the ASCII range is the same as in ASCII.

[corto]

hey thanks hanst99!
that sounds good.

But are you sure the Unicode used in win32 for WinCE 3.0 (Windows mobile 2003) is still UTF16?

[hanst99]

I am not aware of any windows version past win98 that doesn't use UTF16 internally. Though I am not very familiar with the mobile versions.

[corto]

Great, I found this article that describes how to wrap the encoder/decoder from unicode inc, using standard containers like std::wstring and std::string.

http://www.codeproject.com/KB/string/UtfConverter.aspx

[Radioactive]

Wasnt it possible to just convert each 2 byte unicode thingy to a char without any losses unless you use characters not in ascii?

[corto]

It may look that simple, but I'm trying to avoid bugs that are hard to understand (like when this kind of translation doesn't work).
So I needed something that was bullet proof at least, short of being bomb proof! ;)

I've finished it, not using the link I pasted above but using win32's MultiByteToWideChar and it's friend. Then I pack the resulting (LPSTR) into a std::string, using std::string::c_str() to generate the null-terminated form. I also pack the resulting (LPWSTR) into a std::wstring and I also use std::wstring::c_str() to generate the null-terminated form.

I didn't know: the c_str() member of the wstring class returns an (unsigned short*) which is compatible with (LPWSTR).

My object is used to handle paths somewhat efficiently, and allows concatenation of paths, extracting the file name or its container directory path, etc...

I'll work on it a (quite) bit more and I'm hoping to publish it gratis.

Note that using the UFT converter I posted above is probably the best way to make portable code, my current implementation is win32-only.

Thanks all, Simon