Does Google App Engine Display Unicode Differently In StringProperty V StringListProperty Objs?

I have a db.StringProperty() mRegion that is set to some Korean text. I see in my Dashboard that the value is visibly in Korean like this: 한국 : 충청남도 However, when I ta

Solution 1:

It's quite possible that the admin interface displays the two differently, yes. In the latter case it's clearly doing a repr(s), while in the former it's just printing the string.

The admin interface's interface doesn't affect how your code works, though - both Strings and StringLists are stored the same way in the datastore, and will come back as Unicode strings for you to deal with as you wish.

I highly recommend reading this Joel on Software post about unicode. In short, you're dealing with two kinds of things: Binary data, and unicode characters. To confuse you, Python exposes these both as strings - 'unicode strings' and 'raw strings', respectively, but you should only treat the former as actual strings.

The datastore, with its StringListProperty and StringProperty, stores and returns Unicode strings. Your framework should also be giving you Unicode strings, and accepting Unicode strings back, but some poorly designed frameworks don't.

What you need to do is check that you are using Unicode strings everywhere you deal with text, that you explicitly call .encode() to convert a Unicode string to a raw string, and .decode() to convert a raw string to a unicode string, and that the character encoding on the returned response is set correctly, and you're encoding your strings using the same encoding. How you do that will depend on your framework.

Once you've done that, if you still have trouble, I would suggest writing some simple unit tests - storing data to the datastore and retrieving it and manipulating it, then checking it equals what you expect - to pin down where the issue is.