Encoding error in Spanish (at least)

[humitos]

Steps to reproduce it:

1. Open words activity
2. Type "habit"

Issue:

The translated words are shown in the Translation Entry, but they are an incorrect encoding. It should be Unicode.

Expected behavior:

Show the translated words in the correct encoding.

Environment:

• jhbuild in Fedora 16
• git version of words-activiy

[humitos]

File encoding is not properly set

[humitos]

This is the [portion of code ​http://git.sugarlabs.org/words-activity/words-activity/blobs/master/LanguageModel.py#line29] that opens the files:

        with open(filename, 'r') as f:
            for line in f.readlines():
                line = line.rstrip()
                if line.startswith("#"):
                    continue

                words_list = line.split('\t')
                if words_list[0] in self.lang1_lang2:
                    self.lang1_lang2 [ words_list[0].lower() ] += ", " + words_list[-1].lower()
                else:
                    self.lang1_lang2 [ words_list[0].lower() ] = words_list[-1].lower()
                    
                if words_list[-1] in self.lang2_lang1:
                    self.lang2_lang1 [ words_list[-1].lower() ] += ", " + words_list[0].lower()
                else:
                    self.lang2_lang1 [ words_list[-1].lower() ]  = words_list[0].lower()

This code doesn't decode the content of the file. So, the encoding used it's the default encoding in the system: utf-8 in Sugar. So, I strongly suggest to convert those (all the .txt) files to UTF-8

[humitos]

Screenshot with the patch applied

[humitos]

​http://git.sugarlabs.org/words-activity/words-activity/commit/26a5017663bd195d7fe131b45ef587db737bac87

• from the mailing list:

It introduces some whitespace errors and one typo in 'tachygraphical tquigráfica' but will be fixed on release.

[humitos]

This will be fixed in version 17.

[RafaelOrtiz]

Now in V17.