Brazilian Portuguese problem

[cjl]

For reasons I don't understand, pt_BR takes an incredibly long time to respond in Pootle to requests like "update from templates". I've been noticing this for Honey, it is not a general Pootle issue as the other ninety or so Honey project languages update without any issues.

The delay is so long that it often fails to update at all.

[cjl]

Attempted update from templates for Portuguese and it zeroed out the Honey project. Is it possible pt and pt_BR are cross-linked somehow and causing both problems?

[RafaelOrtiz]

update, I'm getting this error while trying to perform re-indexing for pt_BR:

[Thu Jul 21 23:26:02 2011] [error] Traceback (most recent call last):
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/pymodules/python2.6/django/core/handlers/base.py", line 92, in get_response
[Thu Jul 21 23:26:02 2011] [error]     response = callback(request, *callback_args, **callback_kwargs)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/view.py", line 56, in decorated_f
[Thu Jul 21 23:26:02 2011] [error]     return f(request, translation_project, *args, **kwargs)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/view.py", line 66, in decorated_f
[Thu Jul 21 23:26:02 2011] [error]     return f(request, translation_project, *args, **kwargs)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/view.py", line 203, in handle_file
[Thu Jul 21 23:26:02 2011] [error]     return handle_translation_file(request, translation_project, file_path)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/view.py", line 127, in handle_translation_file
[Thu Jul 21 23:26:02 2011] [error]     return find_and_display(request, store.parent, next_store_item, prev_store_item)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/translate_page.py", line 846, in find_and_display
[Thu Jul 21 23:26:02 2011] [error]     store, item = get_position(request, next_store_item, prev_store_item)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/translate_page.py", line 828, in get_position
[Thu Jul 21 23:26:02 2011] [error]     return process_post_main(store_name, item, request, next_store_item, prev_store_item)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/translate_page.py", line 810, in process_post_main
[Thu Jul 21 23:26:02 2011] [error]     prev_item, next_item = process_post(request, store)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/translate_page.py", line 804, in process_post
[Thu Jul 21 23:26:02 2011] [error]     last_item = handle_submits(last_item, request, store, submits, skips, translations, comments, fuzzies)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/translate_page.py", line 744, in handle_submits
[Thu Jul 21 23:26:02 2011] [error]     unit_update.update_translation(store, item, newvalues, request)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/unit_update.py", line 75, in update_translation
[Thu Jul 21 23:26:02 2011] [error]     user=request.user, language=translation_project.language)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_store/fields.py", line 209, in updateunit
[Thu Jul 21 23:26:02 2011] [error]     self.reclassifyunit(item, checker)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_store/fields.py", line 148, in reclassifyunit
[Thu Jul 21 23:26:02 2011] [error]     state = self._statscache.recacheunit(self.path, checker, unit)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/translate/storage/statsdb.py", line 122, in decorated_f
[Thu Jul 21 23:26:02 2011] [error]     result = f(self, *args, **kwargs)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/translate/storage/statsdb.py", line 514, in recacheunit
[Thu Jul 21 23:26:02 2011] [error]     fileid = self._getfileid(filename, check_mod_info=False)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/translate/storage/statsdb.py", line 122, in decorated_f
[Thu Jul 21 23:26:02 2011] [error]     result = f(self, *args, **kwargs)
[Thu Jul 21 23:26:02 2011] [error]   File "/usr/lib/python2.6/dist-packages/translate/storage/statsdb.py", line 376, in _getfileid
[Thu Jul 21 23:26:02 2011] [error]     WHERE fileid=?;""", (mod_info[0], mod_info[1], fileid))
[Thu Jul 21 23:26:02 2011] [error] OperationalError: database is locked
[Thu Jul 21 23:26:02 2011] [error] 
[Thu Jul 21 23:26:34 2011] [error] Traceback (most recent call last):
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/pymodules/python2.6/django/core/handlers/base.py", line 92, in get_response
[Thu Jul 21 23:26:34 2011] [error]     response = callback(request, *callback_args, **callback_kwargs)
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/view.py", line 56, in decorated_f
[Thu Jul 21 23:26:34 2011] [error]     return f(request, translation_project, *args, **kwargs)
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/view.py", line 66, in decorated_f
[Thu Jul 21 23:26:34 2011] [error]     return f(request, translation_project, *args, **kwargs)
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/view.py", line 203, in handle_file
[Thu Jul 21 23:26:34 2011] [error]     return handle_translation_file(request, translation_project, file_path)
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/view.py", line 127, in handle_translation_file
[Thu Jul 21 23:26:34 2011] [error]     return find_and_display(request, store.parent, next_store_item, prev_store_item)
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/translate_page.py", line 847, in find_and_display
[Thu Jul 21 23:26:34 2011] [error]     return view(request, directory, store, item)
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/translate_page.py", line 872, in view
[Thu Jul 21 23:26:34 2011] [error]     items, translations, first_item = make_table(request, profile, store, item)
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_app/views/language/translate_page.py", line 599, in make_table
[Thu Jul 21 23:26:34 2011] [error]     suggestions = {item: store.getsuggestions(item)}
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_store/models.py", line 143, in getsuggestions
[Thu Jul 21 23:26:34 2011] [error]     unit = self.file.getitem(item)
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_store/fields.py", line 161, in getitem
[Thu Jul 21 23:26:34 2011] [error]     return self.store.units[self.total[item]]
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_store/fields.py", line 156, in _get_total
[Thu Jul 21 23:26:34 2011] [error]     return self.getstats()['total']
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/pootle_store/fields.py", line 128, in getstats
[Thu Jul 21 23:26:34 2011] [error]     stats_tuple.stats = self._statscache.filestatestats(self.path, store=self._get_store)
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/translate/storage/statsdb.py", line 609, in filestatestats
[Thu Jul 21 23:26:34 2011] [error]     fileid = self._getfileid(filename, store=store)
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/translate/storage/statsdb.py", line 122, in decorated_f
[Thu Jul 21 23:26:34 2011] [error]     result = f(self, *args, **kwargs)
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/translate/storage/statsdb.py", line 387, in _getfileid
[Thu Jul 21 23:26:34 2011] [error]     return self._cachestore(store, realpath, mod_info)
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/translate/storage/statsdb.py", line 122, in decorated_f
[Thu Jul 21 23:26:34 2011] [error]     result = f(self, *args, **kwargs)
[Thu Jul 21 23:26:34 2011] [error]   File "/usr/lib/python2.6/dist-packages/translate/storage/statsdb.py", line 430, in _cachestore
[Thu Jul 21 23:26:34 2011] [error]     path=?;""", (realpath,))
[Thu Jul 21 23:26:34 2011] [error] OperationalError: database is locked

[alsroot]

The viewslides.po in pt_BR weigh 3M, in particular there was meaningless data in "Forward" word,
​http://translate.sugarlabs.org/pt_BR/honey/viewslides.po?item=4&view_mode=translate
I've reset this word and "update the files from templates" became normal.

[alsroot]

The source of the problem might be /var/lib/pootle/translations/terminology/pt_BR/glossary.po file. it weights 116M and contains bunch of trash.

cjl: could you take a look to this file and clean up it.

[cjl]

Investigating now. will report back after fixing it.

[cjl]

Okay, the glossary has been fixed, pt_BR is updating fine.

Leaving closed.