migrating large data store causes DBus timeout

[sascha_silbe]

Migrating a large data store (from v0 to v2 or for the version support branch v0/v1/v2 to v3) causes a DBus timeout because migration happens during activation (i.e. after sugar-datastore has been sent a DBus message and the sender is waiting for a reply).
Since Sugar doesn't start up properly (Journal not shown, uncolored entries in home view) users might be tempted to restart Sugar right away (especially on XO-1s that don't give any indication of "disk" activity). I haven't tried what happens in that case, but two data store instances both doing migration is quite likely and the outcome might very well be a data store broken beyond repair.

1257886897.213287 DEBUG root: STARTUP: Loading the journal
1257886922.372650 ERROR dbus.proxies: Introspect error on :1.7:/org/laptop/sugar/DataStore2: dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
1257886922.373063 DEBUG dbus.proxies: Executing introspect queue due to error
Traceback (most recent call last):
  File "/home/sascha.silbe/sugar-jhbuild/install/bin/sugar-session", line 125, in setup_journal_cb
    journalactivity.start()
  File "/home/sascha.silbe/sugar-jhbuild/install/lib/python2.5/site-packages/jarabe/journal/journalactivity.py", line 353, in start
    get_journal()
  File "/home/sascha.silbe/sugar-jhbuild/install/lib/python2.5/site-packages/jarabe/journal/journalactivity.py", line 348, in get_journal
    _journal = JournalActivity()
  File "/home/sascha.silbe/sugar-jhbuild/install/lib/python2.5/site-packages/jarabe/journal/journalactivity.py", line 118, in __init__
    self._setup_main_view()
  File "/home/sascha.silbe/sugar-jhbuild/install/lib/python2.5/site-packages/jarabe/journal/journalactivity.py", line 152, in _setup_main_view
    self._main_toolbox = MainToolbox()
  File "/home/sascha.silbe/sugar-jhbuild/install/lib/python2.5/site-packages/jarabe/journal/journaltoolbox.py", line 65, in __init__
    self.search_toolbar = SearchToolbar()
  File "/home/sascha.silbe/sugar-jhbuild/install/lib/python2.5/site-packages/jarabe/journal/journaltoolbox.py", line 120, in __init__
    self.refresh_filters()
  File "/home/sascha.silbe/sugar-jhbuild/install/lib/python2.5/site-packages/jarabe/journal/journaltoolbox.py", line 295, in refresh_filters
    for service_name in model.get_unique_values('activity'):
  File "/home/sascha.silbe/sugar-jhbuild/install/lib/python2.5/site-packages/jarabe/journal/model.py", line 474, in get_unique_values
    return _get_datastore().find_unique_values(dbus.Dictionary({}, signature='sv'), key)
  File "/usr/lib/pymodules/python2.5/dbus/proxies.py", line 68, in __call__
    return self._proxy_method(*args, **keywords)
  File "/usr/lib/pymodules/python2.5/dbus/proxies.py", line 140, in __call__
    **keywords)
  File "/usr/lib/pymodules/python2.5/dbus/connection.py", line 622, in call_blocking
    message, timeout)
dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

[tomeu]

Any suggestion about what we could do here?

[benzea]

Replying to tomeu:

Any suggestion about what we could do here?

Uh, I don't know what this could have to do with sugar-artwork, or whether I am even able to suggest much :-)

Oh well, lets do some guessing.

1. The datastore should not completely block the mainloop during migration
2. The migration (including progress) should be announced via dbus
3. sugar-session should display a big message that the data is migrated, with progress information (keeps people happy if they need to wait long)
4. Migration should likely start before the user can do any action (I am guessing that nothing can be stored during migration, so running activities would be bad. It would also add extra load and slow down the migration process.)

I guess the datastore is launched by dbus (during activation). Maybe the shell could start it very early by checking if migration is happening (it will be launched, and can do the check right away). If it is, go into a nested mainloop, and block everything until migration has finished. During that time one can display a nice dialog informing the user.

[sascha_silbe]

Related: #1160 (no way to wait for index to finish updating/(re)building)