Allow SoaS backup and restore from an XS server

[hamiltonchua]

On ticket 916 ( http://dev.sugarlabs.org/ticket/916) there was some information about enabling backup and restore with SoaS. I have been advised to move all the backup and restore stuff to a separate ticket which is this one.

Basically we want to support backup and restore with Sugar on a Stick.

In order to do so :

1) the sugar release needs to include
 http://dev.laptop.org/~martin/public_rpms/joyride/ds-backup-client-0.8.1-1.olpc3.noarch.rpm

2) modifications will need to be made to ds-backup.sh and ds-backup.py which is from the rpm above

[martin.langhoff]

Right - reviewing this. Thanks for the work you've put in the proposed patch.

I have a couple of comments...

In general, if this is deployed to machines that have been using current Sugar 0.82 it will fail :-/ because they have the info in the 'profile' object.

on the python side...

• if we are not using gconf, drop the import
• rsync line: you are dropping -a and not telling us why. Unless justified, add -z to the mix, keep -a
• why hardcode ~liveuser? Get homedir for current user from os
• why hardcode .sugar/soas? Things land in .sugar/default (and that location may be something that Sugar defines somewhere). In any case, this seems like it could be used in a variety of modes. Maybe store it under a more neutral name like .sugar/default/identifiers ?

Shell:

• If we don't have a nameserver, we have no net, or an adhoc mesh net. Let's keep the test. It's ultra cheap and valid.
• Why do you make skip_lowonbatt conditional? It _does_ have code for non-XO hw.

[hamiltonchua]

"In general, if this is deployed to machines that have been using current Sugar 0.82 it will fail :-/ because they have the info in the 'profile' object."

I don't understand. Are you saying that even without the modificaitons I made, the backup scripts will fail in 0.82 or that it is because of the modifications I made that it will fail on 0.82. The original ds-backup.py scripts reads from a file in /ofw and I made sure that the only time it tries to look for the SoaS settings is if /ofw is absent.

"rsync line: you are dropping -a and not telling us why. Unless justified, add -z to the mix, keep -a "

Sorry, this is a typo on my part. It just keeps coming back doesn't it ;-)
A new patch that I will upload will use -a not -z.

"Why do you make skip_lowonbatt conditional? It _does_ have code for non-XO hw. "

Because during testing we found that it doesn't work on some older desktop (no batteries) hardware. We were testing on some pentium 4 desktops and they silently fail that test.

Thanks very much for taking the time to review and comment on the patches. I'll confer with my colleagues, will have a new patch with your suggestions soon.

[martin.langhoff]

• About 0.82 - actually, we shouldn't worry. As long as it works with .84 regardless of hardware...

• rsync line: -a -z is good, maybe check that the ssh line doesn't have -C (having both compression options will burn cpu pointlessly)

• skip_lowbatt: can we fix it so that it does the right thing on battery machines, and gets skipped only in non-battery-lot machines?

[hamiltonchua]

Thanks for the swift response :-)

We'll investigate further with the battery script and see if we can patch it for such cases.

Also, someone just reminded me that I use to put the identifiers in gconf but they didn't work in the backup script because they are executed in cron.

Using "~/" to get to the home directory might not work in the backup script. I'll need to do some testing.

Thanks again.

[hamiltonchua]

I have applied Martin's suggestions to the new diff/patch.
- Using "~/" seems to be working.
- ssh isn't using -C so I stuck with -a with -z in the rsync options
- still looking at the battery script if it can accurately detect desktops (w/o battery) vs. laptops (with battery)

[tomeu]

ds-backup.py is not part of sugar. I guess it's part of a module maintained by Martin, so I'm reassigning this bug to him.

[tomeu]

Martin, do you want to add a module to the ones maintained at Sugar Labs as part of Sugar? Or want this script to be included in an existing Sugar module?

[martin.langhoff]

I am working on this to get a ds-backup-client that works on 0.82, 0.84 and if possible the build that Hamilton is working on.

In an earlier discussion, we couldn't find how to read values from gconf from cron scripts -- this was clarified by Tomeu today, so we _can_ read gconf values sanely from cron. I'll be reworking the patch so that it uses gconftool-2...

This will mean that the patches to schoolserver.py etc to store SN/UUID outside of gconf aren't needed. Unless they are now part of Sugar proper?

[hamiltonchua]

The build I am working on is the strawberry release I'm not sure if that's 0.82 or 0.84.

The patch for #916 related to registering an SoaS with an XS have been accepted to upstream sugar.

[martin.langhoff]

I have just made a number of changes in ds-backup. Instead of detecting "SoaS or not" we try various ways of getting the data, with reasonable defaults.

See  http://dev.laptop.org/git/users/martin/ds-backup.git/log/

The patches are counting on Sugar storing its config in the oldstyle .sugar/default/config or in gconf (now that we have found a way to read it). If the patch for #916 is "in" then we need to support a 3rd way... which is to get the values from the files.

[martin.langhoff]

Hamilton,

- can you look (in the Sugar control panel) the version of Sugar you are using?
- can you confirm for me the ouput on a registered SoaS, from Terminal.xo of the following commands...?

gconftool-2 -g /desktop/sugar/backup_url
gconftool-2 -g /desktop/sugar/soas_serial

[hamiltonchua]

Martin,

About My Computer says :

Build : SoaS release 1 (Strawberry)
Sugar : 0.84.2

Both commands do not return anything on my setup. I have patched my build to register with SoaS from #916. Note that the final patch does not store that info in gconf, it now stores it in /home/liveuser/.sugar/, specifically ...

/home/liveuser/.sugar/identifiers/backup_url
/home/liveuser/.sugar/identifiers/sn
/home/liveuser/.sugar/identifiers/uuid

I use the following def to extract the data ...

def get_identifier_info(info):
  identifier_dir = os.path.expanduser('~') + '/.sugar/default/identifiers/'
  fh = open(os.path.join(identifier_dir, info),'r')
  data = fh.read().rstrip('\0\n')
  fh.close()
  return data

I use it like this

backup_url = get_identifier_info('backup_url')
uid = get_identifier_info('uid')
sn = get_identifier_info('sn')

[martin.langhoff]

Ok, I've updated ds-backup to read

• oldstyle INI
• gconf
• the 'identifier' directory

The reason I added gconf too is that we've found a way to read gconf from cron scripts, so I suspect Sugar may revert to using plain gconf.

The patches are a in the repo above, and I've cut an RPM for the F11 based builds, so should work on SoaS.
 http://dev.laptop.org/~martin/public_rpms/f11/ds-backup-client-0.11-1.olpc3.noarch.rpm

Can you test and confirm?

[hamiltonchua]

Martin,

Can you kindly correct line 145 on ds-backup.py. The path should be ".sugar/default/identifiers" with an "s".

Aside from the above, things are looking good, I am able to get backups to the XS using your new rpm when I run it from the command line.

Is there a way to fix the cron ? By default it assumes the user that executes the script to be "olpc", I wonder if there's a way to modify it to use "liveuser" when the rpm is installed, I'm not sure if there's a reliable way to detect if the sugar install is on SoaS or otherwise.

[sdz]

Great work, folks! :)

Martin, is there any chance of getting the src.rpm?

[martin.langhoff]

Fixed. RPM and SRPM and at
 http://dev.laptop.org/~martin/public_rpms/f11/

WRT the cron thing... I frankly don't know. If there is a reasonable solution, I will be happy to apply it.

[martin.langhoff]

This is also affected by #1098

[sdz]

I'm not sure why the milestone was changed here. From what I know, the Sugar specific changes have been already committed and it's just about pushing one package to SoaS. #1098 has, from what I know, also been fixed in 0.84.7 (which dsd just pushed today) and SoaS will even ship 0.86.x, so I don't think this is an issue.

[sdz]

Oh, I forgot to say this: Martin, thanks for the src.rpm! :)

[RafaelOrtiz]

ok thanks for the clarification sebastian.:)

[tomeu]

It's not clear to me if there's anything to be reviewed or done for this ticket. I'm taking out of the review queue for now, please update this ticket and close it if possible. Thanks!