Opened 10 years ago

Last modified 6 years ago

#1124 assigned enhancement

Allow SoaS backup and restore from an XS server

Reported by: hamiltonchua Owned by: martin.langhoff
Priority: High Milestone:
Component: Sugar on a Stick (SoaS) Version: Unspecified
Severity: Major Keywords: r! GPA
Cc: martin.langhoff, sascha_silbe, FGrose, sdz, CarolineM, tch, jasg Distribution/OS: Unspecified
Bug Status: Needinfo

Description

On ticket 916 (http://dev.sugarlabs.org/ticket/916) there was some information about enabling backup and restore with SoaS. I have been advised to move all the backup and restore stuff to a separate ticket which is this one.

Basically we want to support backup and restore with Sugar on a Stick.

In order to do so :

1) the sugar release needs to include
http://dev.laptop.org/~martin/public_rpms/joyride/ds-backup-client-0.8.1-1.olpc3.noarch.rpm

2) modifications will need to be made to ds-backup.sh and ds-backup.py which is from the rpm above

Attachments (4)

ds-backup.sh (4.2 KB) - added by hamiltonchua 10 years ago.
ds-backup.py (5.4 KB) - added by hamiltonchua 10 years ago.
ds-backup080109.py.patch (2.0 KB) - added by hamiltonchua 10 years ago.
ds-backup080109.sh.patch (1.3 KB) - added by hamiltonchua 10 years ago.

Download all attachments as: .zip

Change History (40)

comment:1 Changed 10 years ago by hamiltonchua

  • Cc martin.langhoff added

comment:2 Changed 10 years ago by sascha_silbe

  • Cc sascha_silbe added

comment:3 Changed 10 years ago by FGrose

  • Cc FGrose added

Changed 10 years ago by hamiltonchua

comment:4 Changed 10 years ago by hamiltonchua

  • Keywords r? added

comment:5 Changed 10 years ago by martin.langhoff

Right - reviewing this. Thanks for the work you've put in the proposed patch.

I have a couple of comments...

In general, if this is deployed to machines that have been using current Sugar 0.82 it will fail :-/ because they have the info in the 'profile' object.

on the python side...

  • if we are not using gconf, drop the import
  • rsync line: you are dropping -a and not telling us why. Unless justified, add -z to the mix, keep -a
  • why hardcode ~liveuser? Get homedir for current user from os
  • why hardcode .sugar/soas? Things land in .sugar/default (and that location may be something that Sugar defines somewhere). In any case, this seems like it could be used in a variety of modes. Maybe store it under a more neutral name like .sugar/default/identifiers ?


Shell:

  • If we don't have a nameserver, we have no net, or an adhoc mesh net. Let's keep the test. It's ultra cheap and valid.
  • Why do you make skip_lowonbatt conditional? It _does_ have code for non-XO hw.

comment:6 Changed 10 years ago by hamiltonchua

"In general, if this is deployed to machines that have been using current Sugar 0.82 it will fail :-/ because they have the info in the 'profile' object."

I don't understand. Are you saying that even without the modificaitons I made, the backup scripts will fail in 0.82 or that it is because of the modifications I made that it will fail on 0.82. The original ds-backup.py scripts reads from a file in /ofw and I made sure that the only time it tries to look for the SoaS settings is if /ofw is absent.

"rsync line: you are dropping -a and not telling us why. Unless justified, add -z to the mix, keep -a "

Sorry, this is a typo on my part. It just keeps coming back doesn't it ;-)
A new patch that I will upload will use -a not -z.

"Why do you make skip_lowonbatt conditional? It _does_ have code for non-XO hw. "

Because during testing we found that it doesn't work on some older desktop (no batteries) hardware. We were testing on some pentium 4 desktops and they silently fail that test.

Thanks very much for taking the time to review and comment on the patches. I'll confer with my colleagues, will have a new patch with your suggestions soon.

comment:7 follow-up: Changed 10 years ago by martin.langhoff

  • About 0.82 - actually, we shouldn't worry. As long as it works with .84 regardless of hardware...
  • rsync line: -a -z is good, maybe check that the ssh line doesn't have -C (having both compression options will burn cpu pointlessly)
  • skip_lowbatt: can we fix it so that it does the right thing on battery machines, and gets skipped only in non-battery-lot machines?

comment:8 in reply to: ↑ 7 Changed 10 years ago by hamiltonchua

Thanks for the swift response :-)

We'll investigate further with the battery script and see if we can patch it for such cases.

Also, someone just reminded me that I use to put the identifiers in gconf but they didn't work in the backup script because they are executed in cron.

Using "~/" to get to the home directory might not work in the backup script. I'll need to do some testing.

Thanks again.

comment:9 Changed 10 years ago by hamiltonchua

  • Type changed from defect to enhancement

Changed 10 years ago by hamiltonchua

Changed 10 years ago by hamiltonchua

Changed 10 years ago by hamiltonchua

comment:10 Changed 10 years ago by hamiltonchua

I have applied Martin's suggestions to the new diff/patch.

  • Using "~/" seems to be working.
  • ssh isn't using -C so I stuck with -a with -z in the rsync options
  • still looking at the battery script if it can accurately detect desktops (w/o battery) vs. laptops (with battery)

comment:11 Changed 10 years ago by tomeu

  • Owner changed from tomeu to martin.langhoff
  • Status changed from new to assigned

ds-backup.py is not part of sugar. I guess it's part of a module maintained by Martin, so I'm reassigning this bug to him.

comment:12 Changed 10 years ago by sdz

  • Cc sdz added

comment:13 Changed 10 years ago by tomeu

Martin, do you want to add a module to the ones maintained at Sugar Labs as part of Sugar? Or want this script to be included in an existing Sugar module?

comment:14 Changed 10 years ago by gregorio

  • Cc CarolineM added
  • Keywords GPA added

comment:15 Changed 10 years ago by tomeu

  • Bug Status changed from Unconfirmed to Needinfo
  • Component changed from sugar to journal
  • Milestone changed from Unspecified by Release Team to 0.86
  • Priority changed from Unspecified by Maintainer to High
  • Severity changed from Unspecified to Major

comment:16 follow-up: Changed 10 years ago by martin.langhoff

I am working on this to get a ds-backup-client that works on 0.82, 0.84 and if possible the build that Hamilton is working on.

In an earlier discussion, we couldn't find how to read values from gconf from cron scripts -- this was clarified by Tomeu today, so we _can_ read gconf values sanely from cron. I'll be reworking the patch so that it uses gconftool-2...

This will mean that the patches to schoolserver.py etc to store SN/UUID outside of gconf aren't needed. Unless they are now part of Sugar proper?

comment:17 in reply to: ↑ 16 Changed 10 years ago by hamiltonchua

The build I am working on is the strawberry release I'm not sure if that's 0.82 or 0.84.

The patch for #916 related to registering an SoaS with an XS have been accepted to upstream sugar.

comment:18 Changed 10 years ago by martin.langhoff

I have just made a number of changes in ds-backup. Instead of detecting "SoaS or not" we try various ways of getting the data, with reasonable defaults.

See http://dev.laptop.org/git/users/martin/ds-backup.git/log/

The patches are counting on Sugar storing its config in the oldstyle .sugar/default/config or in gconf (now that we have found a way to read it). If the patch for #916 is "in" then we need to support a 3rd way... which is to get the values from the files.

comment:19 follow-up: Changed 10 years ago by martin.langhoff

Hamilton,

  • can you look (in the Sugar control panel) the version of Sugar you are using?
  • can you confirm for me the ouput on a registered SoaS, from Terminal.xo of the following commands...?

gconftool-2 -g /desktop/sugar/backup_url
gconftool-2 -g /desktop/sugar/soas_serial

comment:20 in reply to: ↑ 19 Changed 10 years ago by hamiltonchua

Martin,

About My Computer says :

Build : SoaS release 1 (Strawberry)
Sugar : 0.84.2

Both commands do not return anything on my setup. I have patched my build to register with SoaS from #916. Note that the final patch does not store that info in gconf, it now stores it in /home/liveuser/.sugar/, specifically ...

/home/liveuser/.sugar/identifiers/backup_url
/home/liveuser/.sugar/identifiers/sn
/home/liveuser/.sugar/identifiers/uuid

I use the following def to extract the data ...

def get_identifier_info(info):
  identifier_dir = os.path.expanduser('~') + '/.sugar/default/identifiers/'
  fh = open(os.path.join(identifier_dir, info),'r')
  data = fh.read().rstrip('\0\n')
  fh.close()
  return data

I use it like this

backup_url = get_identifier_info('backup_url')
uid = get_identifier_info('uid')
sn = get_identifier_info('sn')

comment:21 Changed 10 years ago by martin.langhoff

Ok, I've updated ds-backup to read

  • oldstyle INI
  • gconf
  • the 'identifier' directory

The reason I added gconf too is that we've found a way to read gconf from cron scripts, so I suspect Sugar may revert to using plain gconf.

The patches are a in the repo above, and I've cut an RPM for the F11 based builds, so should work on SoaS.
http://dev.laptop.org/~martin/public_rpms/f11/ds-backup-client-0.11-1.olpc3.noarch.rpm

Can you test and confirm?

comment:22 Changed 10 years ago by hamiltonchua

Martin,

Can you kindly correct line 145 on ds-backup.py. The path should be ".sugar/default/identifiers" with an "s".

Aside from the above, things are looking good, I am able to get backups to the XS using your new rpm when I run it from the command line.

Is there a way to fix the cron ? By default it assumes the user that executes the script to be "olpc", I wonder if there's a way to modify it to use "liveuser" when the rpm is installed, I'm not sure if there's a reliable way to detect if the sugar install is on SoaS or otherwise.

comment:23 Changed 10 years ago by sdz

Great work, folks! :)

Martin, is there any chance of getting the src.rpm?

comment:24 Changed 10 years ago by martin.langhoff

Fixed. RPM and SRPM and at
http://dev.laptop.org/~martin/public_rpms/f11/

WRT the cron thing... I frankly don't know. If there is a reasonable solution, I will be happy to apply it.

comment:25 Changed 10 years ago by martin.langhoff

This is also affected by #1098

comment:26 Changed 10 years ago by RafaelOrtiz

  • Milestone changed from 0.86 to 0.88

comment:27 Changed 10 years ago by sdz

  • Milestone changed from 0.88 to 0.86

I'm not sure why the milestone was changed here. From what I know, the Sugar specific changes have been already committed and it's just about pushing one package to SoaS. #1098 has, from what I know, also been fixed in 0.84.7 (which dsd just pushed today) and SoaS will even ship 0.86.x, so I don't think this is an issue.

comment:28 Changed 10 years ago by sdz

Oh, I forgot to say this: Martin, thanks for the src.rpm! :)

comment:29 Changed 10 years ago by RafaelOrtiz

ok thanks for the clarification sebastian.:)

comment:30 Changed 10 years ago by tomeu

  • Keywords r! added; r? removed

It's not clear to me if there's anything to be reviewed or done for this ticket. I'm taking out of the review queue for now, please update this ticket and close it if possible. Thanks!

comment:31 Changed 9 years ago by bernie

Tch and Esteban have been working on a backup UI.

comment:32 Changed 9 years ago by bernie

  • Cc tch jsaldivar added

comment:33 Changed 9 years ago by bernie

  • Cc jasg added; jsaldivar removed

comment:34 Changed 6 years ago by dnarvaez

  • Component changed from journal to sugar

comment:35 Changed 6 years ago by dnarvaez

  • Component changed from Sugar to Sugar on a Stick (SoaS)

comment:36 Changed 6 years ago by dnarvaez

  • Milestone 0.86 deleted

Milestone 0.86 deleted

Note: See TracTickets for help on using tickets.