Automate pulling of weekly test builds for SoaS

[mchua]

We're currently pulling weekly test builds onto sunjammer, in /srv/www-sugarlabs/download/soas/test, and would like to automate the process.

I'm not entirely sure how to write this cron job, but every Thursday at 11:59:59 GMT, we would like to...

1. wget ​http://alt.fedoraproject.org/pub/alt/nightly-composes/soas/soas-i386-'''FOO'''.iso # Get the latest i386 nightly compose from ​http://alt.fedoraproject.org/pub/alt/nightly-composes/soas/. There should only be one soas-i386 image. (FOO is whatever the datetime stamp of that image is... for instance, it may be 20060623.16 - store this variable for later use)
2. ln -s soas-i386-FOO.iso soas-i386-test-latest.iso # Update the symlink to point to the new image.
3. wget ​http://alt.fedoraproject.org/pub/alt/nightly-composes/soas/CHECKSUM-i386 # The SHA256 checksum for that iso - it will always be at this link
4. mv CHECKSUM-i386 soas-i386-FOO.sha # Rename the checksum so it matches the iso
5. ln -s soas-i386-FOO.sha soas-i386-test-latest-checksum.sha # Update the symlink to point to the new checksum

In the end, ​http://download.sugarlabs.org/soas/test/ should have a new image-checksum pair under the image's original datetime-stamp-based name, and the symlinks for ​http://download.sugarlabs.org/soas/test/soas-i386-test-latest.iso and ​http://download.sugarlabs.org/soas/test/soas-i386-test-latest-checksum.sha should be updated to point to that newly-downloaded image-checksum pair.

[lukasz.jagiello]

Simple version:

#!/bin/bash

LATEST_ISO_NAME=`curl -s http://alt.fedoraproject.org/pub/alt/nightly-composes/soas/ | grep "soas-i386" | cut -d ">" -f 2 | cut -d "<" -f 1`

wget -o /dev/null http://alt.fedoraproject.org/pub/alt/nightly-composes/soas/$LATEST_ISO_NAME
wget -o /dev/null http://alt.fedoraproject.org/pub/alt/nightly-composes/soas/CHECKSUM-i386

if [ -f soas-i386-test-latest.iso ]
then
        unlink soas-i386-test-latest.iso
fi
ln -s $LATEST_ISO_NAME soas-i386-test-latest.iso

mv CHECKSUM-i386 ${LATEST_ISO_NAME/iso/sha}
if [ -f soas-i386-test-latest-checksum.sha ]
then
        unlink soas-i386-test-latest-checksum.sha
fi
ln -s ${LATEST_ISO_NAME/iso/sha} soas-i386-test-latest-checksum.sha

[ctyler]

Proposed script at ​http://chris.tylers.info/files/soas-wget-test

[sascha_silbe]

Has the above actually been added to the crontab of some user or is this just a proposal for how to do it?

[sdz]

Replying to sascha_silbe:

Has the above actually been added to the crontab of some user or is this just a proposal for how to do it?

Thanks everybody who worked on that! No, I don't think this has yet been implemented.

[sascha_silbe]

There's already a user soas that anyone in group soas can change to (using sudo), so you should be able to add this to crontab (of user soas) yourself. Is there anything you still need from the infrastructure team? If not, please change the component to SoaS (thanks!).

[mchua]

We just need one more bit of unblocking - the ability to change to the soas user (at least for me, then I can take this the rest of the way).

mchua@sunjammer:~$ sudo su soas
[sudo] password for mchua: 
Sorry, user mchua is not allowed to execute '/bin/su soas' as root on sunjammer.sugarlabs.org.

(It is entirely possible I'm being dumb here, and should be doing something else in order to become the soas user on sunjammer.)

[sascha_silbe]

Replying to mchua:

mchua@sunjammer:~$ sudo su soas

Please try

sudo -u soas -i

to get an interactive login shell or

sudo -u soas echo foobar

to run a single command as user soas.

Running

sudo su soas

means using two different tools to do exactly the same thing twice (just with different users - first root, then soas). It would require us to grant root access, even if just intermediate. There were some nice vulnerabilities in login et al. in the past that used a similar attack vector.

[mchua]

sdz wants to learn about crontabs, so I'm writing this up here in case it helps someone later on.

Change to the soas user as described above.

sudo - u soas -i

Make a directory for the script, go into it, get the script, make it executable, and change its name (the last step mostly so I'd remember better what it was). Run the script to make sure it works (may take a while, is pulling down an entire .iso).

mkdir bin
cd bin
wget http://chris.tylers.info/files/soas-wget-test
chmod +x soas-wget-test
mv soas-wget-test soas-weekly-test-wget

Edit the crontab.

crontab -e

This will open the crontab (a textfile) for editing. Add the following line:

59   23   *   *   4   ~/bin/soas-weekly-test-wget

The last entry is the script we want to run (soas-weekly-test-wget). The first two run it at 23:59 (11:59pm), and the 4 means "every Thursday" (Sunday is either 0 or 7, you can figure out the rest of the days of the week from that.)

[sdz]

This is awesome, thank you Mel! I just did this so we should be good! Closing. We'll know on Thursday whether it worked! :)