Opened 7 years ago

Closed 7 years ago

#2058 closed task (fixed)

Automate pulling of weekly test builds for SoaS

Reported by: mchua Owned by: sascha_silbe
Priority: Unspecified by Maintainer Milestone: Unspecified
Component: infrastructure Version: Unspecified
Severity: Unspecified Keywords:
Cc: sdz, pbrobinson Distribution/OS: Unspecified
Bug Status: New

Description

We're currently pulling weekly test builds onto sunjammer, in /srv/www-sugarlabs/download/soas/test, and would like to automate the process.

I'm not entirely sure how to write this cron job, but every Thursday at 11:59:59 GMT, we would like to...

  1. wget http://alt.fedoraproject.org/pub/alt/nightly-composes/soas/soas-i386-'''FOO'''.iso # Get the latest i386 nightly compose from http://alt.fedoraproject.org/pub/alt/nightly-composes/soas/. There should only be one soas-i386 image. (FOO is whatever the datetime stamp of that image is... for instance, it may be 20060623.16 - store this variable for later use)
  2. ln -s soas-i386-FOO.iso soas-i386-test-latest.iso # Update the symlink to point to the new image.
  3. wget http://alt.fedoraproject.org/pub/alt/nightly-composes/soas/CHECKSUM-i386 # The SHA256 checksum for that iso - it will always be at this link
  4. mv CHECKSUM-i386 soas-i386-FOO.sha # Rename the checksum so it matches the iso
  5. ln -s soas-i386-FOO.sha soas-i386-test-latest-checksum.sha # Update the symlink to point to the new checksum

In the end, http://download.sugarlabs.org/soas/test/ should have a new image-checksum pair under the image's original datetime-stamp-based name, and the symlinks for http://download.sugarlabs.org/soas/test/soas-i386-test-latest.iso and http://download.sugarlabs.org/soas/test/soas-i386-test-latest-checksum.sha should be updated to point to that newly-downloaded image-checksum pair.

Change History (11)

comment:1 Changed 7 years ago by sdz

  • Cc sdz added

comment:2 Changed 7 years ago by pbrobinson

  • Cc pbrobinson added

comment:3 Changed 7 years ago by lukasz.jagiello

  • Resolution set to fixed
  • Status changed from new to closed

Simple version:

#!/bin/bash

LATEST_ISO_NAME=`curl -s http://alt.fedoraproject.org/pub/alt/nightly-composes/soas/ | grep "soas-i386" | cut -d ">" -f 2 | cut -d "<" -f 1`

wget -o /dev/null http://alt.fedoraproject.org/pub/alt/nightly-composes/soas/$LATEST_ISO_NAME
wget -o /dev/null http://alt.fedoraproject.org/pub/alt/nightly-composes/soas/CHECKSUM-i386

if [ -f soas-i386-test-latest.iso ]
then
        unlink soas-i386-test-latest.iso
fi
ln -s $LATEST_ISO_NAME soas-i386-test-latest.iso

mv CHECKSUM-i386 ${LATEST_ISO_NAME/iso/sha}
if [ -f soas-i386-test-latest-checksum.sha ]
then
        unlink soas-i386-test-latest-checksum.sha
fi
ln -s ${LATEST_ISO_NAME/iso/sha} soas-i386-test-latest-checksum.sha

comment:4 Changed 7 years ago by ctyler

  • Bug Status changed from Unconfirmed to New

comment:5 follow-up: Changed 7 years ago by sascha_silbe

Has the above actually been added to the crontab of some user or is this just a proposal for how to do it?

comment:6 in reply to: ↑ 5 Changed 7 years ago by sdz

  • Resolution fixed deleted
  • Status changed from closed to reopened

Replying to sascha_silbe:

Has the above actually been added to the crontab of some user or is this just a proposal for how to do it?

Thanks everybody who worked on that! No, I don't think this has yet been implemented.

comment:7 Changed 7 years ago by sascha_silbe

There's already a user soas that anyone in group soas can change to (using sudo), so you should be able to add this to crontab (of user soas) yourself. Is there anything you still need from the infrastructure team? If not, please change the component to SoaS (thanks!).

comment:8 follow-up: Changed 7 years ago by mchua

We just need one more bit of unblocking - the ability to change to the soas user (at least for me, then I can take this the rest of the way).

mchua@sunjammer:~$ sudo su soas
[sudo] password for mchua: 
Sorry, user mchua is not allowed to execute '/bin/su soas' as root on sunjammer.sugarlabs.org.

(It is entirely possible I'm being dumb here, and should be doing something else in order to become the soas user on sunjammer.)

comment:9 in reply to: ↑ 8 Changed 7 years ago by sascha_silbe

Replying to mchua:

mchua@sunjammer:~$ sudo su soas

Please try

sudo -u soas -i

to get an interactive login shell or

sudo -u soas echo foobar

to run a single command as user soas.

Running

sudo su soas

means using two different tools to do exactly the same thing twice (just with different users - first root, then soas). It would require us to grant root access, even if just intermediate. There were some nice vulnerabilities in login et al. in the past that used a similar attack vector.

comment:10 Changed 7 years ago by mchua

sdz wants to learn about crontabs, so I'm writing this up here in case it helps someone later on.

Change to the soas user as described above.

sudo - u soas -i

Make a directory for the script, go into it, get the script, make it executable, and change its name (the last step mostly so I'd remember better what it was). Run the script to make sure it works (may take a while, is pulling down an entire .iso).

mkdir bin
cd bin
wget http://chris.tylers.info/files/soas-wget-test
chmod +x soas-wget-test
mv soas-wget-test soas-weekly-test-wget

Edit the crontab.

crontab -e

This will open the crontab (a textfile) for editing. Add the following line:

59   23   *   *   4   ~/bin/soas-weekly-test-wget

The last entry is the script we want to run (soas-weekly-test-wget). The first two run it at 23:59 (11:59pm), and the 4 means "every Thursday" (Sunday is either 0 or 7, you can figure out the rest of the days of the week from that.)

comment:11 Changed 7 years ago by sdz

  • Resolution set to fixed
  • Status changed from reopened to closed

This is awesome, thank you Mel! I just did this so we should be good! Closing. We'll know on Thursday whether it worked! :)

Note: See TracTickets for help on using tickets.