Monday, July 05, 2010

» Even shorter openSUSE repository URLs: guessing the distro

I updated the documentation about even shorter and more portable URLs to openSUSE build service repositories. The thing is, the zypp stack (zypper and YaST2) send the version of the openSUSE it is running on as an HTTP header when performing an "add repo" operation. That is now used on the server side when a version is not specifically mentioned in the URL. e.g.: zypper ar -r http://r.opensu.se/network:utilities.repo

Labels: ,

Friday, July 02, 2010

» Shorter .repo URLs

A few days ago, I blogged about using r.opensu.se for shorter package repository URLs, including passing e.g. http://r.opensu.se/network:utilities/11.2/r to zypper ar -r to add a repository. Turned out it didn't work, because zypper wouldn't follow the redirect. So I rewrote that part by handing over such URLs to a CGI script that reverse proxies the .repo file from http://download.opensuse.org/repositories/ (it fetches the proper .repo URL and passes the result to zypper). To cut the long story short, this stuff actually works now :): zypper ar -r http://r.opensu.se/network:utilities/11.2/r

Labels: , ,

Monday, November 23, 2009

» Packman repository for KDE3 on openSUSE 11.2

openSUSE 11.2 doesn't ship KDE3 anymore, because it is barely maintained upstream. But you may still install and use KDE3 as your desktop environment as the packages are available in the openSUSE Build Service. Which is why we decided to set up an additional repository that contains the Packman packages that require KDE3. If you want e.g. kde3-amarok, add the following repository: http://ftp.skynet.be/pub/packman/suse/kde3/11.2/packman-kde3.repo

Labels: , , ,

Friday, November 20, 2009

» osc and self-signed certificates the secure way

Obviously, a much more secure option to use the osc Build Service command-line client with your own server instances that have a self-signed certificate is to actually add that server certificate to the list of blessed ones. To do so, do as follows, all of that as root in a shell:
umask 0033
cd /etc/ssl/certs
echo | openssl s_client -showcerts \
-connect YOUR.SERVER:SSL_PORT \
2>&1 |sed -n '/^-----BEGIN/,/^-----END/p' \
> SOME_FILENAME.pem
c_rehash .
Once you've done the above, osc will no longer complain about an invalid (server) peer certificate, as the certificate of your Build Service instance server is now accepted as a valid one. Note that in order to be really secure, you need to ask an administrator of the server what the fingerprint (also called a hash) of the server certificate actually is, in order to compare and make sure you're not blessing an already compromised server or are already subject to a man-in-the-middle attack. To do so, an user who has root access to the server should run the following command, obviously as root, and then give you the output of that command, ideally through a secure communication channel, or through several channels (e.g. SMS mobile text message and/or a signed email and/or IRC from a registered nickname):
cat /etc/ssl/private/lighttpd.pem\
| sed -n '/^-----BEGIN CERT/,/^-----END CERT/p'\
| sha1sum | cut -f1 -d" "
You can then run the following command and compare the output with what that trusted user has given you:
fp=$(sha1sum /etc/ssl/certs/SOME_FILENAME.pem\
| cut -f1 -d" ")
server="...paste what the trusted user has given you here..."
test "$fp" = "$server" && echo OK || echo COMPROMISED
If you have already patched your osc client as I described in my previous post, you can undo that patch with the following code (again, as root in a shell):
umask 0077
cd $(dirname $(rpm -ql osc|grep /conf.py\$))
mv conf.py.orig conf.py
chmod 0644 conf.py

Labels: , ,

» Patch for osc and self-signed certificates

If you are using your own Build Service instance as we do at Packman and if you are using a self-signed certificates for HTTPS, then this patch might come handy for you as well. Note that it disables the peer certificate validation in osc, so don't use that with Build Service instances you down trust, and only if you are willing to take the risk of not detecting man-in-the-middle attacks. In order to apply the patch, simply copy and paste the following block of shell code in a terminal where you are running as root:
cd $(dirname $(rpm -ql osc|grep /conf.py\$))
umask 0077
curl -s -o /tmp/o.patch \
http://linux01.gwdg.de/~pbleser/files/osc/\
osc-disable-peer-cert-check.diff
test $(sha1sum /tmp/o.patch | cut -f1 -d" ") = \
9812ef2a18c02799b9e33d1402509c3217c39194 \
&& patch -p1 -b -z.orig -i /tmp/o.patch
UPDATE: note that in my following blog-post I describe a secure option.

Labels: , ,

Sunday, October 25, 2009

» Send mail to openSUSE Build Service project maintainers

Just wrote a little script that parses the output of osc meta prj or osc meta pkg, grabs the list of users in there and pokes the openSUSE Build Service API again to retrieve their email address. All that is then used to call Thunderbird in compose mode with the appropriate list of email addresses (and real names, when available). It also uses a cache file to speed things up. Examples:
osc meta prj security | omail
osc meta pkg security keychain | omail
You can download that (Perl) script from here. Simply copy it to ~/bin or /usr/local/bin and do a chmod 755 on it. Of course, you might as well add support for your favourite MUA (email client), which should be fairly trivial, given that the script already does the dirty work ;-) An alternative would be to re-author that script as an osc plugin.

Labels: ,

Thursday, September 10, 2009

» Scripts for OBS monitoring

(yes, I'm back from holidays) Marcus "}-Tux-{" Hüwe and I have been busy the last weeks to set up an openSUSE Build Service instance for Packman. While doing so, I wrote a few little scripts to monitor what is going on as far as building, the scheduler queue and such are concerned. The most interesting ones are obs-building, that displays what is currently being built on all workers: obs-building sample as well as obs-queue, that displays what is currently being scheduled for build: obs-queue sample and finally obs-top, a simple hack that uses watch to display combined output from top, obs-building and obs-queue: obs-top sample

Labels: ,

Friday, July 24, 2009

» osc faillog, for the lazy

As blogged about a while ago, I also wrote a very crude and simple osc plugin for the lazy people (like me!) that applies some simple heuristics to pick the most informative build failure log, to avoid having to type the target distribution and architecture as parameters to osc bl Well, it's now available as a package, namely osc-plugin-faillog To install, add the home:pbleser:osc repository (pick a subdirectory that suits your distribution), and install the package osc-plugin-faillog To use it, just type osc f while being in a package sandbox directory, exactly as with osc bl (but without having to type e.g. osc bl openSUSE_11.1 x86_64 :)). The "algorithm" is quite simple:
  • if factory (SUSE_Factory or openSUSE_Factory, or anything else that has factory in its distribution name) failed, it will display the log for that
  • if not, then it will pick the most recent openSUSE distribution target that failed (i.e. openSUSE_11.1 > openSUSE_11.0, etc...)
From my experience, that (almost?) always makes sense, as the more recent the build target is, the tougher GCC is, and the pickier the rpmlint rules. Yes, it's simple and stupid, but it works -- at least for me. And indeed, it doesn't work for non-openSUSE distribution targets as of now.

Labels: , ,

» osc niceresults, now with icons

I've already written about my osc-plugin-niceresults package, and now I added some nifty UTF-8 "icons" that help quickly understanding the clutter that the output of osc r can be at times, especially when you have a lot of build targets. So, what does it look like ? (make sure to have your browser on UTF-8 ;))
  Repository    │ Arch   │ Status
────────────────┼────────┼────────────────
 openSUSE_10.3 │ i586   │ succeeded      
 openSUSE_10.3 │ x86_64 │ succeeded      
 openSUSE_11.0 │ i586   │ succeeded      
 openSUSE_11.0 │ x86_64 │ succeeded      
 openSUSE_11.1 │ x86_64 │ succeeded      
 openSUSE_11.1 │ i586   │ succeeded      
 SLE_10        │ i586   │ disabled       
 SLE_10        │ x86_64 │ disabled       
  SUSE_Factory  │ i586   │ expansion error
  └─▶ nothing provides autoconf >= 2.62 needed by automake
  SUSE_Factory  │ x86_64 │ expansion error
  └─▶ nothing provides autoconf >= 2.62 needed by automake
In order to install that osc plugin, add the home:pbleser:osc repository (pick a subdirectory that suits your distribution), and install the package osc-plugin-niceresults To use the plugin, do as with osc results (or osc r), but instead, invoke osc niceresults (or the shorthand form osc nr).

Labels: , ,

Tuesday, June 09, 2009

» osc plugin for the lazy

After my first osc plugin that does some output colouring, I've written another one for the lazy people like me. Mind you, osc is the command-line client for the openSUSE Build Service, and it is easy to extend through plugins. My typical workflow, when a build fails, is to do a osc cr to see the list of succeeded and failed builds and then, if applicable, do another osc bl to see the actual build output on a failed target, having to pass the distribution and architecture as parameters, e.g.:
osc bl openSUSE_Factory x86_64
Now, as said, I'm a lazy person, and I hate to do or type stuff a script could do for me. In the list of failed builds, I almost always go for the build log of the newest openSUSE version as it usually represents the most difficult target (more rpmlint tests, newest GCC, etc...), which is typically openSUSE_Factory. So why would I have to type all that, right ? ;) The following osc plugin provides a simple osc f ("f" for "fail" ;P) command that looks at the list of targets, keeps the failed ones and performs some very crude heuristics to pick the most interesting one, which is openSUSE_Factory or, if that one succeeded, the newest openSUSE version. The plugin is available as a .py file -- simply download it and install it into /var/lib/osc-plugins/, make it executable (chmod 755) and that's all. If people use and like it, I'll make an RPM. Promised.

Labels: , ,

Friday, April 10, 2009

» RSS build notifications from OBS

One of the annoying things with the openSUSE Build Service ("OBS") is that while it is acceptable that it can take a while for the build grid to process your package, it is tedious to poll the results with "osc r" (or my contributed, nicely coloured "osc cr") again and again. Now, the openSUSE Build Service has a notification backend called "Hermes" (named so after the Greek mythology, Hermes being the messenger of the gods) that is triggered by OBS events such as build completion. I personally don't like receiving bulks of emails to tell me that a build has succeeded or failed, my IMAP being already sufficiently crowded. But Hermes also publishes RSS feeds, including by user. In order to follow those in your preferred RSS client (I use akregator, but there are many others, including Firefox' "live bookmarks" feature), go to hermes.opensuse.org, log in with your openSUSE iChain account, and there you'll have a page where you can configure the notifications for your builds. I went for enabling "Build Success" and "Build Failure" as well as picking "Web / RSS newsfeed" from the list box on the right. When done, you get a list of your feeds here: https://hermes.opensuse.org/feeds/personal (or by clicking on "My Feeds" in the menu in the left navigation section).

Labels: , ,