Duncan Mac-Vicar P.

You may remember, openSUSE 11.1 shipped with experimental (and by default disabled) software management stack improved HTTP download support, based on aria2c, which gives additional capabilities to take advantage of our famous download infrastructure, on which Peter has been working on.

The ultimate goal of this is to provide a pleasant experience to our advanced users tracking Factory and doing frequent dist-upgrades (feature 305634), and this includes other cool improvements, like the one announced some days ago, based on Stephan Kulow and Michael Matz’s build compare techniques, and of course the one I describe here: robust mirror handling and improved download experience (feature 302923).

The next step for the aria2c backend is to enable it as a default. Everything was working more or less smooth, except file existence detection, which is needed for the repository probing components. This part of the code was implemented as a “always true” stub.

As the MediaCurl code was already a bit messy, duplication was started to be everywhere, and actually MediaAria2c already had quite a lot of code duplicated from MediaCurl. However, a big refactoring required studying the code in deep.

Thanks to the Asian restaurant near the office, which did not clean their surroundings from the ice created from the past snowy days in Nürnberg, I had to stay some days at home (medical order) due to a very ugly fall that made my left arm useless (except for some typing), which gave me the quiet opportunity to study/read that code and figure out a solution.

I tried tons of different things. Even using curl command line from the aria backend only to check file existence, but at the end the problems I encountered (and figuring out some missing features from aria2c) pushed me in the direction of making the aria2c backend a particular case of the curl backend, and therefore I could reuse features from curl, without worrying about reentrant access to media attach points or other stuff. This reduced the code of the aria2c backend considerably. MediaCurl is now much smaller (about 200 lines of duplicated code removed), so we can add range support (I will explain this in a future post) to the API.

On the way to this refactoring, the following features/options (zypp.conf) got implemented:

##
## Maximum number of concurrent connections to use per transfer
## This setting is only used if more than one is possible
## Setting it to a reasonable number avoids flooding servers
##
# download.max_concurrent_connections = 2

The option above only works with the aria2c backend. Not with pure curl.

##
## Sets the minimum download speed (bytes per second)
## until the connection is dropped
## This can be useful to prevent security attacks on hosts by
## providing updates at very low speeds.
##
## 0 means no limit
##
# download.min_download_speed = 0

The option above can be used to prevent a (very theoretical) attack described in this report, where the bad guys will send data very slowly to the client to keep it unpatched. This option should work with both aria2c and curl backends.

Also, we implemented the following options for those people sharing a connection or needing to limit the connection usage somehow:

## Maximum download speed (bytes per second)
## 0 means no limit
# download.max_download_speed = 0

Also, the following option is useful for people doing the dist-upgrade unattended, so they can set it to try hard, or just to keep trying without interaction:

## Number of tries per download which will be
## done without user interaction
## 0 means no limit (use with caution)
# download.max_silent_tries = 5

These features are in libzypp 6.1.0, Factory has 5.11.0 right now. I plan to submit to Factory in the following days (after basic testing). Once aria2c is the default, it can be disabled using ZYPP_ARIA2C=0 which will give you the basic curl back.

Of course, this may result in new bugs or problems. So be prepared!

The last weeks I heard twice about the idea of having something like debconf to configure package default settings. Now I just notices a blog post from Lars about it.

When such ideas appear, I always pay attention, because there we may miss some points:

• Look out of the box. We end with our own custom way to do things, sometimes really coupled with YaST (which is very different to integrated with YaST).
• No research is done about existing solutions, their mistakes and advantages.
• One weakness is not enough to invent something new. People forget the missing feature fast. Compatibility brings more value than one missing feature: You gain tools and existing knowledge. ** For example, we switched to the updateinfo.xml format, which does not support dependencies. People complained, but till now nobody has missed them. However the value is that you can use tools like Bodhi with openSUSE.
• You can still innovate, and add value, by integrating it with your already valuable stuff.

So lets put this feature in perspective. The main use case (extracted from Lars post is):

• Allow the package to define settings
• Ask for those settings at installation (or later).

There is even a “mockup” or how it could look like in a new xml format.

Lets look at debconf, just an overview, to see what it does:

• It supports questions with the following types: string boolean select multiselect note text password
• The myth that it break non-interactive install is not completely true. There is a noninteractive “frontend”.
• It has frontends for command line, KDE, Gnome, and a ncurses/dialog like.
• Packages provide templates, like:

Template: foo/like_debian
Type: boolean
Description: Do you like Debian?
 We'd like to know if you like the Debian GNU/Linux system.

Template: foo/why_debian_is_great
Type: note
Description: Poor misguided one. Why are you installing this package?
 Debian is great. As you continue using Debian, we hope you will
 discover the error in your ways.

• Packages provide scripts. Script get a minimal and simple API to get the values of the answered templates, and apply them. So you can write them in sh and perl (and may be others).

#!/bin/sh -e

# Source debconf library.
. /usr/share/debconf/confmodule

# Do you like debian?
db_input medium foo/like_debian || true
db_go

# Check their answer.
db_get foo/like_debian
if [ "$RET" = "false" ]; then
    # Poor misguided one..
    db_input high foo/why_debian_is_great || true
    db_go
fi

• There is even the concept of “shared settings”:

Template: shared/window-manager
Type: select
Choices: ${choices}
Description: Select the default window manager.
 Select the window manager that will be started by default when X
 starts.

So, the question is, if till now nobody has additional requirements, why invent a new xml format, or force people to code wizards in ycp?

• Why not use exactly the SAME template format. Hey packager! You can copy the templates from Debian, just as you copy patches from Gentoo today
• Why not add the glue with YaST in the script part (so the scripts would be sh, perl or ycp with a MINIMAL api)

If the template format is the same, we could also expect Fedora people could also start using it (of course we haven’t yet talked to them, and then we ask ourselves why every distribution has its own

If the scripts are integrated with YaST, other distributions can implement them in a different way, but with the same minimal API.

Also, hiding the whole YaST infrastructure keeps the concept simple and prevent packagers to start doing evil stuff from these scripts, and even better: the templates are not programs, or ycp maps, but documentation/knowledge which happens to be read from YasT code.

While being in Barcelona like a year ago, I was playing with Android’s SDK and trying to implement a todo-list that could make a phone vibrate when you walk nearby the grocery store, and you had a todo item saying “remind me about buying tomatoes if I walk nearby”.

When I googled for it, I found Naggie (link dead), but it was dead, which was quite strange. After googling more, I think I found out why the site may have been dead.

Then I had the opportunity to play with an Android phone for first time (Martin’s phone), and I asked him whether such application existed. Well Martin, it does, but I could not find the download link.

However, today I found a free app for BlackBerry called got-2. I installed it. However, it claims it has some problems with OS 4.5, which is the one I use. Lets see how it works.