June 2, 2009

Ark Linux to adopt ZYpp

Ark Linux is becoming the first third-party distro other than openSUSE to adopt ZYpp as its package manager.

April 29, 2009

Making package management even faster?

Joshua Bahnsen wrote on the yum mailing list (summarizing):

Is there support for compression types other than gzip for the yum metadata? For example, bzip2 or LZMA? I am keeping track of 16 RHEL channels, using createrepo with the standard gzip I am totaling 1.4 GB of metadata. Compressing those same XML documents with LZMA yields a total of 140 MB. That’s 10x savings overall, I think that’s worth a look.

James Antill asked some questions, and also pointed some issues:

These we can probably try and help with, but we’ve been asking and waiting for 12+ months for RHN and CentOS to move to generating .sqlite files server side. So I wouldn’t bet that we can help in the general case, quickly. Plus any client side support for lzma probably wouldn’t get into 5.x until at least 5.5 (more likely 5.6 or 5.7). So realistically you are targeting Fedora and 6.x for a change like this.

For Fedora the main issue is that they use sqlite version of the metadata generated on the server side, therefore they are more interested in how those files compress.

openSUSE still downloads the raw metadata. Mostly because our sat-solver is really fast converting it to the solv cache. We have thought about shipping solv files in the metadata some day, but we still haven’t look at that in detail.

However I wanted to see how it could make a difference for openSUSE by using lzma when compressing the metadata. I took the 11.0 update repository, as it has quite a lot of stuff in it, which makes it big to download when it changes.

For that, only a minimal patch to /usr/bin/repo2solv.sh is needed. You can get that patch here.

Here are my metadata size results:

I also looked to how fast the parsing of the metadata goes. For that I parsed each file 3 times and averaged each time component. I dropped disk caches before.

As you can see, the repo is 4x smaller and parses 2x faster, which is still very nice(the 10x gains seems to be for files like other.xml which we don’t download).

You can get the data from here.

A change like this is still challenging:

  • We need upstream support to keep repos compatible
  • It can only be done in Factory, and people need to update the software management stack before the format is changed.

What do you think?

August 18, 2008

zypper feature: source and automatic build dependencies install

There are features that are present since some time, but people just don’t know about it. This time I will mention one that is very useful to developers.

Imagine you are a contributor to foo, which is also available to the distribution as foo.rpm. However, you install the distro from packages, but compile your foo source tree by hand. Compiling foo requires lot of packages.

zypper to the rescue. If foo is in the distribution, make sure you have the main distribution repository added, plus the source repository. Source packages have a nice feature: their Requires, are the BuildRequires of the binary package.

The source install command (si) has a -d option to install only the dependencies, and not the source package itself. So lets say we want to compile PackageKit, lets install all the compile dependencies:

# sudo zypper si -d PackageKit
Reading installed packages...

The following NEW packages are going to be installed:
  opensp openjade gtk-doc docbook_3 docbook-dsssl-stylesheets sqlite-devel

Then I can go and compile:

PackageKit> ./autogen.sh --libdir=/usr/lib64
  --prefix=/usr --enable-developer --with-security-framework=dummy 
  --sysconfdir=/etc --with-dbus-sys=/etc/dbus-1/system.d
  --enable-zypp --with-default-backend=zypp

Note, you do need the source repository added in order for this to work, otherwise zypper will not be able to find a matching src.rpm package to read the dependencies from.

June 6, 2008

openSUSE 11 RC1, Mercedes Benz and Volkswagen.

DistroWatch praising zypper:

During my usage of Zypper over the past couple of weeks, I’ve learnt to appreciate the convenience it provides to the end user. There is not doubt in my mind that Zypper is the most powerful package management utility available today

Even further goes CNET with their review.

I’d have to say that next to Mac, this is probably the most beautiful installer program I have ever seen. Beauty isn’t just skin deep, however — a lot of time and work has been invested to make the SUSE installer faster and easier to use.

You can read more about the installer and a short interview with Stephan Kulow here.

And the last great paragraph:

If Ubuntu is a Volkswagen, then OpenSUSE is a Mercedes-Benz.

May 25, 2008

User’s feedback

I got some feedback from users about managing software.

Raúl Moratalla. who has been blogging about using 11.0 ZYpp/YaST in 10.3, suggests two changes:

  • Updated packages filter.

Adding that feature is really easy. However, 11.0 is almost out.

  • New packages filter

This shouldn’t be impossible, but I don’t have an idea on how to do it. I think we don’t have the information on when a package appears in a repository, and even worse, we don’t keep information about the previous refresh when finishing one.

John Tomas asks and suggests an easier way to remove applications. I think the use case is valid, but I don’t like the exact proposal, specifically, I don’t like user interfaces with trees. And adding more check boxes for repo removal clutters the interface more. Moreover, I think this usecase is just part of a more generic one, how to install applications, without worrying about “packages”. This problem is already being solved by, for example, Ubuntu Appinstaller.

This problem could be solved in various way.

  • Packages could be “tagged” as such.

However that requires changing the current world, therefore it won’t work.

  • Do a .desktop file -> package mapping.

Ubuntu appinstaller does this by scanning the complete distribution archive, and generating special metadata from it. This metadata is distributed as a package called appinstaller-data. This method is sane, but the approach of scanning the distribution archive won’t work with 3rd party repositories.

Stephan Binner has a prototype for the same concept on openSUSE, called app-installer.

Any other ideas on how to solve this problem?

March 14, 2008

ZYpp stack on other distributions?

Keeping a permanent build of svn on the build service motivated me to try to build it on other non-SUSE based distros. On the way I discovered not only simple things as different package names, but linking problems, compile errors, etc.

In the past it did not make much sense to try libzypp outside of SUSE as we were trying to catchup with other tools speed. But now that libzypp speed outperforms any other tool in speed, while keeping it complete set of features, we may start thinking about taking over the world. May be other distros want to use libzypp, or why not, the small and powerful sat-solver library alone.

So, since today I got the first successful build of sat-solver/libzypp/zypper on Fedora 8. Mandriva is also close, but I still get a compiler error I should not get when compiling the rpm backend.

At the beginning, it may be a consuming effort, but once it is a continuous process, our software should adopt a more agnostic view of the world and just compile there out of the box. If anyone has the time to test it, I would be interested in the results ;-)

By the way, our colleagues at internal tools finished processing the FOSDEM talk’s videos we had in the openSUSE Developer’s room. You can find the ogg video files here. Also you can watch on google video here.