Duncan Mac-Vicar P.


libZYpp, torrents and metalinks

with 5 comments

Using only http for transfering files will be a thing from the past very soon. We know all the problems distributing content to large audiences companies face, and Peter Poeml knows it very well too.

So Peter mentored Gerard Farràs during the last Summer of Code, and the result was a ZYpp media handler that uses aria2 to download files, instead of curl. (May be aria uses curl for http then, I have no idea ;-) ).

aria2 is a utility for downloading files. The supported protocols are HTTP(S), FTP, BitTorrent (DHT, PEX, MSE/PE), and Metalink. It can download a file from multiple sources/protocols and tries to utilize your maximum download bandwidth. It even supports downloading a file from HTTP(S)/FTP and BitTorrent at the same time, while the data downloaded from HTTP(S)/FTP is uploaded to the BitTorrent swarm. Using Metalink’s chunk checksums, aria2 automatically validates chunks of data while downloading a file like BitTorrent.

The benefits is that aria knows about bittorrent, metalinks, etc, so this open the door for much better download failover in the future.

As we are aready on beta phase and we can’t switch such an important component, I merged the code, but disabled. That means ZYpp still uses curl, unless you enable the environment variable ZYPP_ARIA=1. That way you can play with it while it matures for 11.2 or something. There should be lot of features missing in the aria2 handler, so please have that in mind.

Thanks to Peter and Gerard for the handler.

Written by duncan

September 30th, 2008 at 11:31 am

Posted in Software

Tagged with ,

5 Responses to 'libZYpp, torrents and metalinks'

Subscribe to comments with RSS or TrackBack to 'libZYpp, torrents and metalinks'.

  1. [...] y metalinks. Posted in October 1st, 2008 by toomany in libzypp, opensuse, zypper Tal y como indica Duncan Mac-Vicar, el uso del HTTP para transferir ficheros quedará pronto obsoleto. Peter Poeml, ha asistido como [...]

  2. Wow, this sounds awesome. The use of bittorent and metalinks will place zypper ahead of any package manager. I like the fact that YaST2 devs are not rushing to implement it on openSUSE 11.1. Hopefully the aria2 integration will be “bug free” and ready to use by openSUSE 11.2

    jrdls

    11 Oct 08 at 8:20 pm

  3. [...] when one exists in about 0.61% of install attempts. Then you have: - Support for metalinks (Duncan Mac-Vicar P.

  4. Hi again duncan, what’s the status of this feature? I have to admit this is an exciting feature to me.

    jose ricardo

    13 Nov 08 at 6:17 am

  5. Well, as the blog post says, it is not an official feature but you can use it as a preview with that enviroment variable. That should give you good mirror fallback for free. (by metalinks)

    The usage of things like bittorrent depend on availability of them, and the metalinks themselves, and I don’t think t makes sense for packages yet.

    duncan

    13 Nov 08 at 3:40 pm

Leave a Reply