Tilting at windmills

Duncan Mac-Vicar P.'s blog

yum and ZYpp speed / memory usage

18 Comments

Michael Zucchi [complains][1] about yum memory usage, and points python as guilty.

>Yum isn’t so yummy after-all. Re-enabled python so i could run yum. Wow 120mb of vm to install a couple of packages. Not bad considering the box only has 128mb. This is crap.
>
>Hmm, should I try xubuntu – or will it be just as crappy and bloated and blighted by python poo?

Since our efforts to make the ZYpp really fast, by incorporating and integrating Michael Schroeder’s sat solver together with Michael Matz’s great work on the solv files and data storage, I never took the time to make a “quick comparison” on speed or memory usage. So lets have a quick look.

These are my repositories:

Software configuration management (openSUSE_10.3)
10.3 - Main Repository (NON-OSS)
10.3 - Packman
openSUSE-10.3-Updates
Virtualization:VirtualBox
home:dgollub
KDE:KDE3
Mozilla based projects (openSUSE_10.3)
ZYPP SVN Builds (openSUSE_10.3)
ZYPP SVN Builds (openSUSE_10.3)
home:prusnak
10.3 - VideoLan
openSUSE.org tools (openSUSE_10.3)
SUSE Feature Tracking Tool (openSUSE_10.3)
psmt's Home Project (openSUSE_10.3)
openSUSE:10.3
Duncan Mac-Vicar SUSE rpms (openSUSE_10.3)
Latest YaST svn snapshots (openSUSE_10.3)
building/openSUSE_10.3

All these repos together are about 41.000 packages.

What I did was to symlink ZYpp repositories to the yum repo path so they use the same repos.

# rm -rf /etc/yum.repos.d/
# ln -s /etc/zypp/repos.d /etc/yum.repos.d
*NOTE:* I tested with yum 3.2.4. I know 3.2.14 is available, but that is what I had installed when doing the test. After doing this tests I upgraded to 3.2.14 but it did not accept my .repo file because the character “:” in repo names. However the changelog of yum since 3.2.4 shows:

If using latest yum would invalidate this numbers (not as in 1 second, but as in an order of magnitude), let me know and I will repeat them when I make them work with my repo files.

**Update 14.05.2008 : I did add yum 3.2.14. However it performed even worse, except for memory usage.**

**Update 15.05.2008 : added smart 0.52 numbers**

libzypp is the one you see in factory since some days: 4.21.1.

yum and ZYpp behave differently, as yum downloads and parses filelists.xml and other.xml which we ignore. There fore I skipped the download metadata part and just timed the cache building process.

# yum clean dbcache
...
19 sqlite files removed

# time yum makecache
...
Metadata Cache Created

real    9m41.036s
user    2m34.766s
sys     0m11.545s

Almost 10 minutes. As this time includes parsing the two big files we ignore. I did it again, pressing Ctrl-C After yum finished with the primary data, which is what ZYpp uses:

# time yum makecache
...
Exiting on user cancel

real    4m6.730s
user    0m34.058s
sys     0m3.080s

Now, zypper’s turn:

# time zypper ref -B
...
All repositories have been refreshed.

real    0m18.472s
user    0m16.029s
sys     0m2.024s

So yum takes 13 times the ZYpp needs technically (primary 1:1 comparison), but 30 times the time the end user sees.

Now, installing a package. Times were measured till the “continue? yes/no” prompt, or till the first interactive question.

# time yum install fate
...
Is this ok [y/N]: n
Exiting on user Command
Complete!

real    0m19.143s
user    0m14.057s
sys     0m1.920s

zypper’s turn:

# time zypper in fate
...
Continue? [YES/no]: n

real    0m9.796s
user    0m8.509s
sys     0m0.624s

This time, ZYpp is only twice as fast as yum. Only ;-)

What happens when you want to upgrade your packages?

# time yum upgrade
...
real    0m45.152s
user    0m36.894s
sys     0m7.476s

(Note: yum did not even found a solution here).

# time zypper update
...
Continue? [YES/no]: n

real    0m8.988s
user    0m7.820s
sys     0m0.596s

yum needs 4 times the time ZYpp needs to calculate the upgrade.

**Update 14.05.2008 : I was comparing update to upgrade, I fixed those numbers in the chart. However, I don’t have the update value for the old yum.**

Summary:

Media_httpspreadsheet_vgtuz

Now, how much memory does each one need? For this, I just tested the install command with one package using valgrind massif, a heap profiler.

yum memory usage:

[![yum memory usage][2]][2]

ZYpp memory usage:

[![zypp memory usage][3]][3]

**Update 14.05.2008 : memory usage chart for yum 3.2.14**

[![yum 3.2.14 memory usage][4]][4]

**Update 15.05.2008 : memory usage chart for smart 0.52**

[![smart 0.52 memory usage][5]][5]

Here you can appreciate ZYpp goes a little bit over 20M, while yum goes over 180M, so yum uses about 9 times more memory. **Update 14.05.2008 : yum 3.2.14 uses around 160 in the worst point of time.**

Media_httpspreadsheet_iahnc

I would be interested in tracking cpu usage too, but that will come later. What do you think about it?

[1]: http://blogs.gnome.org/zucchi/2008/05/10/linux-is-bloated/
[2]: http://files.opensuse.org/opensuse/en/e/ea/Yum-in-massif.png
[3]: http://files.opensuse.org/opensuse/en/6/65/Zypper-in-massif.png
[4]: http://files.opensuse.org/opensuse/en/2/26/Yum-3214-massif.png
[5]: http://files.opensuse.org/opensuse/en/f/fa/Smart-in-massif.png

About these ads

18 thoughts on “yum and ZYpp speed / memory usage

  1. Very interesting numbers, thanks for the work!A collegue of mine just asked if there are some numbers comparing zypp with smart… Have you done some testing there, too?smart seems to be very popular among opensuse users…

  2. It would be also interesting to know how the new libzypp manages the “Case Studies” from http://svn.labix.org/smart/trunk/READMEI don’t know, but could be that yum fixed “Case 2″ and libzypp “fails”? So the extra time and memory over libzypp taken by yum to make the dep resolution would be justified.

  3. [...] que la distro tiene muy, muy buena pinta. La gestión de paquetes va fantástica (sobre todo usando zypper frente a yum, por ejemplo) y la verdad es que aunque KDE 4 va un poquito tirado en algunas operaciones con [...]

  4. I have added the smart numbers.As for the solver, those cases are no real testcases, as they don’t describe what was in the system (like our libzypp-testsuite)

  5. Could you do another test for me please: how does this new Zypp compare to Synaptic. A friend of mine has an old laptop and will convert to openSUSE if it’s faster than Ubuntu.Thanks mate!

  6. [...] Duncan Mac-Vicar P. homepage « yum and ZYpp speed / memory usage [...]

  7. We are talking about very simple cases. I don’t think what was in the system is relevant in a test where you create five RPMs with the above spec files, create a repository with them and install ‘A’. There are any real packages that require/provide thing like “C” or “libA”???License: LicenseGroup: GroupSummary: SummaryName: AVersion: 1.0Release: 1Requires: BCD libB libC libD%descriptionDescription%filesLicense: LicenseGroup: GroupSummary: SummaryName: BCDVersion: 1.0Release: 1Provides: libB libC libD%descriptionDescription%filesLicense: LicenseGroup: GroupSummary: SummaryName: BVersion: 1.0Release: 1Provides: libB%descriptionDescription%filesLicense: LicenseGroup: GroupSummary: SummaryName: CVersion: 1.0Release: 1Provides: libC%descriptionDescription%filesLicense: LicenseGroup: GroupSummary: SummaryName: DVersion: 1.0Release: 1Provides: libD%descriptionDescription%files

  8. I had problems with the last libzypp on 10.3, so I’m not going to test it at least until 11.0 final. But just tested yum 3.2.14 with these five RPMs… and yum still tries to install all five files when only BCD is needed to install A!!!

  9. For those smart test cases see my new posts:Solving the famous “smart” case 2Solving the famous “smart” case 3

  10. ( and you will see that you could test those cases yourself :-) we provide the complete simulation environment )

  11. Great work Duncan!

  12. cool – you have Gravatar support I see :-)

  13. I was very surprised that my home project (home:prusnak) appeared between the test repositories. What was so interesting about it that it made to the list? :)

  14. If the repo is on your local disk make sure youecho 1 > /proc/sys/vm/drop_cachesas root to drop the kernel’s file cache before the start of a run (otherwise later runs may have the advantage of not doing disk I/O)

  15. [...] improvements like the fast installation, how YaST became sexy, how YaST/ZYpp/zypper became fast, how YaST/ZYpp/zypper performs better than others and even that our solver is also really [...]

  16. [...] Zypper is wicked fast, and is now my favorite way to manage packages. All it lacks is tab-completion of package names, but running zypper se somepackage is so fast that I don’t mind too much. [...]

  17. [...] után a SOLV fájlokban történt változások, valamint az új solver szignifikáns teljesítménynövekedést eredményezettés kevesebb memóriát igényel összehasonlítva a Smarttal és a [...]

  18. I was asking myself the other day, “Why doesn’t openSUSE just use yum for package management.”I have the answers now.zypper still needs some more work, however. I can’t seem to find a way to show the installed packages that require PACKAGE (so that I can remove PACKAGE without upsetting the system), or show what packages PACKAGE requires (to measure how much I will need to download and install before installing PACKAGE).All-in-all I like the effort openSUSE puts in to create a decent distro.Felipe

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.