August 27, 2007

Super computer running SUSE Linux

link here (gizmodo.com)

This is the 9th biggest supercomputer in the world, MareNostrum. It also happens to be the most gorgeous supercomputer in the world, installed in a former chapel with acres of glass and steel. It belongs to the Barcelona Supercomputing Center and is polished daily by hundreds of grovelling grad students. Probably.

It has 10,240 CPUs, 20TB of RAM, 280 TB of disk storage and runs on SUSE Linux. It can compute at 62.63 teraflops, with a peak performance of 94.21 teraflops, making it the most powerful computer in Europe.

July 19, 2007

package management in openSUSE 10.3

The next alpha6 of openSUSE 10.3 will contain lot of changes in the package management code.

I will try to summarize what is new so you can help us testing. The integration can bring some regressions, so please test it and help us to fix the remaining bugs during the betas.

What were the goals:

  • no more refresh and parsing during startup.
  • more compatibility with tools like yum and smart
  • more speed for the common use case: install a package

Before ZYpp did not make difference between launching the sources and refreshing them. This was quite annoying, and turned into needing network access for any operation requiring launching the sources first, like deleting, adding, searching.

How does it work now?

  • Repositories now have a unique (in your system) alias.

  • Adding a repository is just a matter of copying a .repo file (like those provided by the build service) to /etc/zypp/repos.d directory. If you use zypper or YaST it will create them for you, and perform additional checks.

  • Download of metadata happens during refresh. Also in this operation a binary cache is created.

  • deleting a repository is just a matter of deleting its .repo file, or doing it via zypper/YaST (they take additional checks like vacuum the cache).

  • YaST was just adapted and is still WIP, so in some places the old semantics could be still used (like refreshing repos even if it is not needed).

  • The binary cache allow to start up zypper and install packages in a much faster way. Caching the metadata is slow. But you do it once per refresh, and the data is stored preparsed in the cache. This allows to launch small repos in fraction of a second, and big repos in seconds later. No more parsing files in each startup.

  • Before lot of memory was used because most of the GUI data (descriptions, etc) was read into memory. Now all attributes (except resolvable information, for which we need indexes) is loaded on demand from the cache.

  • refresh operation uses now much less memory. One one object is kept in memory at time while writing the cache.

For the developers:

  • Lot of testcases. We can do more aggressive development without breaking basic stuff.
  • Lot of redesign, to be able to implement new features.
  • We build with cmake now

You expect for the future (post 10.3):

  • Integrate search with the cache, so no need to launch the resolvables and loop for searching
  • solving some queries from the cache?
July 5, 2007

ASUS US$199 Laptop running KDE

One of the biggest news stories out of this year’s Computex wasn’t of a new chipset, GPU, or graphics card, but rather of the announcement of the ASUS Eee PC, a small, slim, and light portable computer that is priced at $199. Introduced by Jonney Shih, Chariman and CEO of ASUS at Intel’s keynote address the first day of the show, the Eee PC has already made headlines world wide.
The Eee PC’s main competition is the One Laptop Per Child (OLPC) project that originally aimed to bring to market a laptop priced at $100 for children in developing countries. However, the OLPC project has seen numerous delays and even a price increase to the $170 range.

Thanks Will for the link (hothardware.com)

Developers cooling on Windows desktop, study finds

In a survey, Evans Data found that almost 65 percent of software developers are targeting some version of Windows for their applications, as opposed to nearly 75 percent last year.

The research group expects the number to drop another 2 percent in the coming year.

The culprit? Linux. Developers are choosing to write applications for Linux desktops in almost 12 percent of cases, which is a 34 percent increase from last year.

“It’s clear that a shift away (from) Windows began about two yeas ago, and the data show that this migration is now accelerating. Linux has benefited, but we also see corresponding growth in niche operating systems for non-traditional client devices. The landscape is changing,” said Evans Data CEO John Andrews in a statement.

Full link

June 28, 2007

Hackweek III (day 4)

Martin Vidner succeeded in making possible to use YaST UI:: from the language bindings. This means you could write a complete module in Perl, and get the benefits of the abstracted Qt/ncurses/gtk user interface.

These changes means that my Ruby bindings will be able to support it too.

I also got the code as yast2-bindings-ruby in the build service, it doesn’t build yet, but soon will appear in the home:dmacvicar project.

Srinivasa Ragavan is working on Desktop status awareness, which means trying to do useful tasks when you are away from your computer. I pointed him to the Kopete motion-auto-away plugin I hacked some years ago in order to set the user away in the messaging client if no motion was detected in a webcam stream.

June 27, 2007

Hackweek II (Day 3)

So today SCR (system configuration repository) works from Ruby. Also if you call Ruby from YCP and the Ruby module calls YaST, Ruby can’t find the library. I fixed that.

require 'yast'

m = YaST::Module.new("SCR")
m.Execute(".target.bash", "firefox")

Will launch firefox

require 'yast'

m = YaST::Module.new("SCR")
modules = m.Read(".proc.modules")
modules.each do | k, v |
  puts "#{k}:"
  v.each do | a, b |
    puts "    #{a} - #{b}"
  end
end

Will output:

de_cd:
    size - 40608
    used - 0
cdrom:
    size - 36896
    used - 2
ehci_hcd:
    size - 34956
    used - 0
... (more)
June 26, 2007

Hackweek

So, it is Hackweek. Hundreds of Novell engineers working on the things they are passionate about. A full week of fully innovation and cooperation.

You can see the flickr photos here. The ideas collected (more than 300, with a hundred in progress).

So what am I doing? There are a few ideas for YaST. YaST needs new light to evolve somewhere, and most of the ideas bring a step in this direction: Making it accessible from web services, other languages support, UI independence from the ycp language, etc.

YaST already has perl bindings and there are some modules written in Perl.

My project is bidirectional ruby support for YaST, and it works already.

So, in Ruby you could write:

module Foo
  def self.multiply_by_eight(a)
    return 8*a
  end

  def self.sum(a,b)
    return a+b;
  end

And then in a .ycp file:

{
    import "Foo";
    integer result = (integer) Foo::sum(2,3);
    UI::OpenDialog(
         `VBox(
         `Label(sformat("%1", result)),
         `PushButton("OK")
         )
         );
    UI::UserInput();
    UI::CloseDialog();
}

It works!!!!:

Image Hosted by ImageShack.us

Now, we can do YaST in Ruby too, using the “yast” module for Ruby, lets try the Arch module from YaST (which is ycp code)

require 'yast'

m = YaST::Module.new("Arch")
puts m.sparc32
puts m.arch_short
puts m.is_xen

The output is:

false
x86_64
false

What about something more complex. The Storage module is a C++ library. In order to bind it easily with YaST and being able to use SWIG, YaST developers access it using the YaST perl-bindings to access the generated SWIG perl-bindings from the C++ library, so calling this from Ruby code would go from Ruby, to YCP, to Perl, to C++:

m = YaST::Module.new("Storage")
dp = m.GetDiskPartition("/dev/sda1")
dp.each do | key, value |
  puts "#{key} #{value}"
end

The output is:

nr 1
disk /dev/sda

I also tried to access the UI, but it does not work. Martin Vidner told me he is working on getting rid of this limitation. SCR access should be doable but I haven’t look at it yet.

I will decide tomorrow if I tune the code or I spend my 2 days left (I travel to aKademy on Friday) researching other stuff (probably I will spend it on YaST too).

May 31, 2007

ZFS presentation : The last work in filesystems

link.

Also, ZFS FUSE implementation.

May 27, 2007

Easy development & test environment for Linux kernels

I really enjoy learning new things, and I spend most of my computer time digging into stuff. My productive times always coincide with the quality of my development environment.

If you can’t keep up with a easy and own system to checkout, develop, compile and try code, it is very likely you will spend more time doing meta-development.

I have been always interested into learning more about the kernel. I would conform myself with having a sandbox where easily modify and try things, so if one day I feel like doing something, I could just sit and start.

We are going to use QEMU, a PC emulator, which is quite nice because:

  • It can launch kernels directly. Uses some magic to load the kernel in the emulator and boot it, so it is not needed to setup a tftp server with a kernel.
  • It doesn’t require kernel modules, except if you need more speed, then you can use kqemu.
  • It is free

The fact QEMU can launch a kernel directly, means we can launch the image we just compiled.

To get the Linux kernel source do:

git-clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux

To avoid having to create a image of the system to use as a root disk all the time, we can just mount a directory on our own machine via NFS.

 make xconfig

Be sure to enable nfsroot option, which at the same time requires kernel ip level autoconfiguration. Also be sure to put your network card in the kernel, not as a module, otherwise you will need to create a initrd image too.

 make bzimage
 make modules

We have everything ready to boot the kernel. But we need a root filesystem too. We will use the Kiwi tool to generate a custom root filesystem. With kiwi you can generate a root filesystem and then turn it into a LiveDVD, a USB stick distro or a Xen image. For us the first stage is enough. See the website for info how to get SUSE packages for Kiwi

Edit /usr/share/kiwi/image/netboot/suse-10.2/config.xml and now create the image:

 kiwi --root /space/nfsroot/work --prepare /usr/share/kiwi/image/netboot/suse-10.2

When trying this, I got a strange error. Looking at work.log, the problem was when kiwi called smart to install the packages, at commit time, the rpm database did not exist.

 Committing transaction...^M
 error: cannot open Packages index using db3 - No such file or directory (2)^M
 error: cannot open Packages database in work/var/lib/rpm

So I edited /usr/share/kiwi/modules/KIWIManager.pm and near line 630 I added these two lines:

 print $fd "mkdir -p $root/var/lib/rpm\n";
 print $fd "rpm --root $root --initdb\n";

I am not sure if it the right fix. I will ask on the mailing list later. But the workaround worked for me. The final section of the code looks now like:

 # Add package manager to package list
 #------------------------------------------
 push (@packs,$manager);
 #==========================================
 # Create screen call file
 #------------------------------------------
 print $fd "smart update @channelList\n";
 print $fd "mkdir -p $root/var/lib/rpm\n";
 print $fd "rpm --root $root --initdb\n";
 print $fd "test \$? = 0 && smart install @packs @installOpts\n";
 print $fd "echo \$? > $screenCall.exit\n";
 print $fd "rm -f $root/etc/smart/channels/*\n";

Edit /etc/exports to share that directory via NFS:

 /space/nfsroot/work   (rw,root_squash,sync)

Now, go to your kernel source directory and launch it:

qemu -kernel arch/i386/boot/bzImage
 -append
  "nfsroot=192.168.0.101:/space/nfsroot/work,rw
   ip=::::diskless:eth0:dhcp root=/dev/nfs"
 -hda /dev/zero -net nic -net user

Final result:

Image Hosted by ImageShack.us

I still can’t login on the machine, but that is a minor issue. Some error messages about read-only filesystem too.

Things I haven’t figured yet:

  • How to install modules to the root filesystem (something like make modules_install DESTDIR=/space/image)
May 21, 2007

truecrypt

Truecrypt updated to 4.3a.

Don’t ask me what KBUILD_STR is, but defining it as #s seems to work.

« Older Entries
Newer Entries »