Duncan Mac-Vicar P.

We have cleaned up the main pages of libzypp and satsolver, mainly to introduce a new feature.

Since today, there is developer documentation generated on every svn build from our cruise control system (continuous build), available for trunk and branches.

ZYpp documentation has already qute some articles and the API is very good documented, still, it will be enhanced in the following months to take advantages of having the documentation always available. SP2 branch documentation will be available soon.

For SAT solver, we have moved the existing documentation into the Related pages of the generated documentation (which already contains very useful articles) and we will do every possible effort to document the API too (which is almost empty right now :-/ ).

Yesterday I sat together with Stefan Haas in front of the whiteboard and analyzed the specification of the DSP1023 (Software Inventory) and DSP1025 (Software Updates) CIM profiles, from the DMTF.

Our goal was to understand it, and therefore we tried to map the concept to the knowledge we already have, which is Linux software management (which in turn can be reduced almost to packages :-). Here are our notes:

(For simplicity, I am skipping the CIM_ prefix for all classes)

Packages

Packages are represented as usual NVRs (name, version, release) using the SoftwareIdentity class.

The url of the package is represented associating the SoftwareIdentity with url instances (SoftwareIdentityResources) trough a SAPAvailableForElement association. Products, subpackages and components can be modeled by using OrderedComponent class, which references the component and one member per instance.

One can associate hardware (or any managed element) with software using ElementSoftwareIdentity

Repositories

Repos are modeled by using SystemSpecificCollection and associated via HostedCollection to a system (that seems to be the local computer).

Packages are associated to the repository via the MemberOfCollection class.

Installed packages are represented by instances of InstalledSoftwareIdentity (which references the ComputerSystem where it is installed).

Groups, Patterns

Just another software identity (for the group itself), and references the grouped identities using OrderedComponent instances (which has two fields, the GroupComponent and PartComponent, which reference the respective instances).

Dependencies

If the target software does not have and instance, a SoftwareIdentity is created and isEntity set to false (kind of named dependency) For dependencies that exists software collection, then OrderedDependency is instantiated for each dependency and Antecedent and Dependent is filled)

For installing the packages, a SoftwareInstallationService is needed. Packages can be installed calling installFromSoftwareIdentity() (name, version release instance) or installFromByteStream() (like an rpm package).

To define an installation service, the SoftwareInstallationService class need to specify what it supports by instantiating SoftwareInstallationServiceCapabilities (which has properties like the supported URI schemes), and then this capabilities are associated back to the service using ElementCapabilities.

To define whether a package is compatible with a given target, one uses the TargetTypes array in the SoftwareIdentity class.

If a SoftwareIdentity is available and there is some SoftwareInstallationService that is compatible (or capable) of installing it, a instance of the ServiceAffectsElement needs to be instantiated.

Before actually installing or updating, the client checks if a SoftwareIdentity can be installed on a element, using the CheckSoftwareIdentity() method on the SoftwareInstallationService. You give this method the software itself, the collection and the target element. You get back a InstallCharacteristics with the details (for example if you need to reboot after installing or not).

Once you start the operation using InstallFromSoftwareIdentity(), you get back a Job instance that represent the task. Also true for InstallFromByteStream(), but here you pass an Image instance instead of a SoftwareIdentity. There is a similar InstallFromURI operation too. (jobs are only returned if the service has async capabilities).

The client can pass options to the install operation using InstallOptionValues.

TODO: figure out ElementSoftwareStatus more.

Additionally, you can see how HP has implemented the inventory profile with rpm. It has some useful information about which rpm tags belong to which classes properties for example.

If you own a Blackberry from Vodafone Germany, you may be able to update to OS 4.5 depending on your model. I did the update yesterday, and it went smooth. You will need:

• Windows machine, as the update is done from the desktop manager
• The right firmware update for your model.
• You need to re-activate against the Enterprise Server in case you use BES. Contact your administrator for that.

You get lot of improvements against 4.2:

• Better browser. Mouse cursor to navigate.
• HTML mails
• Video camera.
• Some Media Player enhancements.
• Decent fonts.
• Documents to go. I can now even open pdfs from mail.
• Lot of user interface improvements. It looks nicer.
• Free/Busy calendar lookup.

(note, features depend on your device model)

Morning

Did not start the day in a good way. I buy mobicards every month so I have full access to the public transport system. I buy them via the internet, so I get an email when it is about to run out, and with one click I get the form to order another one by mail (even with the date range pre filled).

Today I took the straßenbahn (tram), and suddenly the guy next to me transformed itself in 1 second from a civilian to a public transport control employee, and asked me for my ticket. I took my mobicard from my pocket and showed it to him. He nods but after a second he comes back and ask “let me see it again”. He looks again and points out it starts on 12.11 and today was 11.11. No problem, last month one is behind that one (the whole year cards are there). He looks the old one and points out “This one ends at 10.11″. I could not believe it, I did not even realize. Usually I just click OK-OK when buying them. I got a ticket. Luckily the checkbox marked was not “pay 40€” but “Go to the headquarters to clarify” (which still means they could fine me). Of course that means wasting half a morning not counting the time I lost already over the tram (not being able to get out in the right station). If I get fined after buying mobicards every month for years it would be really ironic.

ZYpp

In the office we had the typical “first media” blocker. Luckily Michael found the bug really quickly. Later I wanted to test the fix using linuxrc magic driverupdate tricks I have learned in the past weeks. However it did not work: Linuxrc can use rpms as a container format, but still expects the driverupdate directory structure. Bah, world was not that cool as I thought. Met with Michael and Steffen about it. Steffen realized that it would not be a bad idea to try the rpm first as a driverupdate, and if no driverupdate is there just unpack it in the system (and promised to implement it) That sounds good, easy testing. Happy.

YaST

autoYaST

Katarina wrote a really nice tutorial that explains how use autoYaST to apply a configuration to a running system. Uhm. Looks like an XML API. Wow. Couldn’t we use it for our YaST web service?. Talked with schubi about it. While the command line interface has higher level, exposing this interface would make sense too, because it will offer automatically almost all autoYaST power.

Partitioner

The new YaST partitioner, on which Katarina, Arvin, Matthias and Martin worked really hard during the Code11 cycle got a pretty nice review here. Give it a look. And give the guys feedback. I am sure lots of things can still be improved!.

YaST releases

Stano is reviving the discussion about whether to make independent of openSUSE YaST releases. I talked about this topic with Zonker just after he started as community manager.

I think it would be a pretty good move. Also a big challenge. It would mean a challenge not only for us the YaST teams but also for other internal and external stakeholders.

Right now a new YaST package submission (same with libzypp) is more or less tied with the development of features or some project manager sitting next to your chair to get a blocker fixed for an openSUSE release. Things would change if the distribution’s team would need to take the last released YaST and take patches, forward ports, backports, etc.

Also it would be a challenge for us. Most of YaST testing happens on a openSUSE release. We have lot of ideas on how to improve our automated testing and this would require develop them further. Also I think an approach like this would facilitate making YaST less distro-dependent.

Another point I like of the approach is that it would force us to have a better process to define our core-platform (ir order to be able to release it separately). Right now I see it a bit chaotic: APIs are born in modules, and as soon as they are used by more than one module, they land in the wildcard yast2 package. Something I really don’t find very convenient. APIs should be reviewed with more care and packaged by the area the API covers.

Still and interesting topic and material to innovate!

duncan@tarro:~> xmms

The program 'xmms' can be found in the following package:
  * xmms [ path: /usr/bin/xmms, repository: zypp (packman) ]

Try: sudo zypper install xmms

bash: xmms: command not found
duncan@tarro:~>

made possible by Scout.

Background

My colleagues use awesome tricks to do nfs boots with custom inst-sys and other magical procedures in order to test a modified installation. I find those tricks cool but that I tend to forget them after a few days.

During 11.0 release cycle, Coolo had to test the new installation theming. He explained me his method, using update disks.

Update disks exist since longs time. AFAIK they were originally designed for drivers, but today they can insert almost any kind of file to the instsys. That was what coolo did: had his own tree, with an updated installation css theme, packed it, and then run qemu, passing the updatedisk url (which he served from his own machine) IIRC he passed an iso as a cdrom to qemu.

I find this method pretty cool. But I still had the feeling it could be automated more:

• qemu supports booting directly from a kernel and initrd, so I could use the http repositories and download those instead of needing a iso.
• I could append kernel parameters telling qemu about them
• I could automatically repack my local directory with updated files and avoid running mkcramfs by hand (latter Steffen told me I could even use cpio)
• Why bothering with apache in my machine? As qemu can’t access the driverdisk image from my local disk, I could start a embedded web server on demand and tell linuxrc about it.

So after lot of questions to Steffen and some code reading of Ludwig awesome “setupgrubfornfsinstall”, here is the first version that works:

git clone git://git.opensuse.org/people/dmacvicar/startinstall.git

Usage

The structure of the driver update disk is something like this:

./linux/suse/i386-11.0/dud.config
./linux/suse/i386-11.0/inst-sys
./linux/suse/i386-11.0/inst-sys/usr
./linux/suse/i386-11.0/inst-sys/usr/share
./linux/suse/i386-11.0/inst-sys/usr/share/YaST2
./linux/suse/i386-11.0/inst-sys/usr/share/YaST2/theme
./linux/suse/i386-11.0/inst-sys/usr/share/YaST2/theme/openSUSE
./linux/suse/i386-11.0/inst-sys/usr/share/YaST2/theme/openSUSE/wizard
./linux/suse/i386-11.0/inst-sys/usr/share/YaST2/theme/openSUSE/wizard/logo.png
./linux/suse/i386-11.0/inst-sys/usr/share/YaST2/theme/openSUSE/wizard/installation.qss

In this case, I only want to replace the logo and css file. The important part is the directory with the architecture and version. dud.config has:

UpdateName:     openSUSE 11.0 DUD
UpdateID:       193a399fca4b4da9
UpdatePriority: 100

Which I guess are random values for our specific case This directory tree is available at /foo/driverupdate on my machine.

Now you need a http or ftp repository (I will add more features later…). In the example, it will be http://distros.foo.com/openSUSE-11.0-RC4-DVD/i386/DVD1

Now just run it:

ruby startinstall.rb --tree /foo/driverupdate http://distros.foo.com/openSUSE-11.0-RC4-DVD/i386/DVD1

startinstall.rb will download the kernel, initrd, pack the driverdisk, start a (webbrick based) webserver offering this driverdisk and launch qemu with the right parameters:

qemu -kernel /tmp/kernel20081030-5149-1yn5upf-0
 -initrd /tmp/initrd20081030-5149-uquvmh-0
 -append 
   "install=http://distros.foo.com/openSUSE-11.0-RC4-DVD/i386/DVD1
    driverupdate=http://mymachine.suse.de:9999/updatedisk20081030-5149-ogaqpl-0 "
 -hda /dev/zero
 -no-reboot
 -net nic,macaddr=00:BE:C3:B9:4C:BD,model=pcnet
 -m 666
 -net user
 -usb -usbdevice tablet

After booting, you will see your modified (see the logo) installation:

I will keep a small page for this tool here.

I agree with Robert Reich. A free market needs that companies can fail.

I got pointed to Dan’s Kegel post on Linux Foundation’s packaging mailing list called “Towards trusted third party repositories”. I am not subscribed to the list so I am commenting here.

I’ve been intrigued by http://en.opensuse.org/Standards/OneClickInstall for some time now. (That’s a way to provide a one-click web install experience for .deb/.rpm/.psi etc. packages, implemented as a mime type handler that parses a simple .xml file pointing to the package/repository appropriate for each distro.) When this idea was brought up on the Packagekit mailing list, it generated lots of negative feedback. The summary at http://packagekit.org/pk-faq.html#1-click-install gives a bunch of non-central objections, followed by the central objection that one cannot trust third party repositories: “Allowing to easily add third party repositories and install third party software without a certification infrastructure is like opening the gates to hell”

One Click Install is only a simple description to add a repository and some software, but the security is not a property of this simple description, but of the package manager. A malicious one click install file can point ZYpp to “hell”, but “hell:

• Either is signed with a non trusted key and the user needs to trust it.
• Is signed with the distribution key, which is already in the trusted keyring therefore it just works.

This is a real problem. Here are a couple risks:

No, it is not. It depends on the package manager you are using. Basic security here is independent of one click install.

1) users might click on malware sites and add completely malicious sites to their repository lists

Again, this is not true for ZYpp, and I guess not all package managers allows to download metadata from a repository without trusting it first.

2) a compromised third-party repository might update system packages maliciously.

If you trusted it, there is nothing you can do.

3) several genuinely well-intentioned repositories might include conflicting versions of a commonly needed package not provided by the system repositories.

This has nothing to do with one click install. Packman repository does it already. This is a dependency resolving problem and vendor management: ZYpp for example, does not allow implicit vendor jump on update (only on dist-upgrade). And vendor jump is a conflict you need manually to approve. Only package managers that threat all packages with the same name as the same package suffer here.

After mulling this problem over for a long time, two ideas came to mind: 1) Since the distribution is trusted, it could decide to trust some third-party repositories. For instance, it might decide to trust Adobe’s hypothetical repository so that people could get flash and air updates straight from the source.

This is possible now:

• You can import Adobe’s key into the distribution together with the distribution key, so it goes automatically to the trusted keyring
• You can create metadata in the distribution side, linking to Adobe’s packages, and sign the metadata with the distribution key
• You can do nothing and let the user trust explicitly the repository

This idea of using the distribution as arbiter of trust for third party repositories could be extended to games publishers, etc. This could provide a partial solution to the first threat listed above; if the “good” third-party repositories are already known to the distribution, there’s less need for users to be doing something dangerous like deciding on their own to trust a random third-party repo. This addresses the first threat identified above.

I think repository trusting and the required package manager infrastructure has nothing to do with one click install. It just needs to be there.

2) A simple way to keep repositories from updating packages they shouldn’t is to have package managers enforce some sort of namespacing. e.g. Adobe’s repository could be allowed to only update packages whose names start with “adobe-”. (System repositories would continue to be able to update any package at all.) This addresses the second and third threats identified above.

Why inventing a new thing? You already have the vendor tag there. A package should be considered update candidate only for packages that are from the same vendor.

Imagine you have mediaplayer-1.0 from the distribution, without mp3 support. Then a 3rd party repo offers a compiled 2.0 version with mp3 support but without other features, and then the distribution offers 3.0. You would be getting and losing features all the time as you update. Same package names does not means it is the same package. If your package manager does that, it is a bug. If there is no solution to the dependency solving than jumping vendor, do a conflict and make the user explicitly switch. Once he does, the update candidate would be the same vendor packages.

I think something like this is going to be needed before we can have a thriving — and safe – ecosystem of ISVs providing easily-downloaded-and-installed binary packages for Linux. What do people think about the package namespacing idea? - Dan

I don’t think reinventing the vendor tag and repository signatures is a good idea.

The problem is not at this level, IMHO the only thing we can do is improve the usability of the security chain. For the user is meanless to trust a repository with gpg key 0×32432432 and they will probably just click “yes”. The pending task is to make the cryptographic chain something that makes sense for the user.

Ahora se ha revelado Acuerdo Secreto SCD/Gobierno de Chile (léanlo), hoy sólo quiero comentar un solo punto:

• ARTICULO 85 L. OPERADORAS DE INTERNET Lejos uno de los posibles artículos más populistas. En su afán de lucrarse, quieren cobrarle a las ISP que dan acceso a internet un valor por cada conexión, pues según la SCD TODOS los que tenemos internet pirateamos música, y no cualquier música, sino que la del repertorio que ellos gestionan. Esto, amigos, sería fijar una responsabilidad a las ISP que EN NINGUN PAIS DEL MUNDO está determinado a priori, siendo Chile los primeros en una medida inconstitucional como esta. Y no es que esté defendiendo a las pobres ISPs , sino a todos los consumidores, porque dicho valor se verá SI o SI reflejado en los precios de acceso a Internet, haciendo AUN MAS CARO y PROHIBITIVO la banda ancha, Y todo porque el gobierno acordó ASEGURAR los ingresos a la monopólica de la SCD.

Un ejemplo análogo a esto sería:

Yo soy desarrollador de software, y tengo mi asociación de autores de software, que agrupa a desconocidos como yo más otros conocidos. Hoy termine un nuevo programa, el cual he mantenido bajo “todos los derechos reservados” y pienso venderlo.

Sin importarme lo malo que es mi programa, y que nadie sabe que yo existo, usaremos la teta del gobierno para que este, mediante presunción, dictamine que cualquier persona que tenga acceso a Internet, tiene accesso a las redes p2p, y de esta manera acceso a descargar ilegalmente mi programa, y por lo tanto mi agrupación de programadores tiene derecho a recibir un sueldo pagado por los proveedores de Internet (que será por supuesto traspasado a los usuarios).

ESO ES EL ANÁLOGO DE LO QUE INTENTA LA SCD Y SUS ARTISTAS.

¿Por qué es el análogo al programador desconocido?

Porque si bien sé que Fernando Ubiergo es un músico, jamás he escuchado su música. De Quique Neira sólo he escuchado lo que ha sonado de fondo en alguna TV encendida. De Horacio Saavedra no conozco nada más que su cortina del festival de Viña, y de Eduardo Gatti sólo debo haber tocado “Los momentos” en la guitarra alguna vez en una fogata, pero no tengo ni siquiera la versión original. De esta lista sólo ubico a uno cada cien. ¿Porque tienen los chilenos que mantenerlos si no escuchamos ni disfrutamos sus canciones?.

¿Que sería del mundo si las empresas nos cobraran por lo que ellas creen que hemos hablado por teléfono y no lo que dice el marcador?

La estrategia de estos artistas agrupados en la SCD, es de ser mantenidos por el sistema, es decir, pasar a ser parte del patrimonio nacional sin importar su popularidad o importancia.

Quizás vamos en camino recto al absurdo canon digital Español, que significa un impuesto a cualquier medio de grabación, el cual mediante presunción, podría ser usado para copias ilegales de algún artista (que quizás tampoco conoces). O sea, cuando un español compra un CD virgen, o incluso un disco duro para archivar sus propias fotos, le está pagando a estos artistas.

En la misma línea, los programadores que hayan puesto a disposición obras con “todos los derechos reservados” podrían exigir al gobierno una ayuda, cargando un impuesto a cualquier medio de almacenamiento. Justicia para todos no? Esto se podría extender a cualquier oficio cuyas obras estén sujetas al copyright. Una locura.

Chile, que no te metan otro gol. Tu no tienes porque financiar a Ubiergo o la Cecilia Echeñique si no utilizas su música.

Update

• El post en Fayerwayer sobre el tema.
• Sería interesante también aprovechar la blogósfera para ver que diputados y senadores están a favor y en contra de esta ley.

Update 2

Efectivamente, la SCD busca instaurar un sistema de canon, aunque al parecer no es parte de los aterradores puntos del “secreto acuerdo”. Más sobre eso aquí.

Update 3 - En que posición están los senadores y diputados?

Es importante aprovechar estos polémicos temas para que veas como vota tu senador o diputado.

• Juan Pablo Letelier al parecer está en contra de esta iniciativa.

I noticed some weeks ago that Death Magnetic (especially “The day that never comes” chorus) sounded very distorted. The mix is not good.

Now I found out it is not my imagination, but Wired reported the Guitar Hero version sounds much better (also in Rolling Stone) than the CD, and there is already a petition to remaster it.

Instead of facing the problem Lars Ulrich gave a very lame answer. That is nonsense. Just like artists get upset when people copy their albums, fans also get upset when they buy the albums and the sound is crap.

If you got the CD and you are disappointed with the sound, you can find the Guitar Hero version remastered by fans in some torrent sites. Those should not hurt with earphones. There you have it Lars, the people that bought the CD is fixing the problem for you.

update: This video shows very well we are not talking about sound “style”, but quality. Make sure you are watching the high quality of the YouTube video (there should be a link under the video).

update #2: This video explains in simple words what is this “Loudness war” and how it destroys the sound.