Duncan Mac-Vicar P.

Background

My colleagues use awesome tricks to do nfs boots with custom inst-sys and other magical procedures in order to test a modified installation. I find those tricks cool but that I tend to forget them after a few days.

During 11.0 release cycle, Coolo had to test the new installation theming. He explained me his method, using update disks.

Update disks exist since longs time. AFAIK they were originally designed for drivers, but today they can insert almost any kind of file to the instsys. That was what coolo did: had his own tree, with an updated installation css theme, packed it, and then run qemu, passing the updatedisk url (which he served from his own machine) IIRC he passed an iso as a cdrom to qemu.

I find this method pretty cool. But I still had the feeling it could be automated more:

• qemu supports booting directly from a kernel and initrd, so I could use the http repositories and download those instead of needing a iso.
• I could append kernel parameters telling qemu about them
• I could automatically repack my local directory with updated files and avoid running mkcramfs by hand (latter Steffen told me I could even use cpio)
• Why bothering with apache in my machine? As qemu can’t access the driverdisk image from my local disk, I could start a embedded web server on demand and tell linuxrc about it.

So after lot of questions to Steffen and some code reading of Ludwig awesome “setupgrubfornfsinstall”, here is the first version that works:

git clone git://git.opensuse.org/people/dmacvicar/startinstall.git

Usage

The structure of the driver update disk is something like this:

./linux/suse/i386-11.0/dud.config
./linux/suse/i386-11.0/inst-sys
./linux/suse/i386-11.0/inst-sys/usr
./linux/suse/i386-11.0/inst-sys/usr/share
./linux/suse/i386-11.0/inst-sys/usr/share/YaST2
./linux/suse/i386-11.0/inst-sys/usr/share/YaST2/theme
./linux/suse/i386-11.0/inst-sys/usr/share/YaST2/theme/openSUSE
./linux/suse/i386-11.0/inst-sys/usr/share/YaST2/theme/openSUSE/wizard
./linux/suse/i386-11.0/inst-sys/usr/share/YaST2/theme/openSUSE/wizard/logo.png
./linux/suse/i386-11.0/inst-sys/usr/share/YaST2/theme/openSUSE/wizard/installation.qss

In this case, I only want to replace the logo and css file. The important part is the directory with the architecture and version. dud.config has:

UpdateName:     openSUSE 11.0 DUD
UpdateID:       193a399fca4b4da9
UpdatePriority: 100

Which I guess are random values for our specific case This directory tree is available at /foo/driverupdate on my machine.

Now you need a http or ftp repository (I will add more features later…). In the example, it will be http://distros.foo.com/openSUSE-11.0-RC4-DVD/i386/DVD1

Now just run it:

ruby startinstall.rb --tree /foo/driverupdate http://distros.foo.com/openSUSE-11.0-RC4-DVD/i386/DVD1

startinstall.rb will download the kernel, initrd, pack the driverdisk, start a (webbrick based) webserver offering this driverdisk and launch qemu with the right parameters:

qemu -kernel /tmp/kernel20081030-5149-1yn5upf-0
 -initrd /tmp/initrd20081030-5149-uquvmh-0
 -append 
   "install=http://distros.foo.com/openSUSE-11.0-RC4-DVD/i386/DVD1
    driverupdate=http://mymachine.suse.de:9999/updatedisk20081030-5149-ogaqpl-0 "
 -hda /dev/zero
 -no-reboot
 -net nic,macaddr=00:BE:C3:B9:4C:BD,model=pcnet
 -m 666
 -net user
 -usb -usbdevice tablet

After booting, you will see your modified (see the logo) installation:

I will keep a small page for this tool here.

I agree with Robert Reich. A free market needs that companies can fail.

I got pointed to Dan’s Kegel post on Linux Foundation’s packaging mailing list called “Towards trusted third party repositories”. I am not subscribed to the list so I am commenting here.

I’ve been intrigued by http://en.opensuse.org/Standards/OneClickInstall for some time now. (That’s a way to provide a one-click web install experience for .deb/.rpm/.psi etc. packages, implemented as a mime type handler that parses a simple .xml file pointing to the package/repository appropriate for each distro.) When this idea was brought up on the Packagekit mailing list, it generated lots of negative feedback. The summary at http://packagekit.org/pk-faq.html#1-click-install gives a bunch of non-central objections, followed by the central objection that one cannot trust third party repositories: “Allowing to easily add third party repositories and install third party software without a certification infrastructure is like opening the gates to hell”

One Click Install is only a simple description to add a repository and some software, but the security is not a property of this simple description, but of the package manager. A malicious one click install file can point ZYpp to “hell”, but “hell:

• Either is signed with a non trusted key and the user needs to trust it.
• Is signed with the distribution key, which is already in the trusted keyring therefore it just works.

This is a real problem. Here are a couple risks:

No, it is not. It depends on the package manager you are using. Basic security here is independent of one click install.

1) users might click on malware sites and add completely malicious sites to their repository lists

Again, this is not true for ZYpp, and I guess not all package managers allows to download metadata from a repository without trusting it first.

2) a compromised third-party repository might update system packages maliciously.

If you trusted it, there is nothing you can do.

3) several genuinely well-intentioned repositories might include conflicting versions of a commonly needed package not provided by the system repositories.

This has nothing to do with one click install. Packman repository does it already. This is a dependency resolving problem and vendor management: ZYpp for example, does not allow implicit vendor jump on update (only on dist-upgrade). And vendor jump is a conflict you need manually to approve. Only package managers that threat all packages with the same name as the same package suffer here.

After mulling this problem over for a long time, two ideas came to mind: 1) Since the distribution is trusted, it could decide to trust some third-party repositories. For instance, it might decide to trust Adobe’s hypothetical repository so that people could get flash and air updates straight from the source.

This is possible now:

• You can import Adobe’s key into the distribution together with the distribution key, so it goes automatically to the trusted keyring
• You can create metadata in the distribution side, linking to Adobe’s packages, and sign the metadata with the distribution key
• You can do nothing and let the user trust explicitly the repository

This idea of using the distribution as arbiter of trust for third party repositories could be extended to games publishers, etc. This could provide a partial solution to the first threat listed above; if the “good” third-party repositories are already known to the distribution, there’s less need for users to be doing something dangerous like deciding on their own to trust a random third-party repo. This addresses the first threat identified above.

I think repository trusting and the required package manager infrastructure has nothing to do with one click install. It just needs to be there.

2) A simple way to keep repositories from updating packages they shouldn’t is to have package managers enforce some sort of namespacing. e.g. Adobe’s repository could be allowed to only update packages whose names start with “adobe-”. (System repositories would continue to be able to update any package at all.) This addresses the second and third threats identified above.

Why inventing a new thing? You already have the vendor tag there. A package should be considered update candidate only for packages that are from the same vendor.

Imagine you have mediaplayer-1.0 from the distribution, without mp3 support. Then a 3rd party repo offers a compiled 2.0 version with mp3 support but without other features, and then the distribution offers 3.0. You would be getting and losing features all the time as you update. Same package names does not means it is the same package. If your package manager does that, it is a bug. If there is no solution to the dependency solving than jumping vendor, do a conflict and make the user explicitly switch. Once he does, the update candidate would be the same vendor packages.

I think something like this is going to be needed before we can have a thriving — and safe – ecosystem of ISVs providing easily-downloaded-and-installed binary packages for Linux. What do people think about the package namespacing idea? - Dan

I don’t think reinventing the vendor tag and repository signatures is a good idea.

The problem is not at this level, IMHO the only thing we can do is improve the usability of the security chain. For the user is meanless to trust a repository with gpg key 0×32432432 and they will probably just click “yes”. The pending task is to make the cryptographic chain something that makes sense for the user.

Ahora se ha revelado Acuerdo Secreto SCD/Gobierno de Chile (léanlo), hoy sólo quiero comentar un solo punto:

• ARTICULO 85 L. OPERADORAS DE INTERNET Lejos uno de los posibles artículos más populistas. En su afán de lucrarse, quieren cobrarle a las ISP que dan acceso a internet un valor por cada conexión, pues según la SCD TODOS los que tenemos internet pirateamos música, y no cualquier música, sino que la del repertorio que ellos gestionan. Esto, amigos, sería fijar una responsabilidad a las ISP que EN NINGUN PAIS DEL MUNDO está determinado a priori, siendo Chile los primeros en una medida inconstitucional como esta. Y no es que esté defendiendo a las pobres ISPs , sino a todos los consumidores, porque dicho valor se verá SI o SI reflejado en los precios de acceso a Internet, haciendo AUN MAS CARO y PROHIBITIVO la banda ancha, Y todo porque el gobierno acordó ASEGURAR los ingresos a la monopólica de la SCD.

Un ejemplo análogo a esto sería:

Yo soy desarrollador de software, y tengo mi asociación de autores de software, que agrupa a desconocidos como yo más otros conocidos. Hoy termine un nuevo programa, el cual he mantenido bajo “todos los derechos reservados” y pienso venderlo.

Sin importarme lo malo que es mi programa, y que nadie sabe que yo existo, usaremos la teta del gobierno para que este, mediante presunción, dictamine que cualquier persona que tenga acceso a Internet, tiene accesso a las redes p2p, y de esta manera acceso a descargar ilegalmente mi programa, y por lo tanto mi agrupación de programadores tiene derecho a recibir un sueldo pagado por los proveedores de Internet (que será por supuesto traspasado a los usuarios).

ESO ES EL ANÁLOGO DE LO QUE INTENTA LA SCD Y SUS ARTISTAS.

¿Por qué es el análogo al programador desconocido?

Porque si bien sé que Fernando Ubiergo es un músico, jamás he escuchado su música. De Quique Neira sólo he escuchado lo que ha sonado de fondo en alguna TV encendida. De Horacio Saavedra no conozco nada más que su cortina del festival de Viña, y de Eduardo Gatti sólo debo haber tocado “Los momentos” en la guitarra alguna vez en una fogata, pero no tengo ni siquiera la versión original. De esta lista sólo ubico a uno cada cien. ¿Porque tienen los chilenos que mantenerlos si no escuchamos ni disfrutamos sus canciones?.

¿Que sería del mundo si las empresas nos cobraran por lo que ellas creen que hemos hablado por teléfono y no lo que dice el marcador?

La estrategia de estos artistas agrupados en la SCD, es de ser mantenidos por el sistema, es decir, pasar a ser parte del patrimonio nacional sin importar su popularidad o importancia.

Quizás vamos en camino recto al absurdo canon digital Español, que significa un impuesto a cualquier medio de grabación, el cual mediante presunción, podría ser usado para copias ilegales de algún artista (que quizás tampoco conoces). O sea, cuando un español compra un CD virgen, o incluso un disco duro para archivar sus propias fotos, le está pagando a estos artistas.

En la misma línea, los programadores que hayan puesto a disposición obras con “todos los derechos reservados” podrían exigir al gobierno una ayuda, cargando un impuesto a cualquier medio de almacenamiento. Justicia para todos no? Esto se podría extender a cualquier oficio cuyas obras estén sujetas al copyright. Una locura.

Chile, que no te metan otro gol. Tu no tienes porque financiar a Ubiergo o la Cecilia Echeñique si no utilizas su música.

Update

• El post en Fayerwayer sobre el tema.
• Sería interesante también aprovechar la blogósfera para ver que diputados y senadores están a favor y en contra de esta ley.

Update 2

Efectivamente, la SCD busca instaurar un sistema de canon, aunque al parecer no es parte de los aterradores puntos del “secreto acuerdo”. Más sobre eso aquí.

Update 3 - En que posición están los senadores y diputados?

Es importante aprovechar estos polémicos temas para que veas como vota tu senador o diputado.

• Juan Pablo Letelier al parecer está en contra de esta iniciativa.

I noticed some weeks ago that Death Magnetic (especially “The day that never comes” chorus) sounded very distorted. The mix is not good.

Now I found out it is not my imagination, but Wired reported the Guitar Hero version sounds much better (also in Rolling Stone) than the CD, and there is already a petition to remaster it.

Instead of facing the problem Lars Ulrich gave a very lame answer. That is nonsense. Just like artists get upset when people copy their albums, fans also get upset when they buy the albums and the sound is crap.

If you got the CD and you are disappointed with the sound, you can find the Guitar Hero version remastered by fans in some torrent sites. Those should not hurt with earphones. There you have it Lars, the people that bought the CD is fixing the problem for you.

update: This video shows very well we are not talking about sound “style”, but quality. Make sure you are watching the high quality of the YouTube video (there should be a link under the video).

update #2: This video explains in simple words what is this “Loudness war” and how it destroys the sound.

A new version of enhancerepo (0.3.2) is building in the build service.

The new feature is updateinfo metadata generation (patches) support, in a simple but very automatic way, designed specially for testing purposes, but it may be sufficient for people wanting to generate patches for their own repositories.

Using the –generate-update pkg1,pkg2.. option, enhancerepo will look in your repository (and additionally in another base directory), and look for all packages. If a package has multiple versions available (including the base directory) it will create update metadata in the repoparts directory. If you run enhancerepo with the –updates option (either in the same run or not) it will take all repoparts and index them in the updateinfo.xml. This allow to manually edit the patches before indexing them, or to mix automatically generated ones with hand-crafted ones.

As additional coolness it will look for bugs to generate descriptions/references, update type (for example changes containing CVE references or vulnerabilities are tagged with security automatically).

For example, I have 2 amarok packages in my repo, plus one in the 11.0 base directory. This command will look the 3 rpms, and start looking backwards till it finds some changes in the changelog. You can specify multiple packages per “patch”.

# enhancerepo --generate-update amarok --updates --updates-base-dir /space/repo/11.0 /space/repo/duncan2
generating update...
3 versions for 'amarok'
Found change amarok-1.4.10-17-i586.i586 and amarok-1.4.9.1-53-i586.i586.
'amarok' has 1 entries (68/67)
Saving update part to '/space/repo/duncan2/repoparts/update-amarok-1.xml'.
Adding update /space/repo/duncan2/repoparts/update-amarok-1.xml
Saving /space/repo/duncan2/repodata/updateinfo.xml.gz ..
Adding /space/repo/duncan2/repodata/updateinfo.xml.gz to 
       /space/repo/duncan2/repodata/repomd.xml index
repodata/updateinfo.xml.gz already exists. Replacing.
Saving /space/repo/duncan2/repodata/repomd.xml ..

The resulting updateinfo.xml:

<?xml version="1.0" encoding="UTF-8"?>
<updates>
<update status="stable" from="dmacvicar@piscola" type="security" version="1">
  <title>Untitled updatesecurity update 1 for amarok</title>
  <id>amarok</id>
  <issued>1223293050</issued>
  <release>no release</release>
  <description>
    - update to 1.4.10: fix tmp file vulnerability in the Magnatune
    database parsing. Secunia#SA31418 / CVE-2008-3699 / bnc#417232
  </description>
  <references>
    <reference href="http://bugzilla.novell.com/417232" 
       title="bug number 417232" type="bugzilla" id="417232"/>
    <reference 
       href="http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-3699" 
       title="CVE number 2008-3699" type="cve" id="2008-3699"/>
  </references>
  <pkglist>
    <collection>
      <package name="amarok" arch="i586" version="1.4.10" release="17">
        <filename>amarok-1.4.10-17.i586.rpm</filename>
      </package>
    </collection>
  </pkglist>
</update>
</updates>

Some important notes:

• If you generate an update, and then you don’t add new packages or changes, enhancerepo will generate the same update again, but with a new version.