Tilting at windmills

Duncan Mac-Vicar P.'s blog

Leave a comment

Docker with overlayfs on openSUSE

Docker works better with an union filesystem, but the most popular one (aufs) is not included in the mainline kernel because of stability issues.

openSUSE has been shipping the Device Mapper driver until now, but the most promising driver seems to be the one based on the overlayfs union filesystem from SUSE developer Miklos Szeredi. Overlayfs made it upstream, and the kernel 4.0 has all the pieces that docker needs to make it work as a storage backend.

kernel 4.0.0 arrived in the last days to openSUSE Tumbleweed. So provided you already ran “zypper dup” you can switch to this backend with:

Delete all images and containers:

docker rm $(docker ps -a -q)
docker rmi $(docker images -q)

Edit /etc/sysconfig/docker and add a -s option:

DOCKER_OPTS="-s overlay"

Restart docker:

systemctl restart docker

1 Comment

Security vulnerability regressions and continuous integration

When we released SUSE Manager 1.2 back in 2011, one of the first internal components we open-sourced was our internal testsuite and I have written a bit about it before. Our process was from the beginning completely automated to the extreme. If you do a git commit, you can expect the installable .iso file with the bootable SUSE Manager appliance to appear in a folder after some time. Every couple of hours, this .iso is auto-installed in a reference server and reference client servers, which we use to quickly checkout “things”, and additionally, it is installed on a server, where a list of hundred of tests is performed using a human description of the features, and executed in a real Web Browser.

SUSE Manager Continuous Integration

We could not do this without a bunch of great technologies and products: git and github, Jenkins, where we pull the git repositories and build tarballs + rpm spec files. The Open Build Service, where we submit all the tarballs + rpm .spec files and they get built together as a project and later thanks to the KIWI technology inside the Build Service, turned into an appliance. KVM and libvirt, which we use to run the latest version of our product, and then of course Cucumber, which allows us to describe features in a high-level language, and then implement it to be run on Firefox using WebDriver. But recently we have added another set of features to the testsuite: Security regressions. It was Victor, one of our security engineers, who approached us to talk about using OWASP ZAP to find vulnerabilities and harden the product (his presentation). The project describes itself as:

The Zed Attack Proxy (ZAP) is an easy-to-use, integrated penetration-testing tool. It locates vulnerabilities in web applications, and helps you build secure apps. Designed for use by people with a wide range of security experience, it’s also suited for developers and functional testers who are new to penetration testing. With its automated scanner and powerful REST API, ZAP fits seamlessly into your continuous integration environment, allowing you to automate the finding of common issues while you’re still in development.

As soon as he explained how it worked: the passive tests used a proxy, the pieces of the puzzle clicked: we could run the whole testsuite through the proxy, and add the results analysis as part of one of the last steps of the tests. Those tools produce lot of false positives, so we could keep a “baseline” or “ignore list” and still get the testsuite to fail when new vulnerabilities that we haven’t analysed pop up. Integrating it was straightforward. We packaged the ZAP suite as an rpm that gets installed in the machine that is deployed to run the testsuite against the product. The testsuite code is mostly ruby, so we used the owasp_zap gem for which Victor himself is the author. The testsuite starts the proxy, configures Firefox to go through it and run all the tests. There is an optional step before retrieving the results of performing an active attack on the host (eg. finding SQL injections). The last test retrieves the results, compares against the baseline and then embeds the report in the feature steps, which is considered failed or passed whether new vulnerabilities are found. Martin found a way to organize the results better: we normalize some POST parameters that change all the time so that they don’t result in separate incidents and we also group (using the URL path) the incidents per application component.

Cucumber report

We have just started with this: The initials runs immediately showed the need to harden our apache and tomcat configurations but we still have lot of information to analyse and parameters to tune. For example, our developers have found problems in unreleased code commits which were pointed out and fixed immediately, but it would be nice if we could tune the scanner so that those real-life scenarios are detected by the tool, in case a similar mistake is repeated.

1 Comment

SUSE Manager 2.1

Back in March, Christian Stankowic analysed Spacewalk 2.1 and its new user interface look and feel. He asked himself how SUSE Manager would look like:

I really appreciate this update! The new interface looks more clean and well-designed than the elderly look. I’m really interested to see what the implementation in SUSE Manager will look like and whether Red Hat Satellite will also get a new design. :)

Well, now you can see it yourself, as SUSE Manager 2.1 is out!

SUSE Manager   Systems   Systems   Details   Overview

New features include, among others:

  • A slick setup wizard to guide administrators through the basic steps needed to configure a fully operational SUSE Manager: Proxy, Novell Mirror Credentials, SUSE Products.
    SUSE Manager   Admin   Setup Wizard   Mirror Credentials
  • Action chaining that lets administrators bundle and execute related management actions in one step
  • Unattended bare-metal provisioning that allows customers to power on and off and reboot bare-metal systems via the IPMI (Intelligent Platform Management Interface) protocol.
  • OpenScap (the open source implementation of SCAP – Security Content Automation Protocol), a standardized approach to maintaining enterprise system security.
  • CVE Auditing. This feature goes beyond telling you pending patches but instead assisting you to assign the right content to your systems: what vulnerabilities affect you where you haven’t yet assigned the right channels. For example, you may have an affected system in production. CVE Auditing may tell you that you can fix the security issue by assigning the stage channel to a system.
    CVE Audit
  • Locks packages on the server which are then enforced on the client side (eg. if you login via ssh to the client).

And of course, most of the work is already merged upstream.





Leave a comment

Trying Rust (language) on openSUSE

I am always playing with new languages. I love learning the thinking and philosophy behind them. Usually I throw them away after one evening (mostly mee-too’s), but there are some that are very interesting and get explored a bit further, and some are of course “adopted” into your toolbox.

The last ones that I spend quite some time with were:

  • Scala, which I used as base language for some courses
  • Go, which I used with the intention of writing servers for the Raspberry Pi in combination with Javascript browser apps

I was expecting to find in Go the “right” native-compiled language. Go is simple, has great (integrated) tooling, very pragmatic syntax, etc. But it has a bunch of warts that just made me very uncomfortable when translating ideas into code:

  • It does not support generics. Lists of voids (interface{})? no thanks.
  • For a language having first class functions. the functional aspects of is close to zero. After you did some ruby is hard to think again without using expressions and functional constructs like map/select/etc. “if” as a statement where you assign nil to the variable first… nope

At that time Mozilla’s Rust was still  immature and I did not manage to build a decent package to start playing with it.

Rust is a nice language. It is a bit noisy from the syntax, but it has been improved a lot in the latest releases. Garbage collected pointers (@) are gone and moved to the standard library which means there is now only ~ and &. But it is a language that got a lot (almost everything) “right”:

  • For having lot of features, it is very lightweight if you use the basics
  • It has decent integration with C
  • It has good support for concurrency using light tasks and channels
  • Supports something more flexible than classes, called traits
  • The toolchain is modern and very integrated
  • It does not reinvent everything. No own build system, and that is probably why it fits so well with Linux packaging
  • It uses LLVM as the backend, so we can expect good portability and getting all the optimization work from the LLVM guys for free
  • It has the feeling of ruby: object-oriented, with some functional sugar
  • Pattern matching!

But 0.9 was just released and this time I found some very decent .spec files to start with. I improved those with:

  • Do not install compiler related libraries to /usr/lib
  • Make parallel installation possible using /etc/alternatives

This is very similar how the Java package is structured. I also built a package for rust-bindgen, which uses clang to parse headers and generate bindings for C libraries.

So, grab one of the repositories and:

zypper ar http://download.opensuse.org/repositories/home:/dmacvicar:/rust/openSUSE_13.1/home:dmacvicar:rust.repo
zypper ref
zypper install rust

Then continue with the tutorial and the manual. Happy learning!.


2013 Personal Learning Retrospective (Software)

  • At work, It was an intense and exciting year:

    • I got the chance to focus more in one product.
    • Grew and focused on building the “new” team.
    • Lot of contributions from the team to upstream. And the momentum is just growing.
    • Had the chance to get my hands dirty a lot, and it was very useful to do it while building the team.
    • Because of working very close with development, I got to refresh a lot skills on HTML5, Javascript, CSS and Java.
    • Participated in a lot of interviews of people applying to the company.
    • Had the chance to take part in Scrum training.
    • While hacking my own development tools, learned a lot about libvirt and its API.
    • Redid one of my internal dashboard/report tools with angularjs.
    • Did two ruby-rpm (ffi) releases.
  • I completed two classes at Coursera:

    I am very proud of those two achievements because it was not easy. Both courses involved between 5-10 hours a week of lectures and assignments (the 2nd was harder…) and I did them entirely in my private time and almost without disrupting my day job or family duties. I had to organize myself better. It still meant some weeks going very late to bed and trying to do the assignments while being very tired. But I managed to complete them all: got “with distinction” in the first course and I hope also in the second.

    But I am also happy because both courses give you knowledge that is very applicable to the current problems in Software. With more cores but same stupid brain, you need new models, languages and tools for writing concurrent, distributed and resilient software. These courses gave me a very good foundation into it.

  • Various misc. hacking & learning:

    • A front-end in angularjs with a web service in Go (intended to be run on the Raspberry Pi later on).
    • Played a bit with compiler construction using Parsing Expression Grammars. Discovered the beauty of Parslet (ruby), but did most of the experiments with greg/C++.
    • Learned about lua (and bought the book), and constructed a small tool by embedding its JVM based implementation luaj.


    While it is very hard for me to plan what will get me motivated or enthusiastic over the year, at least from now, I am looking forward to:

    • Travel a bit more. In 2013 it was hard and my wife needed to attend conferences more than me. First step in that direction is Devconf.cz.
    • Do some courses about Music/Sound theory.
    • Hack on some abandoned experiments in my git repos or contribute to some similar topic. Compilers, may be Music, not sure yet.
    • Look at some new ideas in the web arena like react.
    • Learn some Rust and follow either Kotlin or Ceylon on the JVM.

1 Comment

Microsoft Sculpt Ergonomic Desktop


After eight years I have upgraded my Microsoft Ergonomic keyboard 4000 to a newer model. Microsoft hardware has been always very good in my opinion, but in this product iteration you can see how much they have learn about design.


What I like:

  • Magnetic battery covers
  • Magnetic accessories
  • All extra useless buttons gone
  • Numeric pad now a separate accessory leaving your mouse much closer to the keyboard
  • Much shorter keypress and better feeling
  • You can store the dongle inside the mouse


What I don’t like:

  • USB dongle is bigger than the Logitech one.
  • If I want to mix the Logitech MX mouse with the Microsoft Keyboard one I need two dongles
  • You can’t recharge the mouse batteries with an USB cable
  • The keyboard has a weird key layout at the right side. I hope I can get used to it.


Dynamic grid columns with Twitter Bootstrap

In one page of our application, some boxes are displayed in two columns. Sometimes, however, one of the boxes is not displayed.

We did not want the box empty space to stay there. Calculating the grids on the server-side was not the way to go either.

So, with some javascript and we have .col-md-auto (works with other sizes too):

$(document).on("ready", function() {
  $.each(['xs', 'sm', 'md', 'lg'], function(idx, gridSize) {
    $('.col-' + gridSize + '-auto:first').parent().each(function() {
      //we count the number of childrens with class col-md-6
      var numberOfCols = $(this).children('.col-' + gridSize + '-auto').length;
      if (numberOfCols > 0 && numberOfCols < 13) {
        minSpan = Math.floor(12 / numberOfCols);
        remainder = (12 % numberOfCols);
        $(this).children('.col-' + gridSize + '-auto').each(function(idx, col) {
          var width = minSpan;
          if (remainder > 0) {
            width += 1;
          $(this).addClass('col-' + gridSize + '-' + width);
<div class="row">
  <div class="col-md-auto">
  <div class="col-md-auto">
  <div class="col-md-auto">
  <div class="col-md-auto">
  <div class="col-md-auto">

Turns into:

<div class="row">
  <div class="col-md-auto col-md-3">
  <div class="col-md-auto col-md-3">
  <div class="col-md-auto col-md-2">
  <div class="col-md-auto col-md-2">
  <div class="col-md-auto col-md-2">

Only works with < 12 columns :-)


Get every new post delivered to your Inbox.