Tilting at windmills

Duncan Mac-Vicar P.'s blog


1 Comment

Security vulnerability regressions and continuous integration

When we released SUSE Manager 1.2 back in 2011, one of the first internal components we open-sourced was our internal testsuite and I have written a bit about it before. Our process was from the beginning completely automated to the extreme. If you do a git commit, you can expect the installable .iso file with the bootable SUSE Manager appliance to appear in a folder after some time. Every couple of hours, this .iso is auto-installed in a reference server and reference client servers, which we use to quickly checkout “things”, and additionally, it is installed on a server, where a list of hundred of tests is performed using a human description of the features, and executed in a real Web Browser.

SUSE Manager Continuous Integration

We could not do this without a bunch of great technologies and products: git and github, Jenkins, where we pull the git repositories and build tarballs + rpm spec files. The Open Build Service, where we submit all the tarballs + rpm .spec files and they get built together as a project and later thanks to the KIWI technology inside the Build Service, turned into an appliance. KVM and libvirt, which we use to run the latest version of our product, and then of course Cucumber, which allows us to describe features in a high-level language, and then implement it to be run on Firefox using WebDriver. But recently we have added another set of features to the testsuite: Security regressions. It was Victor, one of our security engineers, who approached us to talk about using OWASP ZAP to find vulnerabilities and harden the product (his presentation). The project describes itself as:

The Zed Attack Proxy (ZAP) is an easy-to-use, integrated penetration-testing tool. It locates vulnerabilities in web applications, and helps you build secure apps. Designed for use by people with a wide range of security experience, it’s also suited for developers and functional testers who are new to penetration testing. With its automated scanner and powerful REST API, ZAP fits seamlessly into your continuous integration environment, allowing you to automate the finding of common issues while you’re still in development.

As soon as he explained how it worked: the passive tests used a proxy, the pieces of the puzzle clicked: we could run the whole testsuite through the proxy, and add the results analysis as part of one of the last steps of the tests. Those tools produce lot of false positives, so we could keep a “baseline” or “ignore list” and still get the testsuite to fail when new vulnerabilities that we haven’t analysed pop up. Integrating it was straightforward. We packaged the ZAP suite as an rpm that gets installed in the machine that is deployed to run the testsuite against the product. The testsuite code is mostly ruby, so we used the owasp_zap gem for which Victor himself is the author. The testsuite starts the proxy, configures Firefox to go through it and run all the tests. There is an optional step before retrieving the results of performing an active attack on the host (eg. finding SQL injections). The last test retrieves the results, compares against the baseline and then embeds the report in the feature steps, which is considered failed or passed whether new vulnerabilities are found. Martin found a way to organize the results better: we normalize some POST parameters that change all the time so that they don’t result in separate incidents and we also group (using the URL path) the incidents per application component.

Cucumber report

We have just started with this: The initials runs immediately showed the need to harden our apache and tomcat configurations but we still have lot of information to analyse and parameters to tune. For example, our developers have found problems in unreleased code commits which were pointed out and fixed immediately, but it would be nice if we could tune the scanner so that those real-life scenarios are detected by the tool, in case a similar mistake is repeated.


1 Comment

SUSE Manager 2.1

Back in March, Christian Stankowic analysed Spacewalk 2.1 and its new user interface look and feel. He asked himself how SUSE Manager would look like:

I really appreciate this update! The new interface looks more clean and well-designed than the elderly look. I’m really interested to see what the implementation in SUSE Manager will look like and whether Red Hat Satellite will also get a new design. :)

Well, now you can see it yourself, as SUSE Manager 2.1 is out!

SUSE Manager   Systems   Systems   Details   Overview

New features include, among others:

  • A slick setup wizard to guide administrators through the basic steps needed to configure a fully operational SUSE Manager: Proxy, Novell Mirror Credentials, SUSE Products.
    SUSE Manager   Admin   Setup Wizard   Mirror Credentials
  • Action chaining that lets administrators bundle and execute related management actions in one step
  • Unattended bare-metal provisioning that allows customers to power on and off and reboot bare-metal systems via the IPMI (Intelligent Platform Management Interface) protocol.
  • OpenScap (the open source implementation of SCAP – Security Content Automation Protocol), a standardized approach to maintaining enterprise system security.
  • CVE Auditing. This feature goes beyond telling you pending patches but instead assisting you to assign the right content to your systems: what vulnerabilities affect you where you haven’t yet assigned the right channels. For example, you may have an affected system in production. CVE Auditing may tell you that you can fix the security issue by assigning the stage channel to a system.
    CVE Audit
  • Locks packages on the server which are then enforced on the client side (eg. if you login via ssh to the client).

And of course, most of the work is already merged upstream.

 

 

 

 


Leave a comment

Trying Rust (language) on openSUSE

I am always playing with new languages. I love learning the thinking and philosophy behind them. Usually I throw them away after one evening (mostly mee-too’s), but there are some that are very interesting and get explored a bit further, and some are of course “adopted” into your toolbox.

The last ones that I spend quite some time with were:

  • Scala, which I used as base language for some courses
  • Go, which I used with the intention of writing servers for the Raspberry Pi in combination with Javascript browser apps

I was expecting to find in Go the “right” native-compiled language. Go is simple, has great (integrated) tooling, very pragmatic syntax, etc. But it has a bunch of warts that just made me very uncomfortable when translating ideas into code:

  • It does not support generics. Lists of voids (interface{})? no thanks.
  • For a language having first class functions. the functional aspects of is close to zero. After you did some ruby is hard to think again without using expressions and functional constructs like map/select/etc. “if” as a statement where you assign nil to the variable first… nope

At that time Mozilla’s Rust was still  immature and I did not manage to build a decent package to start playing with it.

Rust is a nice language. It is a bit noisy from the syntax, but it has been improved a lot in the latest releases. Garbage collected pointers (@) are gone and moved to the standard library which means there is now only ~ and &. But it is a language that got a lot (almost everything) “right”:

  • For having lot of features, it is very lightweight if you use the basics
  • It has decent integration with C
  • It has good support for concurrency using light tasks and channels
  • Supports something more flexible than classes, called traits
  • The toolchain is modern and very integrated
  • It does not reinvent everything. No own build system, and that is probably why it fits so well with Linux packaging
  • It uses LLVM as the backend, so we can expect good portability and getting all the optimization work from the LLVM guys for free
  • It has the feeling of ruby: object-oriented, with some functional sugar
  • Pattern matching!

But 0.9 was just released and this time I found some very decent .spec files to start with. I improved those with:

  • Do not install compiler related libraries to /usr/lib
  • Make parallel installation possible using /etc/alternatives

This is very similar how the Java package is structured. I also built a package for rust-bindgen, which uses clang to parse headers and generate bindings for C libraries.

So, grab one of the repositories and:

zypper ar http://download.opensuse.org/repositories/home:/dmacvicar:/rust/openSUSE_13.1/home:dmacvicar:rust.repo
zypper ref
zypper install rust

Then continue with the tutorial and the manual. Happy learning!.


3 Comments

2013 Personal Learning Retrospective (Software)

  • At work, It was an intense and exciting year:

    • I got the chance to focus more in one product.
    • Grew and focused on building the “new” team.
    • Lot of contributions from the team to upstream. And the momentum is just growing.
    • Had the chance to get my hands dirty a lot, and it was very useful to do it while building the team.
    • Because of working very close with development, I got to refresh a lot skills on HTML5, Javascript, CSS and Java.
    • Participated in a lot of interviews of people applying to the company.
    • Had the chance to take part in Scrum training.
    • While hacking my own development tools, learned a lot about libvirt and its API.
    • Redid one of my internal dashboard/report tools with angularjs.
    • Did two ruby-rpm (ffi) releases.
  • I completed two classes at Coursera:

    I am very proud of those two achievements because it was not easy. Both courses involved between 5-10 hours a week of lectures and assignments (the 2nd was harder…) and I did them entirely in my private time and almost without disrupting my day job or family duties. I had to organize myself better. It still meant some weeks going very late to bed and trying to do the assignments while being very tired. But I managed to complete them all: got “with distinction” in the first course and I hope also in the second.

    But I am also happy because both courses give you knowledge that is very applicable to the current problems in Software. With more cores but same stupid brain, you need new models, languages and tools for writing concurrent, distributed and resilient software. These courses gave me a very good foundation into it.

  • Various misc. hacking & learning:

    • A front-end in angularjs with a web service in Go (intended to be run on the Raspberry Pi later on).
    • Played a bit with compiler construction using Parsing Expression Grammars. Discovered the beauty of Parslet (ruby), but did most of the experiments with greg/C++.
    • Learned about lua (and bought the book), and constructed a small tool by embedding its JVM based implementation luaj.

    2014

    While it is very hard for me to plan what will get me motivated or enthusiastic over the year, at least from now, I am looking forward to:

    • Travel a bit more. In 2013 it was hard and my wife needed to attend conferences more than me. First step in that direction is Devconf.cz.
    • Do some courses about Music/Sound theory.
    • Hack on some abandoned experiments in my git repos or contribute to some similar topic. Compilers, may be Music, not sure yet.
    • Look at some new ideas in the web arena like react.
    • Learn some Rust and follow either Kotlin or Ceylon on the JVM.


1 Comment

Microsoft Sculpt Ergonomic Desktop

IMG_20131112_190154

After eight years I have upgraded my Microsoft Ergonomic keyboard 4000 to a newer model. Microsoft hardware has been always very good in my opinion, but in this product iteration you can see how much they have learn about design.

IMG_20131112_190235

What I like:

  • Magnetic battery covers
  • Magnetic accessories
  • All extra useless buttons gone
  • Numeric pad now a separate accessory leaving your mouse much closer to the keyboard
  • Much shorter keypress and better feeling
  • You can store the dongle inside the mouse

IMG_20131112_190226

What I don’t like:

  • USB dongle is bigger than the Logitech one.
  • If I want to mix the Logitech MX mouse with the Microsoft Keyboard one I need two dongles
  • You can’t recharge the mouse batteries with an USB cable
  • The keyboard has a weird key layout at the right side. I hope I can get used to it.


1 Comment

Dynamic grid columns with Twitter Bootstrap

In one page of our application, some boxes are displayed in two columns. Sometimes, however, one of the boxes is not displayed.

We did not want the box empty space to stay there. Calculating the grids on the server-side was not the way to go either.

So, with some javascript and we have .col-md-auto (works with other sizes too):

$(document).on("ready", function() {
  $.each(['xs', 'sm', 'md', 'lg'], function(idx, gridSize) {
    $('.col-' + gridSize + '-auto:first').parent().each(function() {
      //we count the number of childrens with class col-md-6
      var numberOfCols = $(this).children('.col-' + gridSize + '-auto').length;
      if (numberOfCols > 0 && numberOfCols < 13) {
        minSpan = Math.floor(12 / numberOfCols);
        remainder = (12 % numberOfCols);
        $(this).children('.col-' + gridSize + '-auto').each(function(idx, col) {
          var width = minSpan;
          if (remainder > 0) {
            width += 1;
            remainder--;
          }
          $(this).addClass('col-' + gridSize + '-' + width);
        });
      }
    });
  });
});
<div class="row">
  <div class="col-md-auto">
    <p>Hi</p>
  </div>
  <div class="col-md-auto">
    <p>Hi</p>
  </div>
  <div class="col-md-auto">
    <p>Hi</p>
  </div>
  <div class="col-md-auto">
    <p>Hi</p>
  </div>
  <div class="col-md-auto">
    <p>Hi</p>
  </div>
</div>

Turns into:

<div class="row">
  <div class="col-md-auto col-md-3">
    <p>Hi</p>
  </div>
  <div class="col-md-auto col-md-3">
    <p>Hi</p>
  </div>
  <div class="col-md-auto col-md-2">
    <p>Hi</p>
  </div>
  <div class="col-md-auto col-md-2">
    <p>Hi</p>
  </div>
  <div class="col-md-auto col-md-2">
    <p>Hi</p>
  </div>
</div>

Only works with < 12 columns :-)

what-we-have


7 Comments

Modernizing Spacewalk’s user interface

Spacewalk is SUSE Manager upstream project. We base our product on Spacewalk’s codebase and contribute features and fixes back.

Screenshot from 2013-10-30 09:42:09

There are lot of advantages of working in a mature codebase. At the same time, you get to work everyday with legacy code.

We want SUSE Manager to look great and behave like a modern web application. We want it to survive a rapidly changing environment that gives you new challenges like mobile. Our main goals were:

  • Better mobile responsiveness
  • A standard and documented CSS framework as a base
  • Easier customization of the CSS (e.g. LESS)
  • Deprecate the Prototype library and use JQuery
  • Nicer icons

The topic was first discussed as “what could we do” in order to keep everything current and relevant. Then we presented a quick prototype migrating the user interface to something based on Twitter Bootstrap. This is what the prototype looked like:

Image

The idea was well received. There were (valid) concerns. Spacewalk is a huge application, and a refactoring like this was going to be an intrusive change.

For our department workshop, we assembled a squad and did a very succesful first iteration. We managed to port the main structure and also get a first Spacewalk “black theme”. Some of us continued for the SUSE Hackweek. After that we integrated the development into our main sprints and at some point we started pushing our branch to upstream.

What do we have right now?

  • A mobile responsive layout based on Bootstrap 3, the most popular CSS framework.
  • Scalable font-based icons powered by Font Awesome 4
  • A beautiful Spacewalk “black theme”
  • Most of the code was refactored towards HTML5, current CSS, jQuery and LESS.

laptop-mobile

what-we-have-screenshot-002 what-we-have-screenshot-001

spacewalk-mobile

More detailed screenshots of how it looked before on mobile 1 & 2. How it looks now: 1 & 2 & 3 & 4

There are still some items in our backlog:

  • Un-styled pages here and there
  • Cosmetic details due to “misbehaving” html (manual breaks, inline styles)
  • Custom tags unit tests (WIP)
  • Perl-part data list widget (but the layout is there)
  • Replacing gifs with icons (WIP)

Now our main goal is a first upstream merge so that we can continue working on this without blocking other features where we want to take advantage of the base user interface.

I hope you enjoyed this report. I will follow up with a more technical post detailing some refactorings and how we automated some tedious changes.

Follow

Get every new post delivered to your Inbox.