August 27, 2009

ruby block syntax

Reading a commit from DHH I found out a syntax I was not aware of.

Instead of doing:

[1,2,3,4,5,6,7,8,9,10].select { |x| x.even? }

You can do:

[1,2,3,4,5,6,7,8,9,10].select(&:even?)

When I asked in #rubyonrails irc channel about this syntax, I got enlightened: The first part is obvious: last argument starting with & is interpreted as a Proc and equivalent to passing a block, so it is not a &: operator but & and a symbol. How does the symbol work?

According to the people in the channel, it is rails magic. When I said that it works for me in irb I was told I must be running ruby 1.9, which I am not. After some discussion we realized this magic is in ruby 1.8.7 too.

The magic is of course, just Symbol#to_proc method ;-)

Rails engines and static assets (public/)

I was very happy that Rails Engines got merged in Rails 2.3. However I never liked that engines did not support overlaying static assets like they do with controllers, models and views. Even worse, the discussion was around “mirroring” the engine’s assets into the application public/ directory (like you can see in this railcast).

I imagined that doing something more transparent would be non-trivial. But I never asked myself why. When mentioning this limitation to Daniel when explaining how we use engines in our code, he asked the magical question: “Why?”.

I have been looking at Rails’s Rack support recently. I love Rack. And if you see how Rails builds its complex pieces over such simple components based on Rack, you realize how good Rack is for the Ruby community as a base component.

So, I tried the very first though: Create a Rake middleware called StaticOverlay that is initialized with various overlay directories. When the middleware is called it will look for the file in the overlays, and if it finds it in one, it will serve it (using Rack::File), if not, it just call the application (or next middleware in the chain).

Once I put the rack middleware together it worked out of the box, so I started a thread in rails core list to find out whether this would be a valid approach to hack a real patch and add support for this in Rails itself.

March 19, 2009

Easy packaging of ruby gems for openSUSE, part II

I was in the need of learning about osc plugins, because I want to rewrite some scripts that I use to track maintenance updates to the software management stack. Learning osc plugins meant learning python at the same time.

I was inspired by the osc gnome todo functionality, which shows something like this:

vuntz@lyon ~/work/opensuse/tmp/>osc gnome todo
Downloading data in a cache. It might take a few seconds...
Package  | openSUSE:Factory | GNOME:Factory    | Upstream
---------+------------------+------------------+-------------
anjuta   | 2.23.91          | 2.23.91          | 2.24.0.1 (r)
at-spi   | 1.23.92          | 1.23.92          | 1.24.0
gdm      | 2.23.92          | 2.23.92 (s)      | 2.24.0 (s)
gedit    | 2.23.93          | 2.23.93          | 2.24.0

So, in my last post, I wrote how to easily package gems for openSUSE, saving at least 99% of the work. However, how do you know that you need to package a new gem? Well, here is osc ruby todo, in its first 0.0.0.0.1 release:

duncan@tarro:/osc-plugins> osc ruby todo
126 gems in build service projects
Retrieving upstream gem information...
+ libxml-ruby upstream: 1.1.2 bs: 0.9.5
+ radiant upstream: 0.7.1 bs: 0.6.9
+ launchy upstream: 0.3.3 bs: 0.3.2
+ hpricot upstream: 0.7 bs: 0.6.161
+ net-ssh upstream: 2.0.11 bs: 2.0.8

So, it connects to gems.rubyforge.org and tell you which gems are outdated. Do you want to keep openSUSE a nice ruby platform? help me. The following features are waiting:

  • track ruby package versions
  • track other gem repositories, like the rails one or github
  • also compare it with version in factory
  • nice tables
  • caching
  • Use gem2rpm and the openSUSE template to automatically create the specs for outdated gems

You can download it from git.opensuse.org, just symlink the .py file to ~/.osc-plugins. It requires rpm-python and Python 2.6 or newer.

My ultimate goal is to get a similar one for zypp/yast that allows to compare our Head repositories with Factory and maintenance repos, show diffs and make submit requests.

So how does it works? How to create your own osc plugin? Look at the source code of the osc-* projects at git.opensuse.org, You will realize it is as easy as:

  • Open a .py file
  • Create a do_something method, adding the documentation and checking input and arguments
  • Use the meta_ API and urllib to get data from the build service
  • Use makeurl to build the right authenticated urls to do the requests.
March 17, 2009

Easy packaging of ruby gems for openSUSE

Every distribution has its own conventions to package scripting languages addons (perl modules, ruby gems, etc) as native packages. This allows packages to depend on those addons honoring the package dependencies, and at the same time, look like the addon was installed the scripting language tool (cpan, gem).

openSUSE keeps a repository of gems, thanks to Marcus Rueckert.

For ruby, David Lutterkort (augeas lead developer) created a nice tool to help converting gems into spec files.

I created a template for openSUSE-like rubygem-* packages, and David committed it upstream.

So, to create a package for a gem:

Fetch it:

gem fetch foo

Convert it:

gem2rpm -t opensuse.spec.template ./foo-1.1.gem > rubygem-foo.spec

Build and tweak it. Make sure everything is alright. Some gems work out of the box, some not, but still gem2rpm saves 90% of the effort. Consider submitting it to the project if you are willing to keep it up to date ;-)

January 5, 2008

Welcome 2008

Some random thoughts about the world, and the past and following years.

Digital society

2007 was quite active on the digital rights topic.

The inability of the industry to catch up with the current society has created a war on digital rights. Software patents on one side. Digital music transformed music labels into mafias and consumers into rival groups. Politicians trying to implement surveillance systems everywhere.

I have the feeling that we will see some progress on the music topic. Record labels will give up, but it will be too late, and if a bunch of major artists start to use some fair system. Something like amiestreet.com or direct selling comes to my mind.

I don’t think something will happen on patents.

It was funny, some weeks ago I got an idea about using gps to associate location to todo items. This plays well with the getting things done methodology where you organize by contexts and not categories. I started prototyping some stuff on android.

Sadly, I found out this simple idea was patented by Fujitsu. Not only that. But I found a program which does that, and the website dissapeared. Another news article about someone researching on the topic and developing a product on that also dissapeared from the map. However, I haven’t yet seen a product from Fujitsu on the topic (the patent is 7 years old). Software patents destroy innovation. Thanks to that stupid patent, you won’t see any product (unless free software) using that.

The web 2.0

Everybody is sick of the Web 2.0 buzz. The Web 2.0 exists.

It is normal that consultants/analysts start to invent new terms because their business depends on the next “big thing” that will “cut costs” and “save millions” to your company. They repeat the same year after year just replacing the term itself.

However the amount of services on the web is growing really fast, and they are all accessible by really standard protocols. Software is becoming just a support medium and the value is being transfered to services: information, storage, security, etc.

Now, there are new layers over that. Phones with gps will bring a new dimension of services based on our location. This is very important. The information we store on the web becomes more relevant if we map it over real-world dimensions: location, time, mood, energy, context. Open source fits here, you can see companies like Google taking advantage of it.

Question: How services will affect open source and/or free software itself? Google contributes quite a lot to open source software. But once you don’t distribute the software, you are not forced to publish modifications. Will other companies follow this path?

Amazon Web Services is another topic. The way they sell on demand “computing power”, “human processing”, “databases” and “storage” is simply amazing.

I would like to see more about “distributed” environments. I am disappointed on how I have to manage my information having 3 computers and one cell phone. There has to be something better than either being off-line and centralized or being online and ubiquitous (where network is available). I want to be ubiquitous, distributed, fail tolerant, and in a simple, pragmatic way. (I don’t want to setup a cluster on my devices).

What about the bubble?. Yes, there is a bubble. There are a bunch of companies that know what they are doing. And thousands of venture capital groups funding whoolalalhzuzu.com ajax websites which implement a calculator or whatever. Those dying is not a bubble, it is natural selection. Most people already know which ones will die after using them for 2 minutes.

I am really excited about the developments in this area and looking forward what is coming here. The direction is clear.

Software

  • openSUSE / YaST

    I will leave this for a separate post.

  • KDE 4.0

    4.0 is being released in a few days and you will see the most brave release of free software ever. A big bunch of new technologies and visions collected, cooked and packed inside a great community. And better, there is still no big place for politics in KDE, but technical arguments and user experience. Not that all desktops could say the same.

Software Development

Wow, what happened on 2007?

  • Software configuration / Version Control

    The growing complexity of open source codebases, plus the need to maintain them for enterprise purposes, brought the topic of version control really hard on the blog sphere. Every blog and developer talked about git. Lot of talk about mercurial and bzr too. 3 version control systems being popular at the same time? The point is that being “distributed” is “the thing”. I personally switched to git, and it solved the “being distributed” part of working with 3 computers in different places. I want to see something like code.google.com with git support.

  • Android

    Brilliant. I am waiting for the first phone. Some APIs are ugly. But still prettier than uggly guys that reinvent the wheel poorly, and worse, only on Windows.

Nothing that spectacular on other old topics:

  • Java

    While Eclipse is a jewel. Sun is getting better but too slow to move. So slow that it is getting boring to watch.

  • Ruby

    We saw the release of ruby 1.9 on December, a very important milestone. At the same time, JRuby is now fast and very compatible, and other implementations are also very active.

  • C++

    Even more boring than Java eh?

Politics

  • Chile

    The goverment of Michelle Bachelet whose goverment improvisation has made the country again miss the opportunity to develop quickly. Michelle has no strategy at all, so the hope for 2008 is that his sucking team don’t make more mistakes. The public transport system ( Transantiago ) has to start working somehow (both in operation and budget), because till now, it is a joke.

    The opposition hasn’t a good alternative. Nobody is willing to make the important change: universal free and good education, health and social care. Even Michelle, being a socialist, uses the private health and education system.

  • Europa

    Spain’s election coming. Seems that Zapatero will be reelected, which seems reasonable. I am a little lost with german politics and I feel like living in a fantasy world. Time to change that. Still, Europe’s economy is going good and living here is awesome. I love it.

  • USA

    Discussions on whether they should teach non-science on science class?. War. etc. Uhm… was I writing about a middle west country? I am sad, really sad to see a beautiful country being destroyed, destroying, hating and being hated by almost the entire world, and even worse, considered the biggest threat to the rest of the world.

    From latimes.com:

    36% of European poll respondents — who come from Italy, France, Germany, Britain, and Spain consider America as the No. 1 danger to world peace. Even 35% of American 16- to 24-year-olds identify their own country as the chief danger to peace. The poll was consistent with findings by the Pew Global Attitudes Project, which found that favorable ratings of the U.S. had declined in 26 of 33 countries over the last five years. Europeans next concerns are China, 19%; Iran 17%; Iraq 11%; North Korea 9%; Russia 5%.

    Elections aren’t this year. Lets see how it goes.

December 21, 2007

Encapsulation and don’t repeat yourself

Richard Moore comment on Access Anxiety explains why Encapsulation is a good thing. I think both are missing the point. Nobody doubts that encapsulation is a good thing, and Michael Feathers tries to present ruby way as a more “relaxed” thing.

The author was comparing the ‘ruby style’ of direct access to member variables with the getter/setter pattern common in Java code

So what is the ruby style really?

The annoying thing about java setters is that they are functions. So you either start with them (which is overkill for simple things) or don’t use them and then you are lost when you want to add things like Richard showed.

Ruby on the other hand, makes it easy, because you can’t expose member variables, but you can very easily create default accessors.

Michael Feathers example is wrong. Usually you start by actually offering a simple default accessor.

class GlazeObject
  attr_accessor :store, :formatter

  def initialize
    @store = DefaultStore.new
    @formatter = DefaultFormatter.new
  end
end

Now, if you need something more special, like Richard example:

public void setFormatter(Formatter formatter) {
        this.formatter = formatter;
        this.expensive = doCalculation(formatter);
    }

You can just remove attr_accessor :formatter and add to the class:

class GlazeObject
  def formatter
    return @formatter
  end

  def formatter=(f)
    @formatter = f;
     @expensive = doCalculation(f);
  end
end

What is the difference? In ruby, for client code consuming this class, you always see obj.formatter for both cases. In java you have to start from the beginning copy-pasting repetitive accessor code if you don’t want to start changing obj.formatter to obj.getFormatter() and obj.formater = something to obj.setFormatter(something).

C# has a similar feature, called properties.

public class Person
{
  private int _age; 
  public int age
  {
   get
   {
     return _age;
   }
   set
   {
     _age = value;
   }
  }
}

No matter if we start with a public variable called age, we can move it to setters and getters (which are the providers of encapsulation without having to change the operation mode from variables to functions.

November 23, 2007

ycp is not only a language. YaST ruby bindings improvements

After quite lot of hacking nights, I managed to get some yast2 ruby bindings improvements working.

You might know that ycp is not only a language, but a component technology. Every component using the YaST2 technology can be reused from other YaST components no matter what language they are written in, as long as there are bindings for that. That was the original intention of the ruby bindings.

So for example, there is a component call Storage, which is written in the ycp language. The component (well, it is really a namespace, but YaST sees it as a namespace component) is located in /usr/share/YaST2/modules/Storage.ycp (.ybc for the bytecode compiled version) and offers some medium level functions that YaST uses.

This component needs low level access to the system, for which there is another module called Libstorage ( located in /usr/share/YaST2/modules/LibStorage.pm ) which in this case is written in perl which is a language YaST supports as long as you follows some simple rules.

The reason is because libstorage is a C++ library, and there is no easy way to bind it to the YaST environment, but as YaST supports perl, you can generate perl bindings for the C++ library and all the library functionality will be available to other components in YaST.

So repeat after me. ycp is not only a language. It is a communication protocol, used by YaST component framework.

So, with the ruby bindings, you where able to do:

require 'yast'

m = YaST::Module.new("Storage")
dp = m.GetDiskPartition("/dev/sda1")
dp.each do | key, value |
  puts "#{key} #{value}"
end

But I was never happy with the syntax. The idea was that you construct a dynamic proxy for a YaST component and then call methods on it. This had some disadvantages:

  • If the component did not exist, you did not get an error till the method call
  • It was not possible to do introspection to the component, because the methods call where implemented using the method_missing hook. So basically the proxy responded to all methods and only raised an error if the equivalent call to the YaST side was not possible.

So I decided to look for a new approach. This is the result till now:

require 'yast'
require 'ycp/storage'

dp = YCP::Storage::GetDiskPartition("/dev/sda1")
dp.each do | key, value |
  puts "#{key} #{value}"
end

Oh yes, you notice. First, there is no need to instantiate the module, a classic require will do it. The bindings modify the built-in require method, so if you require something under ycp/, a module is created on the fly. A real module, a real symbol. You can also import it manually using YCP::import

Now, you can do this:

puts YCP::Storage.methods

When the module is imported, the symbols are declared in the module, so you can ask for them in the module. I still need to work on other symbol types, but it should be straight forward.

The last improvement is that I simplified the C part of the bindings a lot, and moved most of the magic to pure ruby code, which makes easier to work with.

Next improvement, the other way around, making nicer how to call ruby components from any other component.

October 21, 2007

Taskjuggler

Surprised to find out that Chris is writing Taskjuggler3 from scratch, and this time, in Ruby:

TaskJuggler 3.x is a from-scratch rewrite of TaskJuggler. It’s currently in its infancy and only supports a minimum feature set. It is not intended for users yet. Please join our developer forum if you are interested in contributing. In contrast to the 2.x version TaskJuggler 3.x is written in Ruby.

There is a git repository in

http://www.taskjuggler.org/git-repos/taskjuggler3.git

in case you want to give it a look.

September 19, 2007

yast2-ruby-bindings user interface support

I just commited support for using the user interface (yes, that is Qt, GTK or ncurses) from the ruby language:

require 'yast'
ui = YaST::Module.new("UI")
YaST::Ui::init("qt")
include YaST::Ui

t = HBox( Label("Welcome to Ruby!"), PushButton("Push me") )

puts "#{t.to_s} #{t.class}"

ui.OpenDialog(t)
ui.UserInput()

Result: Image Hosted by ImageShack.us

June 28, 2007

Hackweek III (day 4)

Martin Vidner succeeded in making possible to use YaST UI:: from the language bindings. This means you could write a complete module in Perl, and get the benefits of the abstracted Qt/ncurses/gtk user interface.

These changes means that my Ruby bindings will be able to support it too.

I also got the code as yast2-bindings-ruby in the build service, it doesn’t build yet, but soon will appear in the home:dmacvicar project.

Srinivasa Ragavan is working on Desktop status awareness, which means trying to do useful tasks when you are away from your computer. I pointed him to the Kopete motion-auto-away plugin I hacked some years ago in order to set the user away in the messaging client if no motion was detected in a webcam stream.

« Older Entries