February 11, 2010

You don’t need Kopete Facebook plugin anymore

In May 2008, Facebook announced that they were planning to add XMPP (a.k.a Jabber), the standard messaging protocol behind Google Talk and other chat programs, to their Facebook Chat solution.

In May 2009, seeing that nothing happened, I announced that I was working on Facebook support for Kopete and released a prototype on github. The plugin was not perfect, and it was talking to Facebook using non-standard ways (including html scrapping!), but allowed people to see their contacts and chat.

Yesterday, Facebook finally announced XMPP support. This means various things:

  • You can connecto to Facebook chat using any XMPP compatible program: Kopete (using the standard Jabber plugin)
  • My Kopete plugin is not longer needed and will be deprecated

To setup it with Kopete just add a Jabber account like this:

Kopete preferences for Facebook chat

This is a great move from Facebook. As Lars already mentioned, the Web 2.0 sites have brought lot of innovation and fresh wind to the Web. However, they have ignored interoperability a lot, and he is right, you need “connectors” to get your data.

The Web 2.0 has changed the way users store their data. Now it is everywhere. Without good interoperability we are only adding complexity to users.

If your website implements contacts, don’t forget to add a url with a http accessible vCard list. If your site implement events, provide an url with iCal entries. Google has done a good job with Google Calendar and Google Mail. Worth to mention the urls with calendar/contact entries can be “secret urls” which contain a long random string, but require no authentication, which makes it easy to add to your desktop mashup utilities, organizers, plasmods, etc.

Facebook still could do more. They invented a whole new email system. But they forgot to offer IMAP/SMTP interoperability with it. I am not sure either (feel free to correct me) whether I can generate a secret url with iCal entries of Facebook events as well.

Anyways, big thanks to whoever is responsible of getting this done. You did a big favor to the Internet itself.

As for Kopete. As protocols start to use XMPP, the need of hiding XMPP for the end user arises. The account wizard should display the services known by name, and do the XMPP setup with the known preferences. May be something I can work on now that I don’t need to maintain the protocol anymore. And I almost forget: we need a way to migrate current users of the plugin.

October 26, 2009

On upgrading openSUSE with 3rd party repositories

With the availability of up to date software for openSUSE, thanks to the amazing build service, our users have started managing the software in ways we did not imagined some years ago.

Update? Upgrade?

This has created some confusion between operations like update and upgrade, package vendors and the package selector functionality. This new situation plus the confusion leads to bug 548551

When you install a package, it has a vendor. Official packages have the “openSUSE” vendor. While packman packages vendor is something like “Packman Project”, and packages from the build service, have a vendor based on the project name, like “OBS KDE Project”.

Updating your system means looking for the latest package of the same vendor that can be installed without doing destructive operations to your system, like uninstalling another package due to a conflict. This is the reason for behaviors like:

  • “zypper up” does not pick latest KDE packages even if the repository is added
  • An official update (same vendor) for SoftwareFoo is available, but because your custom libSomething packages conflict with it, the update is not applied.
  • However, if you manually switch to the packman package, “zypper up” will upgrade from now on using new versions of the package in the packman repository, and therefore ignoring the openSUSE one.

This behavior is expected as we don’t consider a package with the same name to be the same package across vendors. A packman package may include mp3 support and the one in the openSUSE repo may not. If packman updates this package every monday, and openSUSE does every Friday, “zypper up” would end with your application having mp3 support depending on the day you are upgrading.

Bleeding edge repositories

If you are using a distribution like openSUSE Factory, packages are removed, split, renamed and aggressively replaced by new versions. This makes moving from a openSUSE Factory installation on a given day, to the state of the repository some time later a process that may require more aggressive operations, like downgrading packages or removing others, in order to reach the goal of moving to the second point.

As we explained above, the update operation won’t do this. The right way to do it is to perform a distribution upgrade. People using “zypper dup” are doing exactly this. This operation will move your system to the versions of the available packages. This may actually mean downgrading and removing installed packages. The recommended way to do this, is to perform it against a specific repository. This helps narrowing the “goal” further than telling the solver to do a “wide” distribution upgrade. This is what “zypper dup –from repo” should do.

If you intend to use “zypper dup” without an specific repository. You need to define repository priorities if you don’t want to encounter some surprises. For example you could set packman to a higher priority than openSUSE Factory so every Factory update does not remove your mp3 support.

What was missing?

Part of the confusion, and the difficulties to follow repositories with latest versions of big components (like KDE) on top of a stable release (like 11.1) comes from two sides:

  • “zypper dup –from” was broken, and therefore performing a “wide” distribution upgrade ( bug #549490), which is fixed now (1.2.7).
  • YaST package selector had no support for performing upgrade operations. This is available starting yast2-qt-pkg 2.18.18.

Upgrade operations are requests to the solver. When selecting packages by selecting the latest one manually (what “upgrade all in this list” did) you are not necessarily selecting the one the solver will select. Therefore such operations are best represented as “requests”, which can be combined with manual user selection/removal of other packages.

When browsing the packages in the repository view, you will see a notification that allows to insert a request to the solver to switch all installed packages to the versions of this repository.

upgrade-repo-1

Once you file the request, you will see the list of repository upgrade requests. You can cancel any of these.

upgrade-repo-2

And finally, you can do this in multiple repositories (equivalent to passing –from multiple times to zypper).

upgrade-repo-3

I hope this post made more clear how this operations work. The changes described above should be available in Factory soon. And may be we need to think what to do with legacy functionality like “upgrade all in this list”.

You can use the new functionality to stay current with your favorite desktop environment community repositories, or why not, to stay current with Factory.

October 16, 2009

kopete-facebook 0.1.4

I have released kopete-facebook 0.1.4 which fixes idle status of contacts and message sending.

I still have some important bugs, which I hope I can fix before the openSUSE 11.2 release.

From those known bugs, one is the inability to reconnect (go offline and offline), which is due to the “myself” contact using the email as userid, which is in turn due to a Kopete limitation of needing to know an id, but facebook sends it after login. I have tried so many hacks (like switching the myself contact at runtime), but all have other side effects.

Another one is a crash at exit, which I still can’t understand. Help appreciated.

Note: this plugin is not affiliated with Facebook in any way.

Should appear here soon (or build it yourself).

August 4, 2009

Kopete Facebook plugin 0.1.3 released

Available for openSUSE Factory in my home project. I also submitted to the KDE:KDE4:Factory:Desktop project where you may find it built for older releases using a newer KDE.

Changes:

  • Some connection fixes
  • The idle status icon is now shown
  • Basic user info widget (bug #198286)
  • Show contact status message! (bug #198284)
  • build system improvements
  • qjson 0.6.x is now required

Known bugs:

  • The plugin crashes when exiting Kopete. Some weird stuff with “myself” contact which seems to be deleted twice

Note: This plugin is not associated with Facebook in any way.

May 24, 2009

Facebook on Kopete, take II

Last week I blogged about Facebook support for Kopete, just after I was able to see my buddies for first time on the screen.

Since then I have made some improvements to message handling and other code cleanups. The code is now available in a git repository at github.

As KDE’s svn trunk is frozen, I will keep it there for now.

You can get packages for openSUSE Factory (version 0.1.2). I gave up trying to build it for openSUSE 11.1, as Kopete API has changed quite a bit. However the package may build on 11.1 plus the KDE 4.2+ repositories. You need libqjson from Flavio Castelli installed (or -devel package if you want to build it).

Roadmap for next 0.1.3:

  • Add caching to avoid downloading the pictures every 3 minutes.
  • More bugfixes

Roadmap for later:

  • Look into adding , searching, and other stuff.

Be aware. This is weeks-old-code. It has not been tested much and has lot of debug messages. Use it if you are a early adopter only.

May 20, 2009

Facebook support: First milestone reached

So, I have been working some weeks on this, and today I reached the first “usable” point. Screenshot:

facebook screenshot

As you may know, Facebook has a chat service. For me at least is slowly becoming the place where I have more people talking to me, and as you may also guess, the value of social systems is very tied to the number of users.

Sadly, Facebook guys where not smart enough as the Google guys and brought yet another damn protocol to this protocol overpopulated world. Then came the worst part. They announced something that was not there and promised Jabber support. One year later nothing has yet happened.

For a such popular service, one starts to think whether waiting another year is worth for a protocol that is so popular. As I wanted it myself now, at some point I decided I was willing to implement it even if a Jabber version was available later.

We already have the problem that users expect to see Google talk in the Kopete list, because developers don’t figure out that grandma does not know what Jabber/XMPP is. So a good improvement would be adding the concept of “services” where we could add a protocol by just saying “it is just jabber, but with this server settings, this logo and this name”. That path would allow for a easy move to other XMPP protocols later.

But for Facebook, no more wait. Yesterday I was able to use it for first time to chat, so I am blogging about it.

Next steps:

  • Add more error handling
  • Fix a bug in the contacts status when they go offline
  • Put it into kopete or playground svn
  • Make an openSUSE package ;-)
  • Cleanup. I started over the testbed plugin and it added some stuff that probably I don’t need
  • Proxy support. I coded the engine using QNetworkAccessManager so it is KDE independent. Only the Kopete plugin is KDE based, so I haven’t looked into proxy support and other stuff

Other stuff with less priority:

  • Adding contacts from the client
  • Configuration (there is no much to configure)
December 23, 2008

The module’s info experiment

This experiment is an idea from Stano. When he saw us hacking and trying new approaches for the control center code base (in order to allow a different design) he sent me this ycp snippet.

{
  import "Mode";
  Mode::SetMode( "autoinst_config");

  string m = (string) WFM::Args(0);
  string p = (string) WFM::Args(1);

  y2milestone( "Writing the result of '%1' to '%2'", m, p);
  any a = WFM::call( m, ["Read"] );

  any res = WFM::call( m, ["Summary"] );
  y2milestone( "%1", res);

  SCR::Write( .target.string, p, res);
}

This snippet would called with two parameters, A and B, would run a ycp client in “auto” mode and write a summary of the configuration in file B.

This would allow in the control panel to display “status” information about the modules before launching them.

To test the concept, I hacked a quick YModuleInfoProvider class that hooked into the Qt::Tooltip role of the YQModulesModel. I filled the implementation of this class with a hacky call to yast2, calling the ycp snippet and reading back the temporal file.

Ideally, the status should be shown in the icon itself or in a “Dolphin” like status bar. However, I just wanted to see it, so I put them in tooltips for now (Not that I want to keep them there).

And yes!, it works, you can see the status of some modules just h-overing them with your mouse:

module status 1

module status 1

module status 1

I would like to replace the implementation of YModuleInfoProvider with code using the YaST2 component API instead of an external call, just to see if it is cleaner, faster and whether we have more control. Stano helped me making this snippet work:

using namespace std;

int main()
{
    Y2Component *c = Y2ComponentBroker::createServer("wfm");

    if ( ! c )
    { cout << "error talking to wfm" << endl; return 1; }
    
    Y2Component *mode = Y2ComponentBroker::getNamespaceComponent("Mode");

    if ( ! mode )
    { cout << "error finding component" << endl; return 1; }

    Y2Namespace *ns = mode->import("Mode");

    if ( ! ns )
    { cout << "error importing Mode namespace" << endl; return 1; }

    Y2Function* fnc = ns->createFunctionCall("SetMode", Type::fromSignature("void(string)") );

    if ( ! fnc )
    { cout << "error calling SetMode" << endl; return 1; }

    fnc->appendParameter( YCPString( "autoinst_config" ) );    
    fnc->finishParameters();    
    fnc->evaluateCall();
    
    cout << "Done call!" << endl;
    fnc = ns->createFunctionCall("mode", Type::fromSignature("string(void)") );
    if ( ! fnc )
    { cout << "error calling mode" << endl; return 1; }

    YCPValue res = fnc->evaluateCall();    
    cout << res->toString() << endl;
    
    return 0;
}

However, for this to work, it requires a small bug to be fixed in core. So I guess I will replace the implementations once this patch is in.

December 17, 2008

Control Center in System Settings?

Comments

After posting A refactoring journey: the control center yesterday, I got some comments quickly.

Of course I got some comments like “The category icons are too small” “Spacing is wrong”. When it was clearly stated at the end of the post:

Note that none of these screenshots mean “this is how the control center will look”. The focus is still on the infrastructure that enables us to try more radical approaches. For example, the old control center look and feel is trivial to emulate.

However, there was another series of comments:

Why don’t you add the yast options into the KDE settings?

I think the answer to this question is the same as the inverse. Why should we?

ls /usr/share/applications/YaST2/  | wc -l
85

There are 85 modules in my system. Well, not all visible. I don’t have all available installed either. But, would you like to have suddenly 85 icons in this view?

KDE4 System Settings

What do others do?

Now, Marçal Juan comment:

Just think in Apple and Microsoft approach… it’s all in the “System settings” not two independent apps.

This is simply not the case. I wonder why Marçal thinks they are mixed.

I think there is a simple reason why they are separate. If you had to put so many items, the control center can’t be read by our brain in one shot anymore . “but lets create sub-items”"… yeah, and then you are again back in the kcontrol tree times, where finding something was hard if you don’t know at which deep-level it was. Yeah, our readers/commenters solve any usability problem by making things a tree or adding tabs :-) .

Actually, Apple has a System Preferences panel:

leopard preferences

For server related settings, they have a separate console (Still note how User accounts is present on both):

leopard server preferences

Not only Apple. Marçal is also wrong with Microsoft, which has a control panel:

windows control panel

And also a server console (this time with a completely different look:

windows server panel

Why?

I think the temptation to merge them comes from two sources:

  • The fact that most Linux user tend to mix the server/advanced administration and basic tasks a lot, and therefore they think their usecase is the common one.

  • This makes them think that everyones wants to go trough the basic settings when configuring some advanced service.

  • Actually the last YaST survey revealed that the only tasks an user does often is software management and in second place, network management. All the rest is done seldom.

  • The fact that Linux lacks some basic administration tools upstream or integrated in the desktop, which YaST provides right now.

I think the last two ponts are relevant. The real problem is not whether you have all icons in one control panel, but where are those icons. Is software management a basic task or it goes together with virtualization settings? That is the problem. Windows and Apple provide software management and network management in the basic settings. However, I think this is only a problem for those specific YaST modules that are used to often, and both cases have a solution.

In the case of Software Management, PackageKit already offers basic software management integrated with the desktop. I do have an icon “Add/remove software” in my KDE System Settings (KPackageKit provides it). The case of networking is also solved since long time, and most desktop users configure their network in the system tray with knetworkmanager.

Any other configuration that becomes common for a desktop user has to be integrated with the desktop in the basic settings, or even further, in the right context (for example, enabling sharing a folder by right clicking the folder instead of going to the system settings ).

Right now we have some technical limitations. We can’t just put the YaST modules in system settings without doing some changes in that code, and we can’t offer all functionality so it can be integrated in the right context. But we are moving into that direction, and that really does not means we have to go and put 80 new icons and ruin the KDE4 control center concept or reintroduce hard to remember categories.

We will do some experiments about it anyway.

Update 1: As Stano points out, it may make sense to offer access to YaST in the “Computer Administration” category of the System Settings, not module per module, but as one icon. (then launch YaST control center, or embed it). What we can’t do is to embed the modules themselves in the system settings (we did this in the past and was problematic as modules have wizard semantics).

December 16, 2008

A refactoring journey: the control center

It is almost Christmas and things are getting more quiet and there is now some time to look at pending TODOs. I mean all those things that did not make into 11.1 because priorities.

The problem

The Qt control center is an ancient piece of code, and it still is a Qt 3.x application.

You may know how it looks:

original

We already have a feature for a reorganization of the control center. Thomas Göttlicher, Jens Daniel Schmidt, Jörg Kress and Martin Schmidkunz worked last year on proposals. I want to leave look and feel out of this post for now. (When it comes to which color to use to paint the walls, everyone has an opinion). I wanted to focus on:

The main reason of why the code became so inflexible is the mixing of presentation with data items everywhere. Yes, not only the control center suffers from this problem but also the Qt package selector.

Qt 4.x has, among a lot of unmatched features, the ability to use a Model/View paradigm. This means, you describe a model, and then you just plug it into a view, and it will be displayed according to the widget’s context, instead of sub classing widgets just to hold data on the user interface.

I have to admit that it is not the first time I look into Qt’s Model/View, but in order to understand it, one needs to dig in some real life problem because it has some concepts like indexes, views, delegates, etc, which start to make sense once one adapts it to the specific problem.

So, my challenge was to add a model architecture to the control center in order to make then trivial to let other people choose the wall color ;-) and also enable other innovations. For example Stano hacked a ycp script to retrieve summary information from the configuration (I can’t wait to integrate that).

Step 0. Find out where are we.

Thomas Göttlicher already did some research on using the KDE4 system-settings style views. This shown that some widgets could be reused (like KCategorizedView), but also showed that pieces like the SystemSettingsModel are too complex for what we need.

Step 1. Compile old control center on Qt 4.x

I took the old control center, branched it and started the boring work to getting it to compile with Qt 4.x. This involved setting cmake up as I did not wanted to waste time figuring how to convert an autotools project to Qt 4.x.

In this stage one sees much design decisions that either help or hurt when porting. Also opportunities, for example the control center had its own .ini file parser, which I replaced with the QSettings class. Note, QSettings is there since Qt 3.3.

At the same time I got it compiling in Qt 4.x, I extracted some of the module listing code to an QAbstractItemModel and created a small test program for it that I could run in parallel.

My model could be plugged into a view and display the groups, with one line of code:

groups model

After getting familiar with the code, and learning it, I decided it was almost impossible to plug my model in the current code. Why did I tried then? Because reading code is always harder than writing it, therefore there is always a tendency to rewrite from scratch, and I wanted to be really sure it was not the “reading code effect”.

Step 2: start with a cleaner base

I took Thomas codebase and compiled it. I realized it was easier. I only needed to get rid of the systemsettings code. I only needed the KCategorizedView part, or even a plain QListView if I had the right model.

Once the code was removed, I needed a modules model (analog to the groups model). After writting some code, I felt like I was writing the same thing again: a model that lists what it is in .desktop files. Closed both models and focused on a DesktopFilesModel as a base. Once this was done, the groups and modules model was a few lines of code (mainly setting the right configure time YaST paths). So I had a modules model too:

groups model

Step 3: KCategorizedView

This component worked out of the box when I plugged my model:

one category

Except that it only displayed a hardcoded category. One needs to enhance the model to reply to the CategoryDisplayRole, which required to start thinking better about the structure of the models and add utility methods to query groups for modules given an index and similar methods.

At the same time once you see it on the screen you get a feeling when the view is querying the model (speed and debug lines), which resulted in a basic cache infrastructure for the base .desktop files model.

The result:

categories

Step 4: Profit from the new structure

At this point I realized there were too many items on the screen, but also I realized how trivial was to add a docking at the right with the group list, or how trivial it would be to set the proxy model to filter certain groups. Just for playing, I added a search bar:

search

and then a groups list dock (which you can move around the main window):

group list.

Now what?

Note that none of these screenshots mean “this is how the control center will look”. The focus is still on the infrastructure that enables us to try more radical approaches. For example, the old control center look and feel is trivial to emulate.

Will it be the old? the new? configurable? We don’t know yet. There is still work to do before we focus on the look and feel.

September 29, 2008

The natural order of things…

Polyglot programmers

In the book “The Productive Programmer”, written by Neal Ford, there is a nice chapter that talks about an old topic: languages, and defines Polyglot programming as the kind of tendency in the industry to manage the complexity of current applications.

It is tempting to think that one language keeps the complexity low, but that is exactly the reason why languages keep coming. They usually try to become general purpose languages and start to get bloated or inheriting nonsense stuff.

While thinking on this topic. I started to remember lot of episodes in my life as a software developer. Having learned lot of my base in the KDE community, I was quite natural to think in the way Neal explains. In KDE people write since the beginning of the platform a good bunch of the static user interface descriptions in domain specific languages, which are transformed to more general languages (C++) at compilation time.

The very basics of Qt is C++ plus a “platform” and some specific add ons (another domain specific language for component communication) which are compiled into support code at compilation time. Code generation using DSL’s goes beyond that: Configuration for applications can be specified in a simple XML schema for the configuration domain and most of the code is generated. Therefore consistent, robust and bug free. For a KDE guy, this sounds natural (however it could be extended much more), and all the platform is like the book “Ola’s Pyramid”, where the lower levels are done using general languages (like C) and upper layers are done using more domain specific languages. We miss the intermediate part of scripting languages, but I am sure Plasma will be the first component following that natural order of things.

At that time I was learning, and the KDE “way” became natural. At the same time, that was the time when I discarded the Gnome platform completely. Mostly because I heard lot of FUD against KDE decisions at that time, lot of attacks and ignores towards a more common world, and then the world kinda inverted itself and used to be criticized stuff like use of preprocessor to add features to the language (Qt’s moc) : Welcome Vala? or the ignored dcop bus : Enter dbus, Object orientation (try to enter GObject ugh)… are now part of the “natural order of things”. I was kind of confused ( famous hackers claiming they won’t program apps in C again, uh? ).

As a side note, some developers, like Cornelius Schumacher and André Duffeck played with this concept of generating code much further. Like mixing models and user interfaces.

polyglot YaST

As a YaST developer, I keep thinking in the current code base, the directions we are taking, etc. The development field is full of new silver bullets, languages and things that make hard to keep the big picture if you are following previously unknown fields.

Surprisingly YaST is a very good example in the whole model. We abstract the low level system with a C++ made platform, and run a domain specific language called YCP on top, and have another domain specific language to manage the system configuration. It is a very good example of a complex platform that manages complexity using different languages, and it is a very mature platform at the same time. Of course there are lot of things that can be improved:

  • No usage of convention over configuration: There is an abuse of “skeletons”, that is, generating trees and code, putting it into version control, instead of doing most things magically by convention. (Very similar to the difference of rails to Java use of XML).

  • Defining user interfaces using code is ok as long as the code is not where it should not be. And the code base has this problem. Having an external DSL for this helps forcing you to separate things.

  • Hard to bind to the platform: Writing components is boring and over engineered. It works for complex cases (like defining your own type of component functions), but fails for the simple case. Try doing a ruby extension. It is easy. I think this is something we should improve. There are some basic support like adding some type information (using a DSL) on the headers and parsing that.

  • We should check and recheck that we are not repeating ourselves. Duplicated functionality, declarations and implementations.

Code generation for services

Now that we are walking towards the world of services, Web and such, it is time for new ideas. It is interesting to see big players trying to solve difficult problems, and coming up with natural solutions, like the case of Google with their protocol buffers, and Facebook with Thrift (paper).

Thrift is more interesting, as it comes from the situation of having to develop different services that need to communicate without needing to care much about what was used to build that component (SOA approach, which is not very different to what we are solving with dcop-ng (aka dbus) in the desktop scenario), but imagine dbus is your network, and thrift is the Qt dbus binding generator or dbusxml2something.

Thrift is a software framework for scalable cross-language services development. It combines a powerful software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, and Ruby. Thrift was developed at Facebook and released as open source.

Thrift allows you to define data types and service interfaces in a simple definition file. Taking that file as input, the compiler generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages.

Now that schubi has been doing quite progress on exposing some YaST functionality via http (result of the last workshop), I am really curious if we can define our platform, and tackle some problems by defining very domain specific tools that can save us from repetition. Will see.

« Older Entries