Debian releases - stable, testing, unstable

When talking about using Debian, one of the first objections people will raise is the fact that it only has "old packages", it is not updated often enough.

This is generally not true. Or well, "true" if you stick to the "stable" release of Debian, which might not be the right version for you.

Also, people don't often realize that it's easy to use more than one "release" on a given system. For example, you can configure apt-get to install "stable" packages by defautl, but allow you to do a manual override to install from "testing" or "unstable", or vice-versa.

Before starting, it's worth noting that this might not be for the faint of hearts: mixing and matching debian releases is generally risky and discouraged. Why is this a problem? Well, it all has to do with dependencies, backward compatibility and the fact that they may not always be correctly tracked.

For example: let's say you install a cool new gnome application from unstable. This cool and new application depends on the latest icons, which apt also correctly installs. Now, in the new icons package, some old icons have been removed. Old applications using them will either need to be upgraded, or their icons break. This is generally handled correctly by apt-get dependencies, assuming the maintainer did a really good job tracking versions. But this is hard to do, and error prone at times. Worst can happen with C libraries or different GCC versions using different ABIs, or systemic changes like the introduction of systemd or similar.

Debian (and technology) has gotten much better over the years, and I must say this will work most of the times. Still, be careful: whenever upgrading, check what is being installed, and try to make a call whatever the risk is worth it.

But let's start from the beginning. In order to make a good choice, you need to understand how this all works.

Stable, Testing, Unstable and Experimental

So, let's talk about the lifecycle of a debian package. Let's say a new version of vim or git is released upstream. What happens next?

  1. A Debian developer will likely notice. Either because he's monitoring their development, their web site, has setup automated scripts, or a user has opened a bug like 'please update vim!'.

  2. The Debian developer will download the new version, build it, make sure his build scripts still work, and if they do, build a .deb package.

  3. The Debian developer will upload the package to unstable. In unstable, it will immediately become available to anyone using that release.

  4. After a several days in unstable, and once a set of criteria has been met (like no new bugs filed of a certain priority, ...), a system move it automatically to testing.

  5. Every once in a while, a group of Debian developers will decide to release a new version of Debian, a new stable. They will thus freeze the current testing archive, fix as many critical bugs as possible, and do a new release.

  6. After a package makes it into stable, only security fixes and certain kind of updates can be made.

What does this all mean?

  1. stable changes rarely, as a result of a manual process. Software is old.

  2. testing changes continuously, as a result of an automated process. Software is relatively recent, although it has baked in unstable for a few days, and proved to be stable enough for the automated systems to move it to testing.

  3. unstable has all the bleeding edge stuff, freshly uploaded by their maintainers. Software here is generally very new, the only exception being software that is not well maintained, where the maintainer is busy doing something else. The drawback of unstable is well, that sometimes there is broken software, or packages that don't interact well with each other.

There's also a fourth release: experimental. This is used by Debian developers to push packages that are really not that ready to be used, experiments for brave people to try.

Choosing one, or mixing them all

One little known feature of apt-get, aptitude and all related tools is that it is relatively easy to mix and match packages from any release.

First, you have to configure your /etc/apt/sources.list correctly. Mine looks like this:

deb ftp://my.debian.mirror.org/debian/packages stable         main contrib non-free
deb ftp://my.debian.mirror.org/debian/security stable/updates main contrib non-free
deb ftp://my.debian.mirror.org/debian/packages testing         main contrib non-free
deb ftp://my.debian.mirror.org/debian/security testing/updates main contrib non-free
deb ftp://my.debian.mirror.org/debian/packages unstable         main contrib non-free

deb ftp://ftp.at.debian.org/debian/ testing main contrib non-free
deb ftp://ftp.at.debian.org/debian/ stable main contrib non-free
deb ftp://ftp.at.debian.org/debian/ unstable main contrib non-free
deb ftp://ftp.at.debian.org/debian/ experimental main contrib non-free

deb-src ftp://ftp.at.debian.org/debian/ testing main contrib non-free
deb-src ftp://ftp.at.debian.org/debian/ stable main contrib non-free
deb-src ftp://ftp.at.debian.org/debian/ unstable main contrib non-free
deb-src ftp://ftp.at.debian.org/debian/ experimental main contrib non-free

deb http://security.debian.org/ testing/updates main
deb http://security.debian.org/ stable/updates main

It might look confusing at first, but really, it's not that hard: it is telling apt-get to download all indexes for all versions of the debian indexes - stable, testing, unstable, and even experimental.

If I was to run apt-get update and apt-get dist-upgrade after just setting this file up, apt-get would update my whole system to the unstable (or experimental) version of all packages. Which is not what I want.

I want to a) be able to pick per package, and b) have a reasonable default.

So the next thing I do, is create an /etc/apt/preferences file. In this file, I generally write something like:

Package: *
Pin: release a=stable
Pin-Priority: 800

Package: *
Pin: release a=testing
Pin-Priority: 950

Package: *
Pin: release a=unstable
Pin-Priority: 700

Package: *
Pin: release a=experimental
Pin-Priority: 500

This pretty much tells apt: "please, my kind apt friend, prefer packages in testing (highest priority). If you cannot find packages there, or cannot resolve dependencies there, fallback to stable first, unstable second, and experimental last". Of course you can change the priorities to pick whichever order you prefer. Just don't go above 1000, for a reason I will explain later.

Now, let's say I run:

apt-get install vim

apt-get will look for the latest vim in testing, and install it. If it cannot find it there - or any of the dependencies are not available there - it will look for packages in stabe, unstable and experimental.

But there's now one more trick I can use: I can tell apt-get manually what to do!

For example, if I want to install the latest and coolest vim, that hasn't made it to testing yet, I can run:

# apt-get install -t unstable vim

or:

# apt-get install vim/unstable

The former will install vim, and all the other packages apt-get decides to install or upgrade, from unstable. The latter will install only vim from unstable, and get all other packages based on the preferences above.

Now, what happens when you update your system? Well, apt-get will not downgrade an installed version of a package unless its priority is above 1000, or you manually specified its version.

One more thing to note is that if you try to install something that only exists in unstable or experimental, apt-get will figure it out and try to do the right thing (well, most of the times).

Marking and holding, with apt-mark

There is one more trick I use. With apt-mark, you can tell all apt tools to not touch a package.

Let's say vim is critical to your business, and you want to install and hold onto the current stable version of it.

All you have to do is:

# apt-get install -t stable vim
# apt-mark hold vim

The second command will instruct apt-get to never automatically upgraded, removed, or installed.

You can see all the 'holds' in place with:

# apt-mark showhold

and remove it with:

# apt-mark unhold vim

Similarly, you can mark a package as manually or automatically installed with:

# apt-mark auto screen
# apt-mark manual screen

The difference is subtle, but important. A manually installed package is a package you, as a user, care about. So apt will keep it up to date, and never remove it.

An automatically installed package, instead, is one that was installed because of a dependency. If that dependency is removed, or no longer necessary itself, the package might be removed during an upgrade.

When I play with new software, or try new packages, I generally mark them as installed automatically until I decide I want to keep using them. That way, they are likely to be removed as I update the system

Conclusions

I have been using Debian since the late 90s. One thing I love about it is apt, and the ability to have a continous update cycle. Rather than reinstalling the system every few years, or doing a giant upgrade every so often, I much prefer to stick to testing, and frequently run apt-get dist-upgrade to update the few packages that have changed since I last run the command.

Small, frequent, updates that give me fairly up to date software, rather than large, rare, updates that give me out of date software. This has worked well for me for production servers as well, with very minor glitches every now and then.


Other posts

  • A simple way to generate snippets in python The Problem Let's say you want to add a search box to your web site to find words within your published content. Or let's say you want to display a l...
  • I/O performance in Python The Problem I am writing a small python script to keep track of various events and messages. It uses a flat file as an index, each record being of th...
  • Cleaning up a CSS Let's say you have a CSS with a few thousand selectors and many many rules. Let's say you want to eliminate the unused rules, how do you do that? I s...
Technology/Python