UDS N Monday plenary: Getting great applications on Ubuntu

Ubuntu Software Center and the future of the universe

[Matthew Paul Thomas:] Good afternoon.

Six months ago, Mark [Shuttleworth] started us talking about “crossing the chasm”.

How Ubuntu can get beyond technology enthusiasts, and their friends and families,

to other early adopters.

How we can get from somewhere in the region of 12 million users,

to hundreds of millions of users.

How Ubuntu can do for operating systems what Firefox has done for Web browsers.

So, how do we cross this chasm?

Well, you may have your own theory, but this is mine.

Ubuntu needs to be useful, desirable, gettable, and keepable.

Useful, in that does things that people need or want to do.

Desirable: people prefer doing those things on Ubuntu, rather than any other system.

Gettable: people know how to get Ubuntu and install it,

or more likely, they buy it preinstalled.

And keepable: people can keep using Ubuntu for months and years on end without trouble.

Now, there’s a whole universe of issues and problems that we need to tackle

to address these four basic requirements.

And if you are working on any of these things, what you are doing is important.

So if anyone tells you, “All Ubuntu needs now is games”,

or “all Ubuntu needs now is advertising”,

they are looking at just a tiny part of this universe.

And you’ll hear a lot this week about Unity.

But the truth is that Unity is only a tiny part of this universe as well.

Because Unity will be most successful if it is so simple and unobtrusive

that people barely even notice that they’re using it.

Nobody is going to turn on a computer so that they can use Unity.

People turn on a computer so that they can do things.

So with that in mind, today I’m just going to look at the first quarter of this chart.

How do we make Ubuntu as useful as possible?

To some extent, this comes from properties of the base platform.

Hardware support, networking, printing, battery life, startup speed, accessibility, things like this.

A few of these things, we are testing and tracking in an automated way.

But most of them, we aren’t yet.

Most of them, whenever they go wrong, or whenever they get worse,

the only way we find out is by accident. By ad-hoc testing.

So, as Mark said this morning, we should do more automated testing.

I look forward to finding out how people here [UDS session: “Improving the desktop testing program”]

plan to automate more of this testing.

The rest of usefulness comes from applications.

Applications are what let people do things with Ubuntu that we could barely imagine.

So how are we doing when it comes to applications?

Well, six years ago, the first version of Ubuntu,

Ubuntu 4.10 had 375 graphical applications in the standard software channels.

And over the past six years, this number has steadily increased,

pretty linearly, and in Ubuntu 10.10 we have 2351 [applications]. Approximately.

[applause]

But unlike the real universe, the Ubuntu universe does not exist in a vacuum.

Here’s Android over the same time period.

Two years and one week ago, the Android Market started out from zero.

And a couple of hours ago, they announced that they have just reached 100,000 applications.

Scale down the graph again, because here’s iOS.

July 2008, the App Store opened from zero, and today it has over 300,000 applications.

This is a big difference.

Now there are several explanations you might make for this difference.

You might say for example, that, well, a lot of those Android and iOS applications are crap.

But that doesn’t seem to be true, at least, according to Android users themselves,

only 28 percent of the ratings they give to applications

are saying that the application is crap [1 or 2 stars out of 5].

You could say, well, applications don’t really matter any more.

Everything’s moving to the Web.

But I don’t think that’s really true either, because that’s what Apple thought

when they first released the iPhone three years ago.

And they pretty realized that it was a mistake, because developers didn’t like that.

And that’s what Google thought, when they first announced Chrome OS,

but it seems to me that Chrome OS has been pretty much sidelined by Android.

You might also say the opposite:

a lot of those mobile applications are applications specifically to replace particular Web sites

because the Web sites are too difficult to use on a mobile browser.

But that argument doesn’t apply to the iPad, and even that has shot past Ubuntu

and now has 35,000 applications after only a few months.

More than ten times the number Ubuntu has.

Next month [November 2010], Apple is going to open the Mac App Store to application developers.

And in the next version of Windows, we can reasonably expect a Windows Store as well.

We have taken six years to get just over [2300] applications.

How long do you think it will take for them to get ten times as many?

A month? Two months, maybe?

There’s something is terribly wrong here.

Now, this does not mean we should give up and go home.

It does not mean we should panic.

But nor does it mean we should just ignore this problem,

or say, ah, well, we’ll think about that in 2012, or 2013, or 2014.

No. We need to figure out what’s going wrong now. And fix it, now.

So. What does it take to attract tens of thousands,

hundreds of thousands of great applications to Ubuntu?

Five things.

First, we need an initial user base. Of course this is a bit of a chicken-and-egg problem,

but the more users Ubuntu has, the more those users can

give application developers their appreciation, or their money, or both.

And with Ubuntu, we have somewhere in the region of 12 million users who can do this.

So far so good.

The second thing we need is a great developer platform. [IDE, SDK, reference materials, frameworks+APIs]

It needs to be not just easy, but even fun to write an Ubuntu application.

So, how are we doing here?

Well, let’s look at the Web sites for example.

This is the Web site for if you want to write Windows applications. [MSDN]

Plenty of information here.

This is the Web site for if you want to write Mac applications. [Mac Dev Center]

Here’s the one for iOS. [iOS Dev Center] Here’s the one for Android. [Android Developers]

Here’s the one for Palm Web OS. [HP Palm Developer Center]

The Web site for if you want to write Ubuntu applications looks like this.

[“Ubuntu Developer” — “Just another WordPress site” — “Hello world!”] [laughter]

But even if this site was up and running, what would it say?

What would we say, for example, to someone who wanted to add audio to their application?

[“The Linux audio mess” diagram]

What would we say to someone who wanted to add spell-checking to their application?

[“aspell? hunspell? enchant?”]

What would we say to someone who wanted to draw a chart inside their application?

[“GDK? Cairo? XRender?”]

What does it say that the Unity developers have developed an entire widget toolkit of their own

[“The Clutter Toolkit”],

plus a replacement for one of the standard Gnome libraries [“libwnck” → “BAMF Application Matching Framework”],

just so that they could implement things that should have been easy already?

We’re in bad shape here.

Ubuntu became popular with millions of end users partly because

we made smart, opinionated decisions about which applications to ship by default,

and we gave special care and attention to those applications.

If we’re going to be successful with application developers,

we need to put just as much effort into frameworks and APIs.

How might we do that? I don’t know, but I want to find out.

The third thing we need is easy software packaging.

In Ubuntu we have a team of 161 “Universe Contributors”,

who spend their spare time packaging software. Mostly other people’s software.

Now, why do we do that?

Well, it’s because we’re a “Linux distribution”,

and that’s what “Linux distributions” do, right?

Yes, but why do they do that? Two main reasons.

Firstly, because until now, every Linux distribution has been so unpopular,

that hardly any application developers actually cared whether their software was

easily installable on that distribution. So the distribution developers had to take up the slack.

Fortunately Ubuntu is now popular enough that this is starting to change.

is a lot more difficult than packaging software for Windows [or other systems].

So for everyone here who has ever packaged software,

the first thing I’d like you to think about is, how can we make this easier?

Maybe we need to make .deb packaging easier.

Maybe we need to have a separate package format for third-party applications.

Ubuntu Universe Contributors will play a vital role in Ubuntu for years to come.

But if we want to get even 50 times as many applications as we have now,

even 50 times as many open-source applications,

can anyone seriously believe that we’ll find 50 times as many Universe Contributors?

Not 160 people, but 8000 people who want to spend their spare time packaging other people’s software?

So there are several sessions at UDS about this.

[“Application packaging and distribution”, “Improving the Packaging Guide”, “Sandboxing applications”]

The fourth thing we need is easy publishing of software.

The standard process if you want to get a new application into Ubuntu takes between two and eight months,

depending on what part of the release cycle we’re at.

If it’s the day before Feature Freeze, it’s two months.

If it’s the day after Feature Freeze, it’s eight months.

Now, in contrast, iOS developers complain

if their application doesn’t get into the App Store within a week.

So we’ve developed two processes to get around this.

One is the Backports process. The other is the Application Review process.

Neither of them are being used particularly well at the moment.

So there [is a session] on this this afternoon. [“App Review Board Review”]

The fifth thing we need is easy installation, feedback, and revenue.

Here, finally, we get to Ubuntu Software Center.

All of this to point out that if you think Ubuntu Software Center is the linchpin of Ubuntu,

or the future of Ubuntu, you too are looking at just a tiny part of the universe.

One month ago we released Ubuntu Software Center version 3.0.

This is a big step forward in several ways.

It’s faster, it’s prettier, the search works better,

it lets you install add-ons for applications, it has a history of past transactions,

and, most importantly perhaps, it lets you buy software.

Right now we have, I think, two applications in the store, another two on the way,

we’re looking to accelerate this very quickly.

[UDS session: “Defining the end-to-end process for commercial applications”]

So we owe a big thankyou to our amazing team of contributors.

[Michael Vogt, Gary Lasker, Canonical ISD team, Andrew Higginson, Didier Roche, K. Vishnoo Charan Reddy…]

[…Matthew McGowan, Mohamed Amine IL Idrissi, Peter Gardenier, Rakesh Ambati, Sebastian Heinlein]

If any of you would like to help out with Ubuntu Software Center,

please come and talk to one of these people.

We’re very friendly, and there’s lots of things we want to do.

Our short-term goal, for Ubuntu Software Center 3.1, is ratings and reviews.

[UDS session: “Ratings and reviews in USC”]

But beyond that, there are many other things we’d like to do for 4.0.

But in the meantime, your homework for today is to think:

[1] How can we improve our developer platform,

[2] How can we make packaging easier, and

[3] How can we make publishing easier.

Thank you.

[Thanks to: Charline Poirier, Evan Dandrea, Florian Boucault, Jason Smith…]

[…Jonathan Lange, Michael Vogt and Canonical IS, Randall Ross, and Scott James Remnant]

[applause]

[Jono Bacon:] All right. I think … Aren’t you both doing this?

Oh, is it just you? Oh? Okay. So, finally — great job from MPT —

Finally we’ve got Evan, and he’s going to get up and talk about

“The restaurant at the end of the universe”. Big round of applause for Evan.

[applause]

[“The restaurant at the end of the universe” — “Evan Dandrea ”]

[Cartoon: Insects swarming out of computer screen.]

[Caption: “My package made it into Debian-Main because it looked innocuous enough;…”]

[“…no one noticed ‘locusts’ in the dependency list.”]

[Evan Dandrea:] Right, so, following Matthew … Possibly not my best decision ever.

That was quite the presentation. I think that deserves another round of applause for Matthew.

[applause]

So I just want to spend a brief period of time today talking about where we’re at right now,

where we should be moving towards, and how we can reduce the barriers

and roadblocks on our path to getting there.

So, here’s where we’re at. This should be quite familiar to all of you. Right?

We have the upstream developers create applications, they release them out onto the Internet.

Ubuntu then comes, takes the applications that they find interesting.

Individual Ubuntu developers, individual Debian developers,

take applications, they package them up, they fix all the bugs that may be there,

and they deliver them to users.

As Matthew said, this takes anywhere from two to eight months,

depending upon where we are in the release cycle.

Users, after those two to eight months, get the applications,

they play around with them, they find some bugs, they generate some feedback,

and they deliver that back up through Ubuntu and back up to the upstream developers.

Again, this takes two to eight months, plus some time

for the users to send that information to us and for us to send that information to developers.

This is an incredibly lossy process.

There are a wealth of applications out there that we don’t even know about

because it hasn’t piqued anyone’s interest in the Ubuntu developer community

to actually go ahead and package them.

And the individuals writing those applications really have little recourse

to getting those applications into Ubuntu.

Equally, the bugs that are filed by our users, while there is an immense effort to push these upstream,

not all of them ultimately make it up there.

I think no-one here has any illusions that all of the bugs that are reported in Ubuntu

make their way back up to the upstream applications.

So, we have a problem. And I’ve really tried to rack my brain as to what we were missing here.

And I tried to think, given the apps that we have, what are we missing?

What are the killer applications, that we use every single day,

that we’re happy to talk to our friends about?

That we actually go and tell people, “Oh my god, you’ve got to try Ubuntu because of this”.

And I really couldn’t think of any. Sure, there’s Firefox, there’s the mail client,

and there’s a word processor, but really, every platform has that.

What we don’t have is, effectively, an Angry-Birds-like application for Ubuntu.

Now, whatever you think of this particular game, it’s been immensely successful.

For those of you that don’t know about it, it’s generated

millions upon millions of dollars in revenue, and users, mostly through word of mouth.

The chief developer of this application was just snatched up by EA.

And it’s a really simple game, and it didn’t take very long to develop on the iOS platform,

and I’m sure it didn’t take long either to develop on Android.

So, taking a quick step back: Why is this?

Well, we’re spending an awful lot of time doing developers’ jobs for them.

As I said at the beginning, we’re taking applications,

we’re smoothing out the rough edges, we’re patching these applications,

we’re packaging them up, and then we’re delivering them to users.

But this is actually handicapping developers. This is removing that direct communication

between developers and users. Why are we doing this?

It’s obviously costing us a lot of resources, so surely there must be a reason.

Well, it turns out the reason is that these are all very dangerous things,

and they require years of experience. dpkg runs as root.

Equally, maintainer scripts in Debian packages have the full range of options

that the shell provides them. So they can do anything on the system.

Because they have maintainer scripts, that means they’re non-atomic,

which means that anything they do can’t be undone.

So of course we’re scared, of course we’ll step in and say,

“We’ll do the packaging for you”, because we do not want to introduce our users

into that world of hurt of really bad packaging.

But maybe there’s another way to solve this. One that’s actually scalable.

Because if you look at this, as Matthew said,

if we hope to get to hundreds of millions of users, hundreds of thousands of applications,

we’re not going to be able to do that with the [Masters of the Universe] alone.

Well, what if we introduced static analysis?