The Ubuntu Project: Overview and Development Model ------------------------------------------------------------------ :Author: Benjamin Mako Hill :Contact: mako@ubuntu.com :Date: Wed Oct 19 16:25:12 EDT 2005 :Copyright: `Creative Commons ShareAlike License `_ Introduction ------------- .. Note:: SLIDE 1 - Title Slide With Title Information Overview of the two talks: - A General Overview - A Discussion of the Development Methodology - Lessons that you all (as free software developers primarily) can pull from our work. .. Note:: SLIDE 2 - Ubuntu Overview Overview Part I -- Overview --------------------- * Introduce Debian and the things it does well and has not done as well in the past; * Tell you a bit about where Ubuntu's comes from; * Give you a little history on the people behind Ubuntu and where it comes from; Then I will: * Introduce the Ubuntu project and give a quick overview; - Releases - Usability - Desktop integration - Philosophy - Other areas we focus Debian ======= So there is this project called Debian which a distribution of software. Question: Who knows Debian? Who uses Debian? .. Note:: SLIDE 3 - Debian overview (can go through quickly) I've heard Debian described as a big place with lots of packages. It is a collection of all sorts of software. * Debian contains over 15,000 packages (and growing by 5 each day); * Debian involves the work of 1,000 *official* volunteers and many non-official; * Debian includes the work of many companies and organizations; * Debian is 100% free software; .. Note:: SLIDE 4 - Things Debian Does Well (can go through quickly) There are great things about Debian and some less great things. What those are will depend a bit on who you ask but you get common answers. What many people say is that are some things Debian has a history of doing very well (these things people like about Debian): * Supporting many, many packages. * Excellent package management tools -- ``apt-get`` as one example; * A consistent (and enforced) distribution wide policy; * A huge volunteer base; * A commitment to Free Software philosophy; .. Note:: SLIDE 5 - Things Debian Does Badly (can go through quickly) There are some things Debian has a history of not doing quite as well: * Releasing frequently and/or predictably; * Ease of use/usability (especially the installer) for beginners; * Consistency on the desktop; * Accountability (in the corporate or institutional sense); It's important to note that many of these things are getting better in Debian (the installer is a good one) and are merely a matter of time while some seem to be rather systemic limitations brought on by what Debian is. Somewhere in South Africa =========================== .. Note:: SLIDE 6 - Mark Shuttleworth, my boss is a cosmonaut Meanwhile, somewhere in South Africa, Mark Shuttleworth decides he wants to give back to and get involved with Debian and fix some of those problems above (I'll go into what these are in a moment). For those that don't know Mark: * He founded Thawte which was a business that used Debian; * I've heard him claim he thinks he was the first person to upload Apache to Debian; * He was the first African in Space; * Funds lots of Free Software work in South African in particular but globally as well; * He founded Canonical Ltd. to, among other things, create a new distribution that was one being done differently (and better) than the others he saw. Ubuntu is Born =============== .. Note:: SLIDE 7 - Ubuntu is born (big Ubuntu symbol) Ubuntu's original name was, and I'm serious, "no-name-yet.com" Finally, Mark settled on the name Ubuntu which he though represented the spirit of sharing and cooperation that he found appealing in Free Software. ?Joke: People say that Ubuntu is the distribution with the funny name -- but that's only because they've never heard of "Beernix, Penguin Sleuth,or Overclockix -- some of other 114 Debian based distributions. In terms of the Ubuntu Linux Distribution, "Ubuntu" refers primarily to the cooperation inherent in the Free Software or Open Source development model. Ubuntu in a Nutshell ===================== * Based of Debian "unstable"; * Frequent, predictable releases; * Highly integrated and consistent GNOME desktop; * Give back to Debian while we do the work and fix the bugs (not at release time); * Strong commitment to free software principles (plus a few of our own); If you look at this at this, this looks like a lot like my list of Debian criticism of Debian that I gave earlier. Lets go back to those criticisms people have with Debian as a way of getting at what Ubuntu does well. Releases ========== .. Note:: SLIDE 8 - Ubuntu releases Problem: "Releasing frequently and/or predictably;" People are upset because Debian doesn't release: * often enough (people are stuck with old software) * predictably is also a problem for people who don't care about frequency but have business plans based on time that they need to balance This introduces security issues as well because people expect frequent releases: * Debian has a system called testing and one called unstable. testing is often a good choice for many people, but Debian doesn't officially support security fixes in either testing or unstable. Solution: Ubuntu will release Every Six Months! Canonical hired Jeff Waugh, the release manager for GNOME to release Ubuntu every six months coordinating with GNOME release schedules. It also pays developers to make sure that it happens. * releases based off of Debian unstable * twice a year * eighteen months support for releases * smooth upgrade paths between releases Usability ============ Problem: Ease of Use (especially the installer) for beginners This problem is relatively self-explanatory. .. Note:: SLIDE 9 - usability Joke: About the Pope. Lets stop talking about the grandmothers and focus on software that is "easy enough that the pope could use it." Solution: * Ubuntu pays folks to work hard on the Debian installer to improve it in Debian. Additionally, Ubuntu modifies the installer for branding and to remove questions or add defaults. By default, the Ubuntu installer Just Works. We don't *have* to be everything to everyone. We don't have to run on every architecture. * GNOME to the rescue * Consistency + Simplicity == Ease of Use Desktop Integration ===================== .. Note:: SLIDE 10 - Emphasis on the desktop We support a small subset of Debian: * About 1000 basic packages * About 2000 supported packages My highlighting a small subset of Debian, we can focus on just getting it right the first time. Packages included in our last release included: * Gnome * Firefox * Evolution * Openoffice * XFree86 Talk about the way everything work together out of the box rather as opposed to the Debian way. That said, Ubuntu can also be run on servers. We've done so well with the desktop that people don't think that it could also be a server based operating system. Accountability (in the corporate or institutional sense) ========================================================= .. Note:: SLIDE 11 - Support/Canonical Joke: You may notice a similarity between the Ubuntu and Canonical logos. That was completely accidental. :) Problem: Debian isn't accountable in the corporate or institutional sense People want someone to turn to for support or service. They wan to be able to call up the authors. Debian's structure makes it very difficult for Debian to provide this in meaningful way. Solution: Canonical offers a single support place for services contracts. Transition: You may be thinking here: "But what about the things Debian already does well?" We don't to sacrifice these things so we've tried to find ways of balancing this while also not duplicating the work in Debian. Philosophy =========== .. Note:: SLIDE 12 - philosophy We keep Debian's commitment to Free Software. Our principles are, in spirit, similar to the Debian social contract and our licensing guidelines are similar to the Debian Free Software Guidelines or the Open Source Definition. Note: Page through the philosophy point by point * We don't hold binary firmware and documentation up to the standards (RFCs and GFDL documentation can go into Ubuntu). * Binary drivers are included for people that want to make the adult decision to use them in a section called "restricted." We cannot, however, support these since they are not free software. Joke: I wrote the free software philosophy page and made it as radical as I could. Everyone actually agreed with me which was a great surprise! Other Places Ubuntu Focuses ============================= .. Note:: SLIDE 13 - places we focus (GNOME and ]Python) Python: One goal is "python everywhere." It is the energy that surrounds us and binds us as Jeff Waugh has said. We are working hard to have *everything* extensible by Python. Mark *loves* Python. GNOME: A great platform for GNOME developers, testers, and users. Ubuntu will continue to carry the latest GNOME releases as soon as they become available. We're also into security. No open ports by default. No root password (only sudo) and more fun and very productive and useful stuff -- especially when you're handing the distro to people who don't have a lot of experience. History So Far =============== * Warty Warthog (not as warty as we thought) * Hoary Hedgehog (great review) * Breezy Badger (just out last week) Part II -- Development Methodology ---------------------------------------- .. Note:: SLIDE 14: Part II Overview * Big Questions and Context - Why derive? - "Fork" is a four-letter word - Difficulties of forking and derivation * Case Studies - Debian and Ubuntu * Approaches/Solutions - Derivation and Problem Analysis - Distributed Source Control - Problem Specific Tools - Social Solutions * Conclusions - Applicability Context ========= .. Note:: SLIDE 15: World of Debian Customizers There are at least 115 derived from Debian. Why Derive? **************************************** - There is a tendency in free software development community aroundq migrating toward super-projects. - The work of these communities is becoming increasingly difficult to recreate. - Single projects end up being asked to serve the needs of large communities with diverse needs. .. Note:: SLIDE 16: One Size Does Not Fit All There are 115 different distributions because there are 129 different needs. Some distributions may be redundant in their implementation but they are not redundant in their needs. Derivations, in one way or another, must exist to fit a diverse group of needs from a large group. The result: - Derivation (ironically) becomes both increasingly important and increasingly difficult to do (or at least do right). What Is Forking? **************************************** .. Note:: SLIDE 17: Fork is a Four Letter Word * Define 4 letter word * Define "Fork" (bifurcation in a project) Fork are not merely, or even primarily, technical. Forks happen on many levels (political, code, social, all of the above). * Examples of forks (emacs, gcc, etc) Difficulties of forking and derivation? **************************************** Historical view: "Forks are Bad" From the Free Software Project Management HOWTO: The short version of the fork section is, don't do them. Forks force developers to choose one project to work with, cause nasty political divisions, and redundancy of work. *In the best situations*: competition, redundancy, tracking outside project in addition. Using poor merge tools *In the worst (common) situations:* things get dropped on the floor. Forking has historically been so bad that a threat can keep the fork from happening. Case Studies ============== The case studies here include: * Debian * Ubuntu Ubuntu is a derived distribution of Debian. The Ubuntu derivation is significant: * Code level changes (mostly trivial) to 1300 packages. Derivation is also different. .. Note:: SLIDE 18: Ubuntu Derivation Model (Explain process.) Mark Shuttleworth has said, "every line of code in our delta that must maintain has a cost. It's in *our* interests to minimize this." This means getting code into Debian or -- in whatever way -- making sure that we don't go in different directions. Approaches/Solutions ====================== Derivation and Problem Analysis **************************************** .. Note:: SLIDE 19: Look at the Type of Problems Break down the problem into a set of component parts. The example in deriving distributions can be: 1. *Selection of individual pieces of software* ``main``, ``universe``, ``multiverse`` -- e.g., UserLinux 2. Changes to the way that packages are installed or run (e.g., in a Live CD type environment or using a different installer) e.g., Anaconda, a Live CD -- also low impact 3. Configuration of different pieces of software Configuration changes can be handled different because they can be organized through a configuration system framework (e.g., Debconf, cfengine). CDDs approach this 4. Changes made to the actual software package (made on the level of changes to the packages code); Most invasive. By breaking down the problem in this way. Debian derivers have been able to approach derivation in ways that focus energy on the less intrusive problems first. Smaller teams can limit themselves to less intrusive types of changes to be successful. Distributed Source Control **************************************** .. Note:: SLIDE 20: Distributed Version Control 5-minute intro to distributed version control Distributed version control aims to solve a number of problems introduced by CVS and alluded to above by: * Allowing people to work disconnected from each other and to sync with each other, in whole or in part, in an arbitrary and ad-hoc fashion. * Allowing deltas to be maintained over time. Recently, Linus Torvalds said: In fact, one impact BK has had is to very fundamentally make us (and me in particular) change how we do things. That ranges from the fine-grained changeset tracking to just how I ended up trusting sub-maintainers with much bigger things, and not having to work on a patch-by-patch basis any more Distributed systems include Arch, TLA, Bazaar, Bazaar-NG, SVK, Darcs, Monotone, Bitkeeper, others. While Ubuntu uses this heavily to maintain it's changes -- and will use it more in the future, this is even more useful for small projects. Distributed version control allows people to maintain deltas over time. Problem Specific Tools **************************************** .. Note:: SLIDE 21: Problem Specific Tools Because there are a number of projects associated with branching a distribution (e.g., different patch system, upstream vs. non-upstream, etc), Canonical is building a front-end to Arch/VCS specifically designed for distributions. It's called HCT. Social Solutions **************************************** .. Note:: SLIDE 22: Social Solutions (Picture of Hotel Key) "Technical Solution to a Social Problem" -- unknown Things we've run into so far: * Keeping changelog entries * Working in "the right way" with projects and trying to work on their terms. * Maintainer field issues (giving credit but not giving too much credit. * Maintaining a good and open relationship with the project * Constructive engagement This is the hard part and this is where a derivation is made or broken. It is has where Ubuntu has suffered most. Conclusions ------------- Applicability ============== .. Note:: SLIDE 23: Conclusions/Applicability While distributions and other large projects are being forced to confront this idea of balancing the benefits of forking and collaboration first, any project of any size can harness this power to make a better distribution right away. Clearly, the amount of code and people is on a different scale. Clearly, the solutions that projects of radically different sizes embrace will be different.1 I believe that in the next decade, the free software community is going to see a shift toward a development methodology where forking is not bad. Through this shift and through many other developments in the community, free software will be faster, better, and and ultimately successful on a scale we can only imagine now. The way this will happen will be different in different projects. General Conclusions ==================== On pragmatic grounds, Free Software succeeds because it harnesses the power of collaboration toward software production in a very deep and meaningful way. Through allowing people to share while diverging, free software will gain a benefit that proprietary competitors *can't* emulate.