March of the Penguin: Ars looks back at 20 years of Linux


Sources:
http://arstechnica.com/open-source/news/2011/08/march-of-the-penguin-ars-looks-back-at-20-years-of-linux.ars
http://arstechnica.com/open-source/news/2011/08/march-of-the-penguin-ars-looks-back-at-20-years-of-linux.ars/2
http://arstechnica.com/open-source/news/2011/08/march-of-the-penguin-ars-looks-back-at-20-years-of-linux.ars/3

By Ryan Paul |

The Linux kernel was originally created by Linus Torvalds, a Finnish computer science student, and first announced to the world on August 25, 1991—exactly 20 years ago today. At the time, Torvalds described his work as a “hobby” and contended that it would not be “big and professional” like the GNU project.

But the Linux kernel turned out to be one of the most significant pieces of open source software ever developed. Over the past two decades, it has grown from a humble hobby project into a global phenomenon that runs on everything from low-cost e-book readers to a majority of the world’s supercomputers. Here’s how it grew.

From Freax to Linux

While it’s easy now to take the name “Linux” for granted, Torvalds had modestly rejected the idea of naming the new kernel after himself, instead calling it Freax. The name “Linux” was selected by Ari Lemmke, an employee at the Helsinki University of Technology (HUT) who created the folder in which the original Linux source code was hosted on the Finnish University and Research Network (FUNET) FTP server. Lemmke reportedly disliked the name “Freax” and thought that “Linux” sounded more appealing.

The original 0.01 release of Linux could not actually run. Torvalds published the source code—despite its technical shortcomings—for the benefit of Lemmke, who had expressed interest after the initial announcement. Not until the 0.02 release on October 5, 1991 could the OS function.

The earliest versions of the Linux kernel source code were published under licensing terms that made the software free to use and to redistribute for non-commercial purposes. The code was transitioned to GNU’s General Public License in 1992 following the release of version 0.12, and the Linux kernel is still distributed under that license today.

The UNIX landscape at the birth of Linux

Other prominent UNIX clones were already under development at the time of Linux’s birth, including some that were distributed in source code form for the benefit of academics and hobbyists.

The Berkeley Software Distribution (BSD), originally derived from Bell Labs UNIX code, gradually emerged as a distinct UNIX variant during the late 1980s. Most of the proprietary UNIX code had been purged by the time that the non-functional BSD Networking Release 2 (Net/2) was published in 1991, one month before Torvalds announced his kernel project. Developer William Jolitz released a functional 386-compatible port the following year.

Although BSD had significant potential and a strong developer base, the project was crippled out of the starting gate due to a lawsuit brought by AT&T’s Unix System Laboratories subsidiary, which held the rights to the proprietary System V UNIX operating system. AT&T had some questions about whether code encumbered by USL’s intellectual property rights had truly been purged from BSD in its entirety. Such concerns stifled BSD adoption until the lawsuit was settled in 1994 under terms that cleared away the cloud of uncertainty.

Richard Stallman, the idealistic founder of the Free Software movement, launched the GNU project in 1983 with the goal of building a UNIX clone under a Free Software license. The GNU project was originally going to adopt MIT’s TRIX kernel in 1986, but later decided to build on top of the Mach microkernel, which was being developed at Carnegie Mellon University (CMU).

At the start of 1991, GNU was waiting for CMU to determine if it could release Mach under a suitable software license. The actual development of GNU’s Hurd kernel on top of Mach didn’t seriously begin until late in 1991. Hurd development never gained significant momentum and consequently hasn’t seen much adoption. Hurd is still under active development, however, and will soon be supported in an official Debian port.

MINIX also played a major role in early Linux history. Vrije Universiteit Amsterdam professor Andrew Tanenbaum created the MINIX kernel and released the source code in 1987 through textbook publisher Prentice Hall, which sold copies to students and researchers for $69. MINIX was built with a microkernel architecture that reflected what the academic community then regarded as the best practices of operating system design.

Tanenbaum had created MINIX and an accompanying textbook to teach operating system design, so he had no interest in evolving the code into a production-quality platform. As he explained in a 2004 essay about the history of MINIX, he wanted to keep the code as svelte and comprehensible as possible for the benefit of his students. He turned down requests to extend the code with various features that would have increased its complexity and diminished its expository pedagogical value.

GNU and MINIX both played an essential part in Linux’s creation. The earliest versions of Linux were bootstrapped on MINIX and drew inspiration from the MINIX kernel. Early Linux users also depended heavily on GNU’s tools and its C compiler—as many still do today.

“Sadly, a kernel by itself gets you nowhere. To get a working system you need a shell, compilers, a library etc. These are separate parts and may be under a stricter (or even looser) copyright. Most of the tools used with linux are GNU software and are under the GNU copyleft,” Torvalds wrote in the original Linux 0.01 release notes. “The linux kernel has been made under minix, and it was my original idea to make it binary compatible with minix. That was dropped, as the differences got bigger, but the system still resembles minix a great deal.” The Torvalds and Tanenbaum debate

Although Linux and MINIX had much in common at the start, including a similar filesystem layout, significant technical differences between the two kernels would later become the subject of a running debate between Torvalds and Tanenbaum.

The Linux kernel has a “monolithic” architecture, whereas Tanenbaum’s MINIX was designed as a modular microkernel. In a 1992 statement, Tanenbaum declared that the Linux kernel was “obsolete” from the start and that building a monolithic kernel was a “giant step back” from the state of the art and “a truly poor idea.” Tanenbaum went on to suggest that people who wanted a modern free operating system would be better off waiting for GNU to deliver a microkernel rather than using Linux.

Torvalds responded sharply to the criticism and defended his design decisions. Although he acknowledged the theoretical benefits of microkernel design, he argued that Linux was a better real-world kernel than MINIX due a number of pragmatic technical advantages in areas like threading and multitasking. He thought GNU might eventually produce a superior alternative, but he continued pressing forward with Linux because there was clearly demand for an immediate solution.

“I’ve got more excuses than you have, and linux still beats the pants off minix in almost all areas,” Torvalds wrote in response to Tanenbaum. “From a theoretical (and aesthetical) standpoint linux loses. If the GNU kernel had been ready last spring, I’d not have bothered to even start my project: the fact is that it wasn’t and still isn’t. Linux wins heavily on points of being available now.”

Linux arrived at a crucial time in computing history, offering the right blend of useful characteristics. Torvalds was willing to collaborate with others and add features that users wanted, the code was a clean-room implementation without any legacy intellectual property encumbrances, and it was being driven at a good pace with a focus on pragmatic considerations. These advantages allowed Linux to attract interest during its formative years.

Architectures and portability

One of the most significant technical topics that surfaced in the debate between Torvalds and Tanenbaum was the issue of portability. The Linux kernel was designed for Intel’s 386 CPU and was tightly coupled to the x86 architecture. But the professor was convinced that x86 was “not going to be around all that long” and that it would gradually be displaced by newer and more sophisticated architectures. On that basis, he believed that building a kernel for a specific processor architecture was short-sighted.

Tanenbaum was obviously wrong about the fate of the x86 architecture, but his complaint about the kernel’s initial lack of portability was based in fact. Torvalds had initially started the project in order to learn more about the capabilities of the 386 CPU in his new computer. He wasn’t even originally trying to develop a kernel—he just wanted to experiment with 386 task switching.

As the experiment evolved into a kernel, Torvalds went out of his way to exercise every corner of the 386 feature set. He also used plenty of raw assembly code in the beginning. These characteristics made the kernel technically interesting and quite powerful on x86 hardware, but they made portability seem like an untenable challenge. Torvalds described Linux as “nonportable” in a 1991 USENET post that he wrote in response to a developer who wanted to adapt Linux to the Amiga.

“Simply, I’d say that porting is impossible. It’s mostly in C, but most people wouldn’t call what I write C. It uses every conceivable feature of the 386 I could find, as it was also a project to teach me about the 386. As already mentioned, it uses a MMU, for both paging (not to disk yet) and segmentation. It’s the segmentation that makes it REALLY 386 dependent,” he wrote. “It also uses every feature of gcc I could find, specifically the __asm__ directive, so that I wouldn’t need so much assembly language objects. Some of my ‘C’-files (specifically mm.c) are almost as much assembler as C.”

Of course, Linux didn’t remain “nonportable” for very long. The modern Linux kernel supports a broad variety of CPU architectures and can run on hardware ranging from specialized embedded devices to mainframes. The transition from a 386-dependent design to diverse multi-architecture support was one of the most significant changes that took place in the evolution of the Linux kernel.

Some of the earliest efforts to bring Linux to additional CPU architectures targeted the Motorola 68000, the SPARC, and the DEC Alpha. Support for the Alpha was particularly significant, however, because it was the first port that was actually done upstream in the tree.

Digital Equipment Corporation (DEC) took an early interest in Linux and thought it could be a compelling entry-level operating system for Alpha hardware. DEC engineer Jim Paradis began working on a 32-bit Alpha port of Linux at the end of 1993 as an internal proof-of-concept.

DEC also helped pay for Torvalds to attend a DEC user conference in New Orleans in 1994 at the request of the conference organizer. The company was so impressed with Torvalds that it decided to give him an Alpha system so that he could work on the port himself. Torvalds was enthusiastic about the project and started collaborating with DEC engineers on a new 64-bit Alpha port that replaced the earlier experimental work done by Paradis.

Linux 1.2 was released in 1995 with support for Alpha, SPARC, and MIPS. Torvalds humorously dubbed the version “Linux 95” and made some jokes about Microsoft’s licensing model in the Linux 1.2 release notes. Corporate involvement

Jon “Maddog” Hall was instrumental in advancing DEC’s early support for Linux. He first read about Linux in 1993 and met Torvalds in 1994 at the New Orleans event. After the meeting, Hall concluded that Linux would be a major force in the future of enterprise computing; he became heavily involved in the project and established a close friendship with Torvalds. Hall has become one of the most recognizable and widely known figures in the Linux community.

During a session this month at the LinuxCon conference in Vancouver, Hall reminisced about his early involvement in Linux. He described a report that he wrote and submitted to DEC management after giving Torvalds the Alpha workstation. In the report, he had a bullet point that said “Linux is inevitable.” At the time, everybody laughed at him. Today, he says, most of the people who laughed are now working at Red Hat.

Corporate involvement has had a major role in the development and adoption Linux. DEC was one of the early stakeholders, but Red Hat—the most successful commercial Linux distributor—is arguably the defining Linux company.

The Red Hat Linux distribution was created by developer Marc Ewing in 1994. He joined forces with Linux entrepreneur Bob Young the following year to found Red Hat Software. (Red Hat is named after an actual red hat that Ewing used to wear when he attended CMU as a student. The original hat was on display at LinuxCon in an impressive gallery that the Linux Foundation assembled to present the history of Linux.)

Young served as the CEO until shortly after the company issued its initial public offering in 1999. Red Hat has seen tremendous growth since its founding. The company acquired development tool vendor Cygnus in 1999, snapped up the open source Java middleware developer JBoss in 2006, and grabbed virtualization shop Qumranet in 2008.

Red Hat, which is expected to generate a billion dollars in revenue next year, is also one of the most prolific Linux contributors. The company contributes code to the Linux platform from the kernel to the desktop. Red Hat was the number one corporate code contributor to the Linux kernel last year and accounted for roughly 12 percent of all kernel development. In recent years, Red Hat’s Linux investments in the area of virtualization have been particularly significant.

Another company with a long history of Linux involvement is SUSE—originally S.u.S.E., an acronym for Software und System-Entwicklung. The German company was founded in 1992 and sought to provide commercial services and support for Linux. The company’s Linux distribution was popular in Europe and began to see significant commercial adoption; it was especially known for its distinctive graphical management system called YaST.

When Novell got into the Linux market in 2007, the company acquired SUSE and Ximian with the goal of converging the two prominent Linux distributions into a unified product stack. Novell decided to keep the SUSE brand name, but simplified it by making the letter case consistent. Under Novell’s ownership, SUSE made some major advancements but always played second fiddle to Red Hat. Novell contributed roughly 5 percent of Linux kernel development last year.

Novell was recently acquired by Attachmate, which has opted to keep the venerable SUSE brand alive.

Outside of the distributor space, IBM is arguably one of Linux’s most committed corporate allies. Dan Frye, Vice President of Open Systems at IBM and the author of the company’s Linux strategy, discussed IBM’s discovery of Linux during a session at LinuxCon. From the Linux Foundation’s Linux History Gallery

Frye says that IBM was “late to the party” and first started to take Linux seriously in 1998. IBM was confident in Linux and enthusiastic about its potential, but the full scope of the operating system’s success has exceeded the company’s expectations.

IBM founded a Linux Technology Center in 1999 with the goal of enabling Linux across IBM hardware platforms; the center still exists today and employs approximately 600 engineers who work on Linux-related technology. In a 2001 statement, IBM executives vowed to invest a billion dollars in Linux.

In 2003, IBM produced a television commercial highlighting the strengths of the platform. The iconic commercial features a young boy who learns from a series of instructors; the boy is then identified by a narrator as Linux. “What he learns, we all learn,” the narrator says.

Although IBM had a lot to learn about open source software when it first started contributing to the Linux kernel, the company has come a long way and has built strong ties with the community. At LinuxCon, Frye discussed the importance of working upstream and engaging with other contributors. IBM increased its Linux productivity, he said, by requiring its employees to collaborate with each other through public community channels rather than private, internal mailing lists.

Distros

Corporate involvement has been critical to Linux’s success, but so has the community which formed around the OS. The history of Linux is tightly interwoven with the history of independent and community-driven Linux distributions that have provided a rich ecosystem to foster volunteer Linux development.

The very first Linux distribution to see widespread use, SoftLanding Linux System (SLS), was created in 1992. It was a complete Linux platform installation with approximately 600 separate tools and a basic graphical environment. The full installation used only 90MB of storage. It could run with as little as 2MB Of RAM—but required 8MB of RAM if the user wanted to run Xwindows.

SLS was soon displaced by a number of alternatives. The venerable Slackware distribution was first released by Patrick Volkerding in 1993. He was originally trying to remedy some technical deficiencies in SLS, but eventually ended up spawning his own distribution. Slackware is still actively developed by a robust community, and issued its latest release—version 13.37—in April. Slackware is a no-frills Linux distribution that is intended to function more like a traditional UNIX environment.

Debian is another important community-driven Linux distribution that was originally born out of frustration with SLS. It was founded in 1993 by developer Ian Murdock with the aim of building a distribution centered on the values of the Free Software movement. He named the distribution after himself and his girlfriend (now his ex-wife) Debra.

The project’s governance model has evolved in alignment with the original vision, leading to the establishment of the Debian Social Contract and the Debian Free Software Guidelines. One of Debian’s most significant technical contributions to Linux is the APT package management system, and Debian’s packaged software repository is massive.

Debian has also spawned a number of significant derivatives, including Ubuntu—a desktop-centric Linux distribution founded by SSL certificate billionaire and space tourist Mark Shuttleworth in 2004. Shuttleworth was previously involved in the Debian community and envisioned Ubuntu as a more modern, user-friendly, and fast-moving Linux environment built on Debian’s traditional strengths.

Today

The Linux kernel has grown tremendously since its birth in 1991. Statistics published last year indicate that the Linux kernel had 13 million lines of code (it’s now 14 million) with an average of 5 patches contributed per hour.

A significant portion of that code is now contributed by major companies, including commercial Linux distributors, consumer electronics manufacturers, and chip vendors. The Linux Foundation estimates that over 500 separate companies are involved in some capacity. Although corporate involvement remains high, statistics show that between 15 and 20 percent of contributions still come from independent developers—a group that collectively accounts for more contributions than any individual company.

During a session about the history of the kernel development process at LinuxCon, kernel developer and LWN writer Jonathan Corbet discussed the demographics of the Linux developer population and the growth of contributorship. Participation is as strong as ever, but the landscape is changing.

Corbet remarked that the volume of contributed patches hasn’t dropped off, contrary to a 1995 prediction made by prominent kernel developer Andrew Morton. The complexity of the code is increasing, however, making Linux less hackable and creating a higher barrier to entry for new contributors.

Given the maturity of the Linux kernel, there simply isn’t as much low-hanging fruit to pluck as there used to be. Corbet also pointed out that the core developer base is getting older; their hair is starting to turn gray, he quipped—if they still have any.

Torvalds remains the lead kernel developer, but a significant amount of work is delegated to a group of dedicated component maintainers. Torvalds says that he spends most of his kernel work time writing e-mails to coordinate the process and to guide technical design decisions. He is employed by the Linux Foundation and lives in Portland, Oregon with his family.

Although Linux hasn’t been a slam-dunk success in every market—it remains a dud on the desktop—the operating system has had a profound impact on the technology industry and on enterprise computing. Columbia University professor Eben Moglen, one of the original authors of the GPL, took a moment at LinuxCon to discuss his views about the role Linux plays in society. He described Linux as the steel and coal of a 21st century industrial revolution. Linux and the Internet, he said, have together changed human civilization more than any other pair of inventions.

Linux runs a majority of the world’s super computers, many of the most popular websites, and a large number of mainstream consumer electronics products. If you search the Web with Google, interact with your friends on Facebook, look up a fact on Wikipedia, read a book on a Kindle, watch Netflix on a Boxee Box, or make a call with an Android phone, you are relying on Linux. Jon “Maddog” Hall looked downright prophetic when his 1994 report called the rise of Linux inevitable.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: