Unix: An Expanding Explanation

A year ago, I wrote Expanding Explanations, an idea for a structured way to explain a concept that lets you drill down into it at your own pace and direction. I'd been meaning to actually explain something using this technique, but never got around to it. So, at last, here is an Expanding Explanation of Unix:

Unix is many things: an important historical operating system,
  • An important historical operating system
    • Unix was an operating systemoperating system, the low-level system software designed to operate computer hardware and allow user software to run. It was developed at Bell LabsBell Labs, primarily by Brian Kernighan and Dennis Ritchie (who also invented the C programming language). It was primarily developed in the 1970s1970s, which was a watershed decade in the history of computing. Research groups like Bell Labs and Xerox PARC invented a lot of what we recognise as a computer today, including Unixa lot of what we recognise as a computer today, including bitmap graphics, the mouse, WYSIWYG editing, C, object-oriented programming and, of course, Unix. Its design was a response to the complexity of operating systems at the timecomplexity of operating systems at the time, especially its predecessor Multics, which was innovative (many Unix ideas came from Multics) but slow and overcomplicated. Unix invented or popularised many important concepts in operating systemsmany important concepts in operating systems, including hierarchical filesystemshierarchical filesystems (the nested files-and-folders structure common today), the process modelprocess model (a particular way of managing multiple running programs), "everything is a file""everything is a file" (a powerful and generic way of representing resources), pipespipes (a system for connecting multiple programs together), and shellsshells (the idea of the user's control program being a user-controlled program itself). After leaving Bell Labs, Unix evolved from a single operating system into a family of operating systemsfamily of operating systems, either licensees of the trademark "UNIX", implementers of the Unix-derived POSIX standard, or spiritually similar "Unix-like" systems. Most modern operating systemsMost modern operating systems, with the notable exception of Microsoft Windows, are officiallyofficially (as in the case of Mac OS X) or unofficiallyunofficially (as in Linux) based on Unix.
the elegant system architecture that it pioneered,
  • The elegant system architecture that it pioneered
    • Much of Unix's success is due to its simplicity and power. There are three major concepts in Unix: the Kernel, The Kernel is trusted system-level codetrusted system-level code, which means it operates in a special unrestricted mode, usually implemented in hardware. It is responsible for low-level hardware accesslow-level hardware access (ie, drivers) and management of system resourcesmanagement of system resources, which includes coordinating simultaneous access to resources, and regulating sharing so that resources are used fairly. An important resource the kernel coordinates access to is the processor,processor, which can only perform a single task at a time, yet must appear to run multiple tasks at once. This is done via schedulingscheduling, where the kernel runs a task for a short period of time and then stops it so another one can run. While the kernel is the foundation of the operating system, most code runs outside of the kernelmost code runs outside of the kernel for reasons of flexibilityflexibility (the whole kernel has to be updated at once and requires a reboot) and securitysecurity (kernel code has unrestricted hardware access, so it's a juicy target for hackers). Code outside of the kernel is known as user-space. Processes, Processes are a representation of code running in user-space, along with everything needed to run iteverything needed to run it, including the code itself, processor stateprocessor state (data stored by the processor, like where in the code it's up to), memorymemory (each process gets its own virtual memory that maps to various parts of physical memory or disk space), open filesopen files (a list of which files the process has open), and scheduling informationscheduling information (like whether the process is waiting for something and how long it's been waiting). Processes are untrusteduntrusted, meaning they are not allowed to access the underlying hardware directly, so to perform trusted operations they use system callssystem calls, which are special commands that stop the current process to trigger code in the kernel. Using system calls, processes can create copies of themselvescreate copies of themselves using the "fork" system call and run new processesrun new processes using the "exec" system call (replacing the current process, which makes fork-exec a common pairing), among many other thingsmany other things. Almost any operation, from getting the time to reading from the keyboard, involves system calls. Many important system calls are used to deal with data, in the form of files. and Files. Files are Unix's representation of resources. Most commonly, files are static datastatic data like a document or a video, but in Unix files are also used for all sorts of thingsall sorts of things: audio, live system information, random numbers... even an infinite sequence of zeroes. This is a philosophy called "everything is a file", because (almost) all resources are represented this way(almost) all resources are represented this way, although modern Unix environments break from this by having separate systems for things like networking and graphics. Files are also used to represent a process's input and outputinput and output (known as stdin and stdout). Two processes can be joined together by connecting the input of one to the output of anotherconnecting the input of one to the output of another (known as piping). When connected in this way, programs form a pipeline, where each individual program is simple, but powerful in combination with others. This model came to be a central part of the Unix philosophy.
and a general philosophy for software development.
  • A general philosophy for software development
    • Unix, as a whole, is notable not so much for what it did do as how much it didn't do. Unix, as a whole, is notable not so much for what it did do as how much it didn't do. Comparable systems existed at the time and have been created since, but what sets Unix apart from them is its economy of ideas, its sense of simple and elegant power. You can see this in its historical origins, in its minimal architecture, and most of all in its use of pipes and the toolbox model that resulted. Instead of one enormous system to do everything, Unix was lots of tiny systems that could do anything. Instead of one enormous system to do everything, Unix was lots of tiny systems that could do anything. The Unix philosophy is said to be "do one thing and do it well". Perhaps it's better reflected in Sun's old motto, "the network is the computer". The number of combinations of all those tiny tools was far greater than any one piece of software could hope to be. In a time when many modern pieces of software run into the tens of millions of lines of code, it can be helpful to remember this. Unfortunately, in many ways, even modern Unix is no longer a good representative of the Unix philosophy. Unfortunately, in many ways, even modern Unix is no longer a good representative of the Unix philosophy. Successive versions have become more complex and monolithic, mostly in the pursuit of greater performance and control over the user experience. In that sense, the Unix philosophy is at odds with the modern product philosophy: you can't control and optimise every facet of an experience when that experience is actually made up of a hundred independent parts. Maybe the transformation of software from tool into product means the end of the Unix way. However, even if operating systems are no longer Unixy, there is still hope in the web. However, even if operating systems are no longer Unixy, there is still hope in the web. Everything is a link, human-readable text still reigns, and the value isn't any individual member, but the connections between them. Although programming for the web is often far from elegant, it does provide an enormous amount of power, flexibility and decentralisation. The implementation might be questionable, but the web's principles echo those that appeared in Unix many years ago.