The Man Also Known As "I'm Batman!"

Hello Citizen! Would you like to be my sidekick? Would you like to ride with... Batman?

Thursday, July 07, 2005

We've moved! Check out all these great articles and others at!

Linux Desktop Distribution of the Future Follow-up Part 1

Category: Commentary

Part 1: Clearing Up Misconceptions
Part 2: Refining the Ideas

Originally this week was slated for an article unrelated to Linux, yet jam packed with useful and interesting information. Sadly, that article will have to wait. The response I received to my four part series "The Linux Desktop Distribution of the Future" was overwhelming, and demands a follow up. As a result, this article will focus on answering some of the common questions and criticisms leveled against the previous series.


The following is a list of everyone I know who answered the series outside of the blog comments. Feel free to dig through these to get a feel for both the criticisms and the accolades that were put forth. Story
OSNews Story
Linux Today Story
Mark R. Hinkle's Blog (Editor of Linux World)
Brains Factory
Phil Crissman's Blog
Eric's "Extreme Boredom" Blog
House of Zeus

I do know of a few more, but my Russian is rather poor and I don't speak most Germanic and Arabic languages.

Direct Responses

There are two articles I am aware of that were published as direct responses to the series. The links and my responses are listed below:

EMerge Random

- or -

Does that Toaster Come With a Manual?

Dev/URandom's premise for responding is very odd at best.. His idea is that Linux is ready for the Desktop today, just as long as the user reads the manual first. He then attempts to drill home the concept by using expanded text like this: "What I answer to my John Doe's that want to install Linux is: read documentation first. Please understand to the last word: f i r s t."

The problem is that his argument does not hold water. Users don't read the manual before they drive a car, microwave their food, use a refrigerator, or even use a Desktop computer. In fact, Mac OS X comes with no manual to speak of! If Linux requires that the user read the manual before using the interface, it has already failed.

Thankfully most respondents set him straight. He has now changed his tune and moved on to a slightly more intelligent premise.

Jeff Williams

Mr. Williams' rebuttal was much more even handed than Dev/URandom's, and was a pleasure to read. Unfortunately, Mr. Williams seems to misunderstand my post in a few places.

In Part 2 of his response he refers to "installation" software for the disk images, and pretends that the disk images are just a reformatting of existing package systems. What Mr. Williams does not understand is that the AppDisks are never installed. This is a very important thing to understand. The AppDisks are mounted in place, then run directly out of the image. When the application exits, the image is unmounted and thus leaves the system. This is worlds different than the packaging systems of today.

In Part 3 of his response he refers to the DBFS as "more conjecture than something really implementable." Unfortuantely, he does not expand on this, so it's difficult to formulate a response. All I can say is: Yes, it is doable. Most everyone out there could do it with a standard database server. I'm suggesting doing it at the file system level. In this case, CompSci is on my side.

In the section entitled "Core system libraries," Mr. Williams attempts to agree and disagree at the same time. On one hand he says that I have identified the key issue with Linux (a lack of standardized APIs) then states that Fedora and Debian are standards. What I think Mr. Williams is missing is that the exact APIs can be modified at install time. The user can choose to install KDE, or he can choose not to. He can choose to install the SAMBA components, or he can choose not to. That choice is a powerful advantage for workstations and servers, but a very poor concept for a Desktop system where the user wishes to make his applications Just Work(TM).

In the section called "Documents", Mr. Williams argues that I have contradicted myself about special directories. However, he fails to grasp that the /Documents directory is not a special folder. It is, instead, a replacement for the user's Home directory. Inside it, the system is normalized to allow the user easy access to his documents, applications, and other functionality. This is different from GNOME and KDE which create various bits of files and folders (both hidden and visible) that have special meaning inside the system.

Finally, under "Part 4: Desktop", Mr. Williams fails to understand the concept of the saved query. His belief is that the results of the query would be saved. That was not my suggestion. My suggestion was intended to be to save the query itself, then rerun it on demand. Since the file browser would constantly be running queries, the saved query file would merely pass its contents to the file browser for execution. The results would then be displayed as if it were a "special" folder.

Now on to the more general points.

A DBFS is not an SQL Server!

One concept that seemed particularly difficult for people to accept was the idea that a file system could both serve files and be a database at the same time. From the responses, everyone seems to have gotten the idea that a heavyweight SQL Server would be required to support the file system, and that the linkage would be weak at best. If you're one of those people, place your mind at ease. I suggested no such idea.

To understand why a DBFS does not need a separate server, one must first understand that a file system is a database. As databases go, most are quite poor, but they are still databases. Despite the fact that most commercial databases maintain relationships and provide client/server abilities, all that is required to have a database is to store data in a structured fashion. By that definition, even a comma delimited file is a database!

Speaking of comma delimited file, the original computer databases were not far from these. Data was stored in records of fixed lengths, and retrievals were done by scanning the file. Columns existed only inside the program's definitions. i.e. A program might say "Characters 0-10 are the name, while characters 11-25 are the address". This scheme worked well enough, but wasn't particularly fast if you needed only a few records from the data.

Indexes were created as a solution to this issue. Indexes are really nothing more than a data structure that is faster to navigate than a full database scan. When these indexes were added to a flat database, the resulting database was referred to as an "ISAM" file. ISAM files are still used in many systems today due to their inherent simplicity.

Indexes can come in many shapes, sizes, and forms. The only purpose of an Index is to improve access to a particular piece of data. Now think for a moment. When you access a file on your hard drive, does your machine spin through every file on the system looking for the correct one? No! It provides you with a directory structure to navigate that allows you to pinpoint the correct file. Believe it or not, by navigating the directory structure, you've just followed through an index.

Fast forwarding to today, the underlying database technology has seen almost no change what so ever. Database servers build relational theory and client/server technology on top of indexes and data access, but offer little more at a lower level other than abandoning the fixed record sizes for indexed record/column locations.

In the context of my original article, my suggestions were nothing more than reorganizing the data structures that already exist on disk drives today. This is to provide a query method that does not require a jaunt through the directory structure. There is nothing incompatible about this concept, and it can already be seen in the myriad of file systems that are available today. NTFS is probably the closest file system to what I am suggesting, but BeFS and HFS+ both provide highly advanced data structures and indexing in their designs. Isn't it interesting that Linux supports all of those file systems just fine?

Yes, disk creation and repair tools would be required for this new file system. However, the same is true for every file system ever added to Linux. When you run FSCK, you're not running a generic program, you're running a wrapper around a file system specific program.

Have you tried Distro XYZ?

This is quickly becoming the most annoying question in the history of mankind. It doesn't matter *which* distro you've tried or haven't tried. They all run on similar principles. No one distro magically solves all the issues facing Linux today. Sure, some have strengths over other distros, but they also have weaknesses. As a result, you find yourself in a conversation like this:

Me: I tried RedHat, but XYZ was giving me headaches.
FanBoy1: You should try Gentoo, it fixes all your problems!
Me: I tried Gentoo, but problem ABC was a show-stopper for me.
FanBoy2: Gentoo is crud! Use SuSE!
Me: I tried SuSE, but it didn't support 123.
FanBoy3: That's because you should have been using Debian all along!
Me: I tried Debian, but it broke.
FanBoy4: That's because Debian is out of date. Ubuntu is the future!
Me: Ubuntu still has issue Z.
FanBoy5: I understand all your issues with Ubuntu. I ran into them myself, so I switched distros. You should really try RedHat!

And so this crazy circle completes.

For the record, I have tried nearly all mainstream Linux distros. You can find some very nice reviews of some of them in the history of this blog. (Originally posted in my Slashdot Journal.) No, I have not tried Ubuntu yet. I'm still waiting for my CDs to arrive due to this blog keeping me too busy to download and burn CDs.

Installation != Ease of Use

An interesting fallacy I've seen as of late is the idea that an easy installation somehow equates to ease of use. Let me put this to rest. The only reason why the user cares about how easy it is to install Linux is because Linux doesn't come bundled with hardware. Windows and Mac OS X don't have to worry, because they are already installed for the user.

Most Linux distros today do an excellent job of providing a user friendly installer. Which is great if the user has to install his OS. For long term usage however, the user doesn't really care about the installer. He cares about things like using whatever application he wants, having document compatibility, playing games, keeping his work organized, and above all else having his machine work for him instead of against him.

Far too often Linux works against its users, because it wants someone more experienced at the helm. There's nothing wrong with that as long as Linux is targeted at the workstation and server markets. But if you really want to see Linux adopted on the home or work desktop, then you need to consider how to keep Linux from fighting its users.

Shortcuts vs. Labeling

Another item that seems to have confused readers was the issue of keeping shortcuts off the desktop. I understand the confusion, so allow me to clarify: The desktop I'm proposing has no shortcuts in the traditional sense. Instead, the labeling system is used to provide a desktop label. Anything linked to the desktop label will show up on the user's desktop. This works because files can be linked to multiple labels. Thus if I have a file under the "Text Documents" label and add it to the "Desktop" label, the file will appear in both places.

Dumbed Down for the "Smart" People

An interesting criticism of my article was that the concept was "dumbed down" for the idiot savant user. If you'll forgive me, I find this to be an offensive idea. "Dumbing Down" implies that something is reduced in functionality and features until only the absolute minimum is left. An example of this is the arcade games designed for three year olds that only have one button and no joystick.

The concepts I introduced were far more sophisticated than those used today. The current file systems, for example, are quite dumb. (Except for ReiserFS. I considered the possibility of Reiser solving the DBFS need, but I found its meta-data support to be incomplete.) The DBFS I proposed would actually be quite smart. The interface I suggested contains all existing user interface component, adds some functionality, and has far more potential for the future.

The is nothing "dumb" about the suggested changes. All they do is Keep It Simple Stupid.

AppDisks vs. AppFolders

A very important distinction that seems to have slipped past many is the fact that I suggested the use of AppDisks, not AppFolders. These AppDisks are *similar* to AppFolders, save for the fact that they wrap a complete unix hierarchy into a mountable disk image.

This means that /bin, /lib, /share, and other Unix style directories still exist. For example, shared libs can be stashed inside the /lib directory, in direct opposition to the way that Mac OS X AppFolders work.

Linux Applications are so easy to install, they don't work

A common response to my article was to gloss over the issues with the packaging system and claim that there's nothing easier. Well, yes, it's quite easy to click and install from a manager like Synaptic just as long as the package is available. Many programs that users wish to run have no package, out of date packages, or the repository has moved on to a newer version, thus making their Linux system useless. In the latter case, the solution is to upgrade your system. Yet a common complaint leveled against Microsoft is that Microsoft forces users to upgrade!

In addition, there is no way to fully test a package repository. Since every package modifies the base system, the only way to prove that a package will work is to test it against every possible package configuration available! In case you're wondering, the math for that is P * P, where P is the number of packages available. A mere 100 packages could potentially result in 10,000 available configurations! That's a lot of potential for breakage! Now consider that most distros today have thousands of packages under their care, and the number is not declining.

Minor Correction: Reader Bradley Momberger has correctly pointed out that my math was a little screwy on this one. The correct forumla for the number of combinations is 2^P, which is actually quite a bit worse. 100 packages yields 1.26e30 possible combinations!

In the AppDisk system I've proposed, the application is merely a visitor to the system. It lives in its own directories and its own environment, separate from the rest of the system. Thus there is precisely P possible configurations to test. And since the one to one relationship exists, it is quite feasible to place the difficulty of testing the application on the developer.

This is all Sci-Fi Technology!

Perhaps the most amusing criticism is the suggestion that the ideas presented are "Sci-Fi" technology that couldn't possibly exist without multi-million dollar backing and years of development.

This argument, I'm afraid, ignores the fact that I'm building on top of a great deal of existing work. i.e. Standing on the shoulders of giants as it were. Take the example of applications as disk images. While I invented the concept independently, I can't take credit for being the first to deal in the concept. That honor goes to the Klik project and their CMG file technology. Granted, their CMG technology is still very immature, but it is here and working today.

Another example is the concept of creating a running system with the applications outside the /usr directory. GoboLinux has gone to great pains to experiment with alternate file system layouts. Yet another example is Rox Filer/Rox Desktop, the pioneering project behind the use of AppFolders in Linux. (GNUStep had AppFolders first, I believe, but the applications are restricted by the OpenSTEP Desktop.)

Look at the work being done by the DBFS and Beagle projects for examples of Database File System technology.

The only thing I have attempted to do is make suggestions about how disparate technologies may be pulled together to make a cohesive whole. Make no mistake. I am not suggesting a Linux desktop of the future, I am suggesting the Linux desktop of the future. Whether I play a part in it or not is irrelevant. It is already happening. The question is: Will we beat the competition or will we continue to play follow-the-leader?

In the next article, I will attempt to refine some of the concepts presented based on the feedback I've gotten. So tune in next time, same Bat Time, same Bat Channel!

Go to Part 2: Refining the Ideas >>


CMG Files
Rox Desktop

Questions? Comments? Business Ideas? Hate Mail? You can send it all to (Except for the hate mail. That belongs in /dev/null.)

WARNING: Comments have been temporarily disabled during maintenence. Comments will reappear soon.