There has been much buzz about new features and functionality in MacFUSE 2.0. Besides the MacFUSE 2.0 release, there are still more new and interesting things to be discussed in the realm of user-space file systems. As I described and briefly demonstrated during my recent talk at the Googleplex, I wrote several new user-space file systems to "celebrate" two years of MacFUSE.

Now that the talk’s video is available. I’ve written some notes on the new file systems themselves. There is a lot of new information and new code for those interested in file systems, Mac OS X, and operating systems in general. Much of it is academically and practically useful, and some could have significant potential down the road. You could also think of it as a new year gift to the Mac OS X and open-source communities.

This is a meta note on how you can go about discovering and understanding the what, how, and why of everything I’m talking about.

1. MacFUSE State of the Union Talk (2008) Video

If you were not at the talk, watching the video would provide useful context and should help greatly in understanding what follows.

2. AncientFS

Next, you could look at AncientFS, which I introduced, among other things, during the talk. AncientFS lets you mount ancient (and in some cases, current-day) "data containers" as regular volumes on Mac OS X. It supports the following formats.

• v1tap—DECtape tap tape archive; UNIX V1
• v2tap—DECtape tap tape archive; UNIX V2
• v3tap—DECtape tap tape archive; UNIX V3
• ntap—DECtape/magtape tap tape archive; 1970 epoch
• tp—DECtape/magtape tp tape archive
• itp—UNIX itp tape archive
• dtp—UNIX dtp tape archive
• dump—Incremental file system dump (512-byte blocks, V7/bsd)
• dump1kI—Incremental file system dump (1024-byte blocks, V7/bsd)
• dump-vn—Incremental file system dump (512-byte blocks, bsd-vn)
• dump1k-vn—Incremental file system dump (1024-byte blocks, bsd-vn)
• v1ar—Very old (0177555) archive (.a) from First Edition UNIX
• v2ar—Very old (0177555) archive (.a) from Second Edition UNIX
• v3ar—Very old (0177555) archive (.a) from Third Edition UNIX
• ar—Current (!<arch>\n), old (0177545), or very old (0177555) archive (.a); use (v1|v2|v3)ar for UNIX V1/V2/V3 archives
• bcpio—Binary cpio archive (old); may be byte-swapped
• cpio_odc—ASCII (odc) cpio archive
• cpio_newc—New ASCII (newc) cpio archive
• cpio_newcrc—New ASCII (newc) cpio archive with checksum
• tar—ustar, pre-POSIX ustar, or V7 tar archive
• v1—First Edition UNIX file system
• v2—Second Edition UNIX file system
• v3—Third Edition UNIX file system
• v4—Fourth Edition UNIX file system
• v5—Fifth Edition UNIX file system
• v6—Sixth Edition UNIX file system
• v7—Seventh Edition UNIX file system
• v10—Tenth Edition UNIX file system
• 32v—UNIX/32V file system
• bsd—BSD file system (V7-style with fixed-length file names; e.g. 2.9BSD or 4.0BSD)
• bsd-vn—BSD file system (pre fast-file-system “UFS” with variable-length file names; e.g. 2.11BSD for PDP-11)

To learn more about AncientFS and how to use it, please read the AncientFS article. Enjoy seeing ancient data seamlessly in the modern namespaces of Mac OS X!

3. The UnixFS Layer

AncientFS also led to UnixFS, a general-purpose abstraction layer that proved useful in getting several other "alien" file systems up and running on Mac OS X. It is particularly useful for "Unix-style" file systems, where you need concepts such as on-disk and in-memory superblocks and inodes. As I mentioned during the talk, I used UnixFS in conjunction with ad-hoc "Linux emulation" to bring support for the UFS, System V, and Minix file system families on Mac OS X. That amounts to a large number of new file systems, but it was easier than it sounds because the idea was to take existing Linux kernel implementations of these file system families and make them work in user space on Mac OS X!

UnixFS is currently rather "beta" and not a formal API by any means. Since it is a programming interface and doesn’t do anything by itself, it will be of interest only to developers at this point. In future, it may also evolve into a "LinuxFS" layer that could make it even easier and faster to systematically make Linux kernel-based file systems work on other platforms with very few code changes. For now, you can browse some bits of code.

You can, however, use the aforementioned UnixFS-based file system families: UFS, System V, and Minix. To do so, check out the relevant part of the MacFUSE source tree and compile one or more (or all) of the UnixFS-based file systems—it is quite straightforward: a single make in the filesystems/unixfs/ subdirectory in the MacFUSE source tree should build all of them. (You must have MacFUSE installed, of course.)

$ svn co macfuse.googlecode.com/svn/trunk/filesystems/unixfs
$ cd unixfs
$ ls -F
Makefile	common/		sysvfs/
ancientfs/	minixfs/	ufs/
$ make
...
$

4. The UFS Family

This is a user-space implementation (read-only) of the UFS file system family. Most of the UFS-specific code comes from the Linux kernel and is largely unchanged. Specific UFS flavors supported are as follows.

• old—the oldest UFS format
• sun—used in SunOS/Solaris
• sunx86—used in the x86 versions of SunOS/Solaris
• hp—used in HP-UX
• nextstep—used in NEXTSTEP
• nextstep-cd—used in NEXTSTEP CDROMs
• openstep—used in OPENSTEP
• 44bsd—used in FreeBSD, NetBSD, OpenBSD, and Mac OS X
• ufs2—used in FreeBSD 5.x

5. The System V Family

This is a user-space implementation (read-only) of the System V file system family. Most of the sysvfs-specific code comes from the Linux kernel and is largely unchanged. Specific sysvfs flavors supported are as follows.

• svr2—used in SVR2
• svr4—used in SVR4
• xenix—used in Xenix
• coherent—used in Coherent Unix

6. The Minix Family

This is a user-space implementation (read-only) of the Minix file system family. Most of the minixfs-specific code comes from the Linux kernel and is largely unchanged.

The following image summarizes the new file system capabilities Mac OS X gets as a result of these exercises.

7. The "One More Thing" Thing

Although it isn’t directly file system related, the last thing I demonstrated during the talk was the ability to run ancient Unix (PDP-11) binaries seamlessly on Mac OS X. PDP-11 aside, the techniques used to do so are generally useful for research and experimentation because unlike Linux, Mac OS X does not allow developers to extend the set of binary formats that can be "natively" executed by the kernel. The demo shows the Fifth Edition Unix kernel being compiled on Mac OS X using the original C compiler toolchain from a Fifth Edition disk image mounted using AncientFS. Additionally, there’s an "authentic" reproduction of the following ominous error message by running the original mv executable from Sixth Edition Unix.

	values of β will give rise to dom!

For more details, please watch the last section of the talk video. This is very preliminary work for which no source code or binaries are available yet.

This entry was posted on Tuesday, December 16th, 2008 at 12:46 pm and is filed under Computer Science, Mac OS X, MacFUSE, Operating Systems. You can follow any responses to this entry through the RSS 2.0 feed. Responses are currently closed, but you can trackback from your own site.