TweetFollow Us on Twitter

File System Metadata

Volume Number: 18 (2002)
Issue Number: 12
Column Tag: Section 7

What's in there, beside the data...

by Rich Morin

One of Apple's larger challenges, in merging Mac OS and BSD, has been the need to reconcile differences between their respective ways of handling files. Previous columns have touched on some of these issues; this month, we'll go into the topic in a bit more depth.

In order to keep both BSD and Mac OS program(mer)s happy, Apple has had to implement nearly every feature found in either HFS+ (Mac OS's "Extended Hierarchical File System") or UFS (BSD's "Unix File System"). This is a challenging task; the systems evolved independently and a variety of decisions were made in different manners.

A look at links

To take just one example, consider file links (BSD) and aliases (Mac OS). Conceptually, these are very similar: they provide ways to have multiple names for a single file. The devil, of course, lies in the details.

Hard links have been around since the earliest days of Unix. A hard link is a directory entry, within the same file system, to the same item. So, for example, the "." and ".." entries that you see in an "ls -al" directory listing are actually implemented as hard links.

Because all of BSD's file metadata is stored in the file's inode (information node), anything that affects any hard link to a file affects every hard link to the file. So, for example, you can't change the ownership or permissions on just one of a file's hard links.

Symbolic links (i.e., symlinks) were invented by the BSD developers, but other versions of Unix quickly picked up on the idea. A symlink is a specially-interpreted a file, containing the name of some other file, directory, etc. When the kernel is asked to access a symlink, it grabs the contained file name, then wanders off to resolve it. The referenced file may be anywhere on the system; in fact, it may not even exist!

Aliases are quite a bit like symlinks, but they differ in some subtle ways. For instance, Mac OS will try to find the file that an alias references, even if it has been moved or renamed. As long as the referenced file stays in the same file system, Mac OS will typically succeed in finding it. Symlinks, in contrast, either match completely or not at all.

These issues aside, symlinks act (mostly) like aliases, when viewed from the Finder. An alias, however, is unusable from the BSD command line; it simply appears to be a zero-length file. In short, if you need a link that is visible everywhere, use either a hard link or a symlink...

File System Metadata

Apple has done a very creditable job in merging file system metadata. Both Mac OS and BSD programs have access to the sorts of information they need; users and even programmers can largely ignore the underlying issues. To take full advantage of the combined environment, however, we need to know what kinds of metadata it supports.

The tables below list a variety of file system metadata and ways for users to get access to their values. Programmers will have to look up the appropriate libraries and/or frameworks, but the discussion below may provide some useful hints.

Perl programmers should look at Dan Kogai's MacOSX::File module, which provides access to a wealth of file metadata. Perl versions of several Mac OS X file access commands are included in the distribution (available on the CPAN: cpan.perl.org).

My table entries are a bit cryptic, so some explanation may be in order. Get Info (GI) is part of the standard Mac OS X distribution. File Buddy (FB) and Super Get Info (SGI) are available from SkyTag Software (www.skytag.com) and Bare Bones Software (www.barebones.com), respectively.

As their terse names hint, du and ls have been in the Unix lexicon for a number of years. GetFileInfo (and a companion command, SetInfo) are part of the Developer Tools and may be found (along with several other file-related utilities) in /Developer/Tools.

I realize that I may have left out some interesting data items (not to mention useful tools!); please drop me a line if you spot an egregious error or omission. As Apple continues to evolve the file system, I'm sure that a follow-on article will be needed...

Access
   Ownership     FB, GI, SGI   ls -l
   Permissions   FB, GI, SGI   ls -l
   set[gu]id                   ls -l
   sticky                      ls -l

Prior to OSX, Mac OS didn't have much in the way of access control. In fact, the only limitations were on file sharing. As convenient as this may have been, it did not protect users from each others' mistakes, nor protect the system files from user errors. BSD, as a multi-user system, has long had the notions of file ownership and permissions. OSX now has these, as well, but you can expect to see other (e.g., capability-based) forms of access control being added over time.

The setgid and setuid bits allow programs to run with the gid (group id) or uid (user id) of the file, rather than that of the user running the command. This is a convenient way to let programs do privileged things, in a controlled manner.

The "sticky" bit has different meaning, depending on the item involved. Adding the sticky bit to a directory makes the directory "append only"; see sticky(8) for details. On some systems (but not OSX), adding the sticky bit to an executable file causes its binary image to stay in memory after the program has finished.

The first character of the permission string printed by "ls -l" indicates the type of the file. This is not the same as the Type (four-character code) shown by some Mac utilities. In fact, it is more similar to the "Kind" field: it specifies whether the item is a plain file, a directory, a symlink, etc.

Date/Time
   Access            FB            ls -lu
   Backup            FB
   Creation          FB, GI, SGI   ls -cl, GetFileInfo
   Modification      FB, GI, SGI   ls -l,  GetFileInfo

Size
   Data fork
      file length    SGI
      size on disk   SGI   ls -l
   Resource fork
      file length    SGI
      size on disk   SGI
   Total
      file length    FB, GI, SGI   ls -l
      size on disk   FB, GI, SGI   du -k

The access and modification times, as you might expect, are the last times that the file's contents were accessed or modified, respectively. The creation time, on the other hand, is quite inconsistent and potentially confusing.

In both BSD and Mac OS, the creation time is initially defined as the time that the file was created. In BSD, however, the creation time gets updated when the file's directory entry (e.g.., its permissions) has been changed. To get the latter version, use ls -cl.

Because BSD does not have the notion of file "forks", standard command-line tools do not deal with them in a particularly consistent manner. In particular, ls may list the size of a resource fork as zero.

Attributes
   Alias file          FB            GetFileInfo
   Bundle              FB            GetFileInfo
   Custom icon         FB            GetFileInfo
   Desktop                           GetFileInfo
   Extension hidden    FB, GI, SGI   GetFileInfo
   Inited              FB            GetFileInfo
   Invisible           FB, GI, SGI   GetFileInfo
   Locked              FB, GI, SGI   GetFileInfo
   No INIT resource    FB            GetFileInfo
   Shared              FB            GetFileInfo
   Stationery Pad      FB, GI, SGI   GetFileInfo
   System file                       GetFileInfo

Comments
   Attached            FB
   Finder              FB, GI, SGI

Creator, etc.
   Creator             FB, SGI       GetFileInfo
   Kind                FB, GI, SGI
   Open with           SGI
   Path                FB, SGI
   Type                FB, SGI       GetFileInfo

Miscellany
   Name locked         FB
   Package             FB
   Preview             FB, GI, SGI
   Version             FB, SGI

To the best of my knowledge, all of these items are irrelevant to most BSD commands. That is, BSD commands neither affect nor respond to these items. Consequently, I won't discuss them in any detail.

The "Comments" information is, however, an exception. OSX stores comments in a binary file called ".DS_Store", located in the same directory as the file being commented upon. It also performs some hidden magic, copying the comments to the appropriate copy of .DS_Store when a file is moved, etc.

Because the BSD commands know nothing about this mechanism, they do NOT perform the same magic. So, if you use "cp" or "mv" on a file, don't expect the comments to follow along!

In Other News

If you're reading this column, you should probably take a look at O'Reilly 's "Mac OS X for Unix Geeks" (Jepson & Rothman) and "Learning Unix for Mac OS X (Taylor & Peek). Both volumes are slim and unassuming, but contain valuable nuggets of OSX lore .

O'Reilly has just released the second (and substantially larger) edition of "Mac OS X: The Missing Manual", by David Pogue. This looks like a handy book for all sorts of Mac OS X users, whatever their background. If you want a quick reference to the parts of OSX that aren't in your area(s) of specialization, give this book a once-over.


Rich Morin has been using computers since 1970, Unix since 1983, and Mac-based Unix since 1986 (when he helped Apple create A/UX 1.0). When he isn't writing this column, Rich runs Prime Time Freeware (www.ptf.com), a publisher of books and CD-ROMs for the Free and Open Source software community. Feel free to write to Rich at rdm@ptf.com.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Latest Forum Discussions

See All

Top Mobile Game Discounts
Every day, we pick out a curated list of the best mobile discounts on the App Store and post them here. This list won't be comprehensive, but it every game on it is recommended. Feel free to check out the coverage we did on them in the links... | Read more »
Price of Glory unleashes its 1.4 Alpha u...
As much as we all probably dislike Maths as a subject, we do have to hand it to geometry for giving us the good old Hexgrid, home of some of the best strategy games. One such example, Price of Glory, has dropped its 1.4 Alpha update, stocked full... | Read more »
The SLC 2025 kicks off this month to cro...
Ever since the Solo Leveling: Arise Championship 2025 was announced, I have been looking forward to it. The promotional clip they released a month or two back showed crowds going absolutely nuts for the previous competitions, so imagine the... | Read more »
Dive into some early Magicpunk fun as Cr...
Excellent news for fans of steampunk and magic; the Precursor Test for Magicpunk MMORPG Crystal of Atlan opens today. This rather fancy way of saying beta test will remain open until March 5th and is available for PC - boo - and Android devices -... | Read more »
Prepare to get your mind melted as Evang...
If you are a fan of sci-fi shooters and incredibly weird, mind-bending anime series, then you are in for a treat, as Goddess of Victory: Nikke is gearing up for its second collaboration with Evangelion. We were also treated to an upcoming... | Read more »
Square Enix gives with one hand and slap...
We have something of a mixed bag coming over from Square Enix HQ today. Two of their mobile games are revelling in life with new events keeping them alive, whilst another has been thrown onto the ever-growing discard pile Square is building. I... | Read more »
Let the world burn as you have some fest...
It is time to leave the world burning once again as you take a much-needed break from that whole “hero” lark and enjoy some celebrations in Genshin Impact. Version 5.4, Moonlight Amidst Dreams, will see you in Inazuma to attend the Mikawa Flower... | Read more »
Full Moon Over the Abyssal Sea lands on...
Aether Gazer has announced its latest major update, and it is one of the loveliest event names I have ever heard. Full Moon Over the Abyssal Sea is an amazing name, and it comes loaded with two side stories, a new S-grade Modifier, and some fancy... | Read more »
Open your own eatery for all the forest...
Very important question; when you read the title Zoo Restaurant, do you also immediately think of running a restaurant in which you cook Zoo animals as the course? I will just assume yes. Anyway, come June 23rd we will all be able to start up our... | Read more »
Crystal of Atlan opens registration for...
Nuverse was prominently featured in the last month for all the wrong reasons with the USA TikTok debacle, but now it is putting all that behind it and preparing for the Crystal of Atlan beta test. Taking place between February 18th and March 5th,... | Read more »

Price Scanner via MacPrices.net

AT&T is offering a 65% discount on the ne...
AT&T is offering the new iPhone 16e for up to 65% off their monthly finance fee with 36-months of service. No trade-in is required. Discount is applied via monthly bill credits over the 36 month... Read more
Use this code to get a free iPhone 13 at Visi...
For a limited time, use code SWEETDEAL to get a free 128GB iPhone 13 Visible, Verizon’s low-cost wireless cell service, Visible. Deal is valid when you purchase the Visible+ annual plan. Free... Read more
M4 Mac minis on sale for $50-$80 off MSRP at...
B&H Photo has M4 Mac minis in stock and on sale right now for $50 to $80 off Apple’s MSRP, each including free 1-2 day shipping to most US addresses: – M4 Mac mini (16GB/256GB): $549, $50 off... Read more
Buy an iPhone 16 at Boost Mobile and get one...
Boost Mobile, an MVNO using AT&T and T-Mobile’s networks, is offering one year of free Unlimited service with the purchase of any iPhone 16. Purchase the iPhone at standard MSRP, and then choose... Read more
Get an iPhone 15 for only $299 at Boost Mobil...
Boost Mobile, an MVNO using AT&T and T-Mobile’s networks, is offering the 128GB iPhone 15 for $299.99 including service with their Unlimited Premium plan (50GB of premium data, $60/month), or $20... Read more
Unreal Mobile is offering $100 off any new iP...
Unreal Mobile, an MVNO using AT&T and T-Mobile’s networks, is offering a $100 discount on any new iPhone with service. This includes new iPhone 16 models as well as iPhone 15, 14, 13, and SE... Read more
Apple drops prices on clearance iPhone 14 mod...
With today’s introduction of the new iPhone 16e, Apple has discontinued the iPhone 14, 14 Pro, and SE. In response, Apple has dropped prices on unlocked, Certified Refurbished, iPhone 14 models to a... Read more
B&H has 16-inch M4 Max MacBook Pros on sa...
B&H Photo is offering a $360-$410 discount on new 16-inch MacBook Pros with M4 Max CPUs right now. B&H offers free 1-2 day shipping to most US addresses: – 16″ M4 Max MacBook Pro (36GB/1TB/... Read more
Amazon is offering a $100 discount on the M4...
Amazon has the M4 Pro Mac mini discounted $100 off MSRP right now. Shipping is free. Their price is the lowest currently available for this popular mini: – Mac mini M4 Pro (24GB/512GB): $1299, $100... Read more
B&H continues to offer $150-$220 discount...
B&H Photo has 14-inch M4 MacBook Pros on sale for $150-$220 off MSRP. B&H offers free 1-2 day shipping to most US addresses: – 14″ M4 MacBook Pro (16GB/512GB): $1449, $150 off MSRP – 14″ M4... Read more

Jobs Board

All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.