Sunday, September 15, 2024

Portability across platforms is a topic that is not often heard of anymore nowadays. In a certain way, the problem has kind of been solved. The POSIX standard defines a common interface for systems programming. On top of that we have frameworks, SDKs, software stacks, web apps, and otherwise we will emulate or throw a virtual machine at it. On a deeper level however, the incompatibilities between operating systems still exist, and a systems programmer will run into this problem every once in a while.

Even in this day and age, we should not take for granted that our software compiles and runs on Linux, Windows, and mac flawlessly, and I still catch myself being surprised when I see applications that do. It may not be immediately obvious, but it testifies that the developer went the extra mile to polish that code, build and test on multiple platforms.

In the C programming language (it’s near impossible to talk about systems programming without mentioning C) there is the #ifdef compiler directive to support conditional compilation. Conditional, based on whether a symbol is defined or not, so that we write code that can make use of a platform-specific feature, where available. When using platform-specific features in C it is not unusual to write a platform.h include file that contains a whole tree of #ifdef statements. It’s also not unusual having to wrestle with this tangled mess when trying to compile on a platform that is not x86_64. It’s a trap to keep adding more #ifdefs in order to support more platforms; for portability, you need to generalize, not specialize.

In Rust we have the #[cfg] attribute to do conditional compilation. The platform-specific configuration attributes are set by the cargo build system. For example, we can test for target_arch=x86_64 or aarch64. Additionally, we can test for platform-specific features like avx. At the operating systems level, we can select by target_os. But again, this is specialization, and for portability reasons we should generalize instead. This is where Rust offers something more modern and convenient than C; target_family=unix covers all UNIX-like platforms; Linux, mac, BSDs, iOS, Android.

The #[cfg] attribute applies to the next line of code, which may be a single statement, or a whole block of code.

As a practical example I have a dir program that displays a directory listing. The program has an is_hidden() function that determines whether a file is hidden or not, and thus should be omitted from being displayed.

In UNIX, a file is considered hidden (by convention) if the filename starts with a dot.

#[cfg(unix)]
pub fn is_hidden(path: &Path) -> bool {
    // file is hidden if name starts with a dot
    let first = path.to_string_lossy().chars().next().unwrap();
    first == '.'
}

In Windows, files may have a HIDDEN attribute. Files that are marked SYSTEM are also considered to be hidden.

#[cfg(windows)]
pub fn is_hidden(path: &Path) -> Result<bool> {
    use std::os::windows::fs::MetadataExt;
    let attribs = path.metadata()?.file_attributes();

    const FILE_ATTRIBUTE_HIDDEN: u32 = 2;
    const FILE_ATTRIBUTE_SYSTEM: u32 = 4;

    Ok(attribs & (FILE_ATTRIBUTE_HIDDEN | FILE_ATTRIBUTE_SYSTEM) != 0)
}

Note the use of the MetadataExt; the Rust filesystem API offers a common Metadata structure, that only contains a limited set of file properties (such as file size) that are common across different operating systems. We can use a platform-specific metadata extension, that pulls a platform-specific implementation of metadata() into the namespace. Oddly enough, the platform-specific constants are nowhere to be found; therefore we have to declare const FILE_ATTRIBUTE_HIDDEN ourselves, and I literally grabbed the value from Microsoft’s Windows developer documentation.

Getting the metadata may fail, so now we have to return a Result. Note that this changes the function signature, and we must go back and adapt the cfg(unix) version of is_hidden() accordingly.

On macOS we follow the UNIX convention, but there files may also have a UF_HIDDEN flag. The macOS documentation contradicts itself in places; sometimes it reads that hidden files should not be displayed in a directory listing, while at other times it explicitly states not in a GUI. Indeed, by default macOS Finder does not show hidden files, while the macOS ls command in a Terminal will show them.

#[cfg(target_os = "macos")]
pub fn is_hidden(path: &Path) -> Result<bool> {
    // like UNIX, file is hidden if name starts with a dot
    let s = path.to_string_lossy();
    let first = s
        .chars()
        .next()
        .expect("panic: this should not have happened");
    if first == '.' {
        return true;
    }

    // macOS file is hidden if the UF_HIDDEN flag is set
    use std::os::macos::fs::MetadataExt;
    let flags = path.metadata()?.st_flags();

    const UF_HIDDEN: u32 = 0x8000;

    flags & UF_HIDDEN != 0
}

From here on things are starting to get mucky.

First off, we get a compiler error because #[cfg(unix)] is also true on macOS, causing it to include two implementations of the is_hidden() function. Therefore we need to write exclusive cfg attributes, like so:

// specifically macOS
#[cfg(target_os = "macos")]

// other UNIX variants such as Linux, but not macOS
#[cfg(all(unix, not(target_os = "macos")))]

What about other BSD operating systems? FreeBSD, OpenBSD, DragonFly also support the UF_HIDDEN flag.

I have declared const UF_HIDDEN as 0x8000. There is no guarantee that this value is exactly the same across BSDs. Crate nix does this more elegantly, which grabs the value from the system’s libc. However, crate nix does not help us with BSD file metadata; what I was really looking for is bsd::fs::MetadataExt, but that does not seem to exist. That said, std::os::macos is also not documented, but I did succeed in using it..!

It looks like you can’t group the BSDs nicely other than making a construct like:

#[cfg(any(
    target_os = "openbsd",
    target_os = "netbsd",
    target_os = "freebsd",
    target_os = "dragonfly",
    target_os = "macos",
    target_os = "ios"
))]

I have a problem with this, it borders on unmaintainable. This kind of code becomes very hard to test, very quickly. You can’t easily cross-compile, so you can’t even verify whether it compiles for that platform at all. Moreover, if another popular BSD arises tomorrow, then we have to fix all occurrences of #[cfg], and that means also touching code that is actually meant for other platforms as well … and that doesn’t sit well with me. In this case #[cfg] displays the same shortcomings as #ifdef in C.

In the end I want to say that the catch-all #[cfg(unix)] is a very welcome attribute that aids portability across platforms. Generalization is better for portability than specialization; beware to not fall into the trap of over-specializing when the goal is portability. If you do specialize (because you have to), then there are a lot of things that #[cfg] can do that I haven’t even shown here; basically it is a more matured version of #ifdef with more features out of the box.