Add std::fs::{Home|Media}Dirs

[CAD97]

View all comments

• New ACP: ACP: std::fs::UserDirs or similar libs-team#830

Replacement for std::os::unix::xdg as suggested by libs-api in #157515 (comment). Exposes media directories common between the three big OSes in addition to the cache/config/data/state directories under a separate feature gate. API summary:

// mod std::fs
struct HomeDirs { /* ... */ }
impl HomeDirs {
    fn empty() -> Self;
    fn take(&mut self) -> Self;

    fn config_home(&self) -> Option<&Path>;
    fn data_home(&self) -> Option<&Path>;
    fn state_home(&self) -> Option<&Path>;
    fn cache_home(&self) -> Option<&Path>;

    fn set_config_home(&mut self, path: PathBuf) -> &mut Self;
    fn set_data_home(&mut self, path: PathBuf) -> &mut Self;
    fn set_state_home(&mut self, path: PathBuf) -> &mut Self;
    fn set_cache_home(&mut self, path: PathBuf) -> &mut Self;
}

struct MediaDirs { /* ... */ }
impl MediaDirs {
    fn empty() -> Self;
    fn take(&mut self) -> Self;

    fn desktop(&self) -> Option<&Path>;
    fn documents(&self) -> Option<&Path>;
    fn downloads(&self) -> Option<&Path>;
    fn music(&self) -> Option<&Path>;
    fn pictures(&self) -> Option<&Path>;
    fn videos(&self) -> Option<&Path>;

    fn set_desktop(&mut self, path: PathBuf) -> &mut Self;
    fn set_documents(&mut self, path: PathBuf) -> &mut Self;
    fn set_downloads(&mut self, path: PathBuf) -> &mut Self;
    fn set_music(&mut self, path: PathBuf) -> &mut Self;
    fn set_pictures(&mut self, path: PathBuf) -> &mut Self;
    fn set_videos(&mut self, path: PathBuf) -> &mut Self;
}

// mod std::os::darwin::fs
trait HomeDirsExt for HomeDirs {
    fn sysdir() -> io::Result<Self>;
}

trait MediaDirsExt for MediaDirs {
    fn sysdir() -> io::Result<Self>;
}

// mod std::os::unix::fx
trait HomeDirsExt for HomeDirs {
    fn xdg() -> io::Result<Self>;

    fn runtime_home(&self) -> Option<&Path>;
    fn config_dirs(&self) -> Option<env::SplitPaths<'_>>;
    fn data_dirs(&self) -> Option<env::SplitPaths<'_>>;

    fn set_runtime_home(&mut self, path: PathBuf) -> &mut Self;
    fn set_config_dirs(&mut self, paths: impl IntoIterator<Item: AsRef<OsStr>>) -> Result<&mut Self, env::JoinPathsError>;
    fn set_data_dirs(&mut self, paths: impl IntoIterator<Item: AsRef<OsStr>>) -> Result<&mut Self, env::JoinPathsError>;
}

trait MediaDirsExt for MediaDirs {
    fn xdg() -> io::Result<Self>;

    fn templates(&self) -> Option<&Path>;
    fn set_templates(&mut self, path: PathBuf) -> &mut Self;
}

// mod std::os::windows::fs
trait HomeDirsExt for HomeDirs {
    fn known_folders() -> io::Result<Self>;
}

trait MediaDirsExt for MediaDirs {
    fn known_folders() -> io::Result<Self>;
}

---

This implementation diverges from the directories crate's mapping in that we set state_dir in the non-unix constructors (to ~/Library/Application Support on Darwin and %APPDATA% on Windows). This mapping is derived from the idea that "state" files are application support files that are not important nor portable enough to the user to be "data" files.

---

The XDG paths are as described in the XDG Base Directories Specification and the xdg-user-dirs tool. $XDG_CONFIG_DIR/user-dirs.dirs is parsed directly to avoid delegating to potentially arbitrary shell execution.

The Darwin paths are loaded via the sysdir(3) API from libSystem.dylib (introduced in macOS 10.12 with a similar timeline for other Darwin OSes, deprecating the earlier NSSystemDirectories.h API). Using the File System Effectively points to preferring the Foundation framework's NSSearchPathForDirectoriesInDomain(_:_:_:) or NSFileManager.URLForDirectory instead, but calling those correctly requires an active Objective C autorelease pool, IIUC. The Library/Application Support directory is used for config_home, data_home, and state_home; the Apple documentation The Library Directory Stores App-Specific Files directly calls out placing data and configuration files in Library/Application Support, and state files are just less user-meaningful data files.

The Windows paths are loaded via the Known Folders API (introduced in Vista). config_home and data_home are placed in AppData\Roaming as files intended to be important and portable to the user, while cache_home and state_home are placed in AppData\Local as files that aren't.

---

I'm not fully confident about the handling of the XDG base directory paths which don't have good cross-platform analogs, as well as the exact API for the search path dealing functions, but I'm confident that the shape of the rest of the API does match the stdlib API style. Common paths are platform-independent enough of a needed concept to be exposed by std, IMHO, but platform-specific that a struct with public fields (even #[non_exhaustive]) seems incorrect, specifically because of platform-specific paths that we may want to expose like is already done for XDG.

The one API change I could see doing is moving state_dir into the XDG UserDirsExt. I chose not to do this for this initial implementation, though, as getting the ideal choice of fallback for both Darwin and Windows can't be achieved in an OS-agnostic way:

-	Darwin	Windows
cache	~/Library/Caches	~/AppData/Local
config	~/Library/Application Support	~/AppData/Roaming
data	~/Library/Application Support	~/AppData/Roaming
state	~/Library/Application Support	~/AppData/Local

A more drastic change would be to move all four onto the unix UserDirsExt, adding caches/application_support to the Darwin UserDirsExt and roaming_app_data/local_app_data to the Windows UserDirsExt. This would be more "correct" but seems a bit heavy-handed, as it would mean applications need to pull in OS-specific extension traits just to place their support files in something more appropriate than a ~/.appname directory.

We could also separate the "home" directory API from the "media" directory API. I'm neutral on this with one relevant note: the app-specific cache/config/data/state files need a subdirectory named after the application, so "ProjectDirs" would exclude the media directories; it could make sense to have a type with just those and a push_application_subdir method.

Switching the impl to using a pal imp::UserDirs could be reasonable, but seems at odds with the desire to have the target agnostic way to "build your own" UserDirs. An ExtraUserDirs instead of the #[allow(dead_code)] fields would make sense, I just didn't know how to best set up that in the pal layer.

---

Disclaimer: This was worked on as part of my employment at Canonical. I initially proposed it independently of my employment, but improving std's functionality is part of my job description, so Canonical told me I should use work time on it.

AI Disclosure: I did not use AI to generate any of the code, with a partial exception for VSCode's AI-assisted smart autocomplete helping with the repetitive parts of the code. All nontrivial code was handwritten. As an experiment, I did use some AI to assist in exploring the problem and API design spaces.

I tested locally on my Ubuntu developer machine, but am relying on CI for Darwin and Windows tests. 🤞

• Unanswered question: Should macOS and Windows use the same system API to set user_home (to NSHomeDirectory and FOLDERID_Profile respectively) instead of env::home_dir? (Should env::home_dir be changed to call those?)
• Unanswered question: Should Windows avoid eagerly linking SHGetKnownFolderPath eagerly? If so, how? Add std::fs::{Home|Media}Dirs #158936 (comment)
• Unanswered question: Should the set_* methods do any kind of validation, such as ensuring the path is non-empty or even absolute?
• Unanswered question: Should the set_* methods take impl AsRef<Path> instead of PathBuf? (Would introduce needless copies without a separate ownership-taking option like replace_* below.)
• Unanswered question: Do we want fn replace_*(&mut self, x: Option<PathBuf>) -> Option<PathBuf> style methods to allow transferring ownership and setting paths back to None?
• Unanswered question: The lookup APIs could theoretically return non-absolute paths. Is this something we should check for and defend against more than this already does?
• Future work: What other NSSearchPathDirectory make sense to expose in the Darwin UserDirsExt?
• Future work: What other KNOWNFOLDERID make sense to expose in the Windows UserDirsExt?

cc @joshtriplett @nia-e

[rustbot]

r? @JohnTitor

rustbot has assigned @JohnTitor.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: @ChrisDenton, libs
• @ChrisDenton, libs expanded to 13 candidates
• Random selection from 6 candidates

[rustbot]

⚠️ Warning ⚠️

• There are issue links (such as #123) in the commit messages of the following commits.
  Please move them to the PR description, to avoid spamming the issues with references to the commit, and so this bot can automatically canonicalize them to avoid issues with subtree.
  • fcd04d6

[CAD97]

That took longer to resolve CI than I would've liked… but we're green now and ready for review without any caveats.

@rustbot ready

I might come back to this in anger and rebase and squash these commits to present a cleaner history, or I might not. Depends how much I want to slack off at work