How it compares

bagof.paths is not a new filesystem. It sits on top of the path libraries you already know: the standard library's pathlib, universal-pathlib (UPath), and cloudpathlib (CloudPath and AnyPath). It gives them one shared set of methods, so the same code reads and writes local files, s3:// objects, and anything else those libraries reach.

At a glance

The same task, three ways

Read a file that might be local or in a bucket, with one function:

from bagof.paths import Path

def load(where: str) -> bytes:
    return Path(where).read_bytes()

load("/data/train.bin")           # a local file
load("s3://my-bucket/train.bin")  # an object on S3 (with a cloud library installed)

Path(where) looks at the start of the string to decide what kind of path it is. A plain path is a local file. A URL uses universal-pathlib, or cloudpathlib if you have it. read_bytes then means the same thing in both cases.

The parts that only describe a path read the same whatever is underneath:

>>> from bagof.paths import Path
>>> p = Path("/data/sets/train.zarr")
>>> p.name
'train.zarr'
>>> p.suffix
'.zarr'
>>> (p / "chunks").match("*/chunks")
True

What each one is for

pathlib

The standard library's path type. It is great for local files, and it is what bagof.paths uses for a local path. It has no idea about cloud storage, and no await version.

universal-pathlib (UPath)

UPath("s3://…") gives you a pathlib-style path over a wide range of remote stores. It has the widest reach, and it is the first thing bagof.paths tries for a URL. It does not need a cloud library to make the path, only to read or write.

cloudpathlib (CloudPath / AnyPath)

A focused, well-typed path for the big three clouds (s3, gs, azure). It uses the official cloud libraries and a local cache. bagof.paths can use it too, and it wraps a CloudPath you already have.

bagof-paths

One set of methods over all of the above, and over a path library it has never seen. For each method it uses the underlying library when it can, builds the method from simpler ones when it cannot, or raises one clear error when neither is possible. Anything a library offers that these methods do not name is still there, on path.wrapped.

What bagof-paths adds

• Local or not, in one call. is_local() says whether a path is a file on this machine, the one os.fspath accepts, without touching storage.
• Local copies for libraries that open by name. local_copy() downloads a remote file or directory under its exact name and removes it when the block ends. In one local_stage(), copies keep their places relative to each other, so a file finds its companion files.
• One error for every library. When an operation is not possible, you get the same UnsupportedPathOperation. It names the operation and the library.
• Safe defaults. Where two libraries disagree in a way that could lose data, it picks the safe answer. Removing a folder does not delete a non-empty tree unless you ask for that.
• Paths compare by where they point. Two paths to the same place are equal even when they come from different libraries. Different spellings of a scheme (s3 and s3a, gs and gcs) count as the same. A path also equals the pathlib path, UPath or CloudPath it was made from, and a local one hashes like the pathlib path, so the two mix in a set or a dictionary. A string is never equal to a path.
• A URL's query is not part of its name. The name, suffix and parent of https://host/x.nii.gz?token=1 or s3://bucket/x.nii.gz?versionId=... are x.nii.gz, .gz and the folder, so a name moves from one store to another without a token or a version riding along. The path itself keeps the query, so a read still sends it. without_query() gives the whole path without it.
• Writes that never leave a broken file. staged() builds the new version (a file or a directory) beside the target and moves it in when your code is done, or not at all if it fails; atomic=True does the same for write_bytes, write_text and open. It works on cloud paths too, and a library that saves by file name sees the target's own name.
• An await version. AsyncPath gives you the same methods as coroutines. It runs a blocking library in a background thread, so your event loop keeps moving.
• Extensible. Add a new URL scheme, or a new library's quirks, with a single function call.
• Parsing with nothing installed. Every method that only describes a path works on a URL with no backend present, and a read or write raises one clear error until a library is installed. PurePath is the same parsing surface with no I/O methods at all, for a value that must never touch storage.