Registration and errors

How to add support for a new URL scheme or a new path library, and the errors these paths can raise.

bagof.paths.register_protocol

register_protocol(scheme: str, *, bucketed: bool = False, absolute: bool = False, aliases: Iterable[str] = (), query: bool = True, fragment: bool = False, driver: Callable[..., Any] | None = None, storage_options: Mapping[str, Any] | None = None) -> None

Register (or replace) the traits for a URL scheme.

A later registration replaces an earlier one wholesale: any trait not passed reverts to its default, aliases and all. Use this to define a scheme, not to tweak one. In particular, re-registering a built-in scheme (s3, gs, az) to add storage_options would drop its bucketed/absolute traits and detach its aliases -- to attach connection options to a scheme that already exists, use set_storage_options, which keeps the other traits.

storage_options gives default connection options (endpoint, credentials, ...) forwarded to the driver for every path of this scheme; a per-call storage_options overrides them key by key.

query=False says a ? in this scheme's URLs is an ordinary character of the path, kept in its name; by default it starts the query, which the name leaves out. fragment=True says a # starts a fragment, as in http; by default it is an ordinary character.

Register at import time -- see the module note on identity.

bagof.paths.set_storage_options

set_storage_options(scheme: str, storage_options: Mapping[str, Any] | None = None) -> None

Set the default connection options for a URL scheme.

Unlike register_protocol, this keeps the scheme's other traits (bucketed, aliases, the preferred driver). It is the way to attach an endpoint or credentials to a scheme that is already understood::

set_storage_options("s3", {"endpoint_url": "https://minio.local"})

The options replace this scheme's current defaults; a per-call storage_options on the constructor still overrides them key by key. Setting them on an alias sets them on the store it names. Call at import time -- see the module note on identity.

bagof.paths.ProtocolTraits

ProtocolTraits(*, bucketed: bool = False, absolute: bool = False, aliases: Iterable[str] = (), query: bool = True, fragment: bool = False, driver: Callable[..., Any] | None = None, storage_options: Mapping[str, Any] | None = None)

What varies about a URL scheme, as data.

Parameters:

Name	Type	Description	Default
`bucketed`	`bool`	The first path component is a bucket/container -- drives `bucket` and the drive/root split.	`False`
`absolute`	`bool`	Paths of this scheme are always absolute (most remote stores).	`False`
`aliases`	`Iterable[str]`	Other schemes that name the same store (`s3a` for `s3`); folded onto this scheme for identity so the spellings compare equal.	`()`
`query`	`bool`	A `?` in a URL of this scheme starts its query, so the path's name, parts and parent leave it out, and a name moves between stores without it. On by default; turn it off for a store whose keys hold `?` as an ordinary character that must stay in the name.	`True`
`fragment`	`bool`	A `#` in a URL of this scheme starts its fragment, which the name leaves out too. On for `http` and `https` only: elsewhere `#` is an ordinary character of a name (`Invoice #12.pdf`).	`False`
`driver`	`Callable[..., Any] \| None`	A preferred factory (a driver class, or a `str -> path` callable) used to build a path of this scheme before the availability order.	`None`
`storage_options`	`Mapping[str, Any] \| None`	Default connection options (endpoint, credentials, ...) forwarded to the driver for every path of this scheme. Per-call `storage_options` override these key by key. Held as data and never printed -- these commonly hold secrets.	`None`

A path library of your own

Any object that looks like a path can be wrapped: it needs __fspath__, a parts attribute, or a path attribute. The wrapper then uses the object's own members, by name:

I/O (read_bytes, open, iterdir, ...) is called on the object. Missing members are built from the ones it has where possible: read_text from read_bytes, read_bytes from open, and so on.
The path's parts (name, stem, suffix, parts, parent, joinpath, /, with_name, with_suffix, relative_to, ...) must come from the object itself. They are not computed from its string, because a new path, such as a parent, has to keep the object's configuration (its credentials, client or session), which a string does not hold.
with_segments(*parts), pathlib's way to make a new path of the same kind, lets the wrapper build a related path, for instance without its query, or as a copy target. Give a remote path library one.

The simplest way to get all of this is to subclass a path class that already has it, such as pathlib.PurePosixPath or universal-pathlib's UPath. A member the object lacks raises UnsupportedPathOperation naming it.

For a test, a hand-written fake is rarely needed: universal-pathlib's memory:// is a real remote path that needs no network or credentials.

from bagof.paths import Path

p = Path("memory://bucket/data/scan.nii.gz")
p.write_bytes(b"...")
p.name        # 'scan.nii.gz'

bagof.paths.register_driver

register_driver(base: type, adapter: GenericAdapter) -> None

Register adapter for a driver base class and its subclasses.

A later registration wins over an earlier one for an object matching both, so a small adapter for a specific class overrides a family default.

bagof.paths.NoDriverError

NoDriverError(scheme: str, *, hint: str | None = None)

Bases: ValueError

No installed driver can interpret a URL's scheme.

Raised at construction, when Path("scheme://...") names a scheme that neither an explicit driver= nor any installed backend can build. It is a ValueError -- the constructor's historical error type for a string it cannot interpret -- so existing except ValueError handlers still catch it, and it says nothing about the scheme being unsupportable in principle: installing a backend (or registering one) can make the same string work.

The scheme is available on the scheme attribute.

bagof.paths.UnsupportedPathOperation

UnsupportedPathOperation(operation: str, *, driver: Any = None, hint: str | None = None)

Bases: _Base

A path operation is not supported by the wrapped object.

Raised when the wrapped object does not implement an operation and it cannot be synthesized from what the object does provide. The message names the operation and, where known, the wrapped driver.

bagof.paths.NoLocalPathError

NoLocalPathError(operation: str, *, driver: Any = None, hint: str | None = None)

Bases: UnsupportedPathOperation, TypeError

A path has no local file name.

Raised by os.fspath (and anything that calls it, such as open) on a remote path: an s3:// key is not a file on this machine. It is both an UnsupportedPathOperation and a TypeError, so the usual probe for "is this a local path?" still works:

try:
    name = os.fspath(path)
except TypeError:
    ...  # not a local file; use str(path), path.open(), or local_copy()

path.is_local() answers the same question without raising.

bagof.paths.CommitError

CommitError(target: Any, staged: Any, error: BaseException, previous: Any = None)

Bases: OSError

A staged version could not be moved onto its target.

Raised by commit on the staging area that Path.staged returns. target is the path the data was meant for, and the original error is chained as __cause__; an errno from it is carried over. It is an OSError, so an except OSError around a write still catches it.

Usually the target is left as it was (or restored), the staged data is removed, and staged is None. The data is kept, and staged is its location, when the stage was made with keep_on_failure=True, and always when the commit had already changed the target before it failed (a remote directory uploaded in part, say): it may then be the only complete version. When an old directory was moved aside and could not be put back, previous is where it is; it is never removed.