bagof.paths.Path
Bases: PurePathMixin, LocalCopyMixin, BaseWrapper
A path that works like pathlib.Path, local or in the cloud.
Make one from a path string, a URL, or any path object:
>>> from bagof.paths import Path
>>> p = Path("/data/sets/train.zarr")
>>> p.name
'train.zarr'
>>> p.parent
Path('/data/sets')
>>> (p / "chunks").suffix
''
Each method uses the underlying library when it can, builds the method
from simpler ones when it cannot, or raises UnsupportedPathOperation
when neither is possible.
Methods:
exists
Whether the path exists.
On a local path this follows Python 3.14 on every version: an
error returns False rather than raising. A name too long for the
file system is not there, and neither is one whose parent you may
not search. To see the error, call stat(), which still raises.
The other status checks (is_file, is_dir, is_symlink, ...)
do the same.
follow_symlinks=False (also on is_file and is_dir) asks about
a symbolic link itself, on every Python version.
is_file
Whether the path is a regular file.
Like exists, it returns False on an error on a local path.
is_dir
Whether the path is a directory.
Like exists, it returns False on an error on a local path.
samefile
Whether the path and other refer to the same file.
As in pathlib, both files must exist. With strict=False they
need not: the answer is whether writing one would write the other,
which is what a writer checks before it replaces the file it reads.
>>> import tempfile
>>> from bagof.paths import Path
>>> folder = Path(tempfile.mkdtemp())
>>> (folder / "sub").mkdir()
>>> new = folder / "new.bin"
>>> new.samefile(folder / "sub" / ".." / "new.bin", strict=False)
True
>>> new.samefile(folder / "other.bin", strict=False)
False
A local path resolves symbolic links and .., and a hard link to an
existing file is the same file. A name that does not exist yet
compares as it is written below its nearest existing directory, so
on a case-insensitive file system (macOS) two spellings of a new
name count as different. A remote path is the same file as another
remote path when the two are equal, and never the same as a local
one.
open
open(mode: str = 'r', buffering: int = -1, encoding: str | None = None, errors: str | None = None, newline: str | None = None, *, atomic: bool = False) -> IO[Any]
Open the path and return a file object, like open.
With atomic=True (a writing mode only) the file is written in a
staging area and moved onto the path when it is closed, so readers
see the old content or the new, never a half-written file; leaving
a with block with an error discards it. An appending or r+
mode starts from a copy of the current content. Like the plain
open, it refuses a directory and its errors name this path; if it
fails, closing included, it leaves nothing behind and raises the
error as the plain open would. See
staged.
read_text
read_text(encoding: str | None = None, errors: str | None = None, newline: str | None = None) -> str
Read the whole file as text.
write_bytes
Write data to the file as bytes, replacing any content.
With atomic=True the new content is written beside the file and
moved over it in one step, so readers never see a partial file
(see staged). Like the plain write, it
refuses a directory and its errors name this path; if it fails, it
leaves nothing behind and raises the error as the plain write would.
write_text
write_text(data: str, encoding: str | None = None, errors: str | None = None, newline: str | None = None, *, atomic: bool = False) -> int
Write data to the file as text, replacing any content.
atomic=True works as in
write_bytes.
staged
A staging area for a new version of this path.
Create a file or a directory at the staged path; on leaving the
with block it replaces this path. An error inside the block
leaves this path as it was, and staging nothing changes nothing.
>>> import tempfile
>>> from bagof.paths import Path
>>> target = Path(tempfile.mkdtemp()) / "scan.txt"
>>> with target.staged() as tmp:
... tmp.name == target.name
... _ = tmp.write_text("new")
... target.exists()
True
False
>>> target.read_text()
'new'
The staged path has the same name as this one, so a library that saves by file name (and reads the extension) works on it. It does not exist until your code creates it, as a file or a directory:
with Path("out/scan.nii.gz").staged() as tmp:
nibabel.save(image, tmp)
with Path("s3://bucket/store.zarr").staged() as tmp:
zarr.save(tmp, data) # a local directory, uploaded on exit
Staging nothing is not an error. When a library may fail quietly or write somewhere else, check that it made something before leaving the block; raising there leaves this path as it was:
with Path("out/scan.nii.gz").staged() as tmp:
nibabel.save(image, tmp)
if not tmp.exists():
raise FileNotFoundError(f"nothing was written to {tmp.name}")
copy=True starts the staged version as a copy of this path (a
file, or a whole directory), for jobs that update it in place: zarr
in "r+" mode, h5py in "a" mode, appending. On a remote path that
is a full download. If this path does not exist, nothing is copied.
Outside a with block, call commit() or discard() on the
returned Stage; its path is the staged path.
A file never replaces a directory, nor a directory a file, unless
you pass replace_type=True: as with pathlib, committing raises
CommitError with the errno of
IsADirectoryError or NotADirectoryError, and this path is left
as it was. Pass it when the change is the new version you mean, such
as a zarr store moving from a zip file to a directory.
What committing guarantees:
- A file replaces a local file in one atomic step on POSIX (readers that have the old file open, even memory-mapped, keep reading it), and a remote object in one upload.
- A directory, or a change between file and directory (with
replace_type=True), is all-or-nothing but not atomic: the old version is moved aside and the new one moved in, so it is briefly missing. On a remote path it is uploaded file by file, and readers can see a mixture meanwhile. - An error inside the
withblock leaves this path as it was and removes the staged data. - If committing fails, this path is left as it was (or restored), the
staged data is removed, and
CommitErroris raised. Withkeep_on_failure=Truethe data is kept instead, for work that is costly to redo: the error says where, and the returnedStagecancommit()again ordiscard(). Theatomic=Truewriters raise the original error rather thanCommitError, as the plain writers would, and refuse a directory at this path, as the plain writers do. - A failure that leaves this path changed is the exception: a remote
directory uploaded in part, or an old directory that could not be
moved back. Then nothing is removed, whatever the options: the
staged data is kept, and the old directory is left in a hidden
directory beside this path, never removed;
CommitErrorsays where both are.
An error that leaves the with block names this path, not the
staged one, which is gone by then: a path below or beside the
staged path is named below or beside this one. It is the same
error, with its filename changed. Inside the block, before it
leaves, it still names the staged path your code wrote.
>>> import tempfile
>>> from bagof.paths import Path
>>> target = Path(tempfile.mkdtemp()) / "store.zarr"
>>> try:
... with target.staged() as tmp:
... (tmp / "missing" / "chunk").write_bytes(b"x")
... except FileNotFoundError as error:
... error.filename == str(target / "missing" / "chunk")
True
The staging area is a hidden directory beside this path (beside the file a symbolic link names), so the move stays on one file system; a remote path is staged in a local temporary directory. While a directory is replaced, the old one waits in a second hidden directory beside this path. A crash can leave either behind. Replacing a file gives it a new inode, so other hard links keep the old content; the old file's permissions are kept. On Windows, the replacement fails while another process has the file open.
glob
glob(pattern: str, *, case_sensitive: bool | None = None, recurse_symlinks: bool = False) -> Iterator[Self]
Yield the paths matching pattern under this directory.
rglob
rglob(pattern: str, *, case_sensitive: bool | None = None, recurse_symlinks: bool = False) -> Iterator[Self]
Like glob, recursively.
mkdir
Create a directory at the path.
touch
Create the file at the path, or update its modification time.
rmdir
rmdir(*, recursive: bool = False) -> None
Remove the directory at the path.
With recursive=True the whole tree is removed. The default is
non-recursive and stays safe even on drivers whose own rmdir
recurses by default (universal-pathlib).
copy
Copy this file or directory to target; return the new path.
copy_into
copy_into(target_dir: Any, *, follow_symlinks: bool = True, preserve_metadata: bool = False) -> Self
Copy into target_dir, keeping this path's name.
Within the same store the copy keeps the store's own name for the
object, query and all, so two objects whose names differ only after
a ? stay apart. Into another store it takes name, which leaves
the query out.
move_into
Move into target_dir, keeping this path's name.
The name follows the same rule as in copy_into.
walk
walk(top_down: bool = True, on_error: Callable | None = None, follow_symlinks: bool = False) -> Iterator[tuple[Self, list[str], list[str]]]
Walk the tree, yielding (path, dirnames, filenames) per dir.
chmod
Change the file mode and permission bits.
symlink_to
Make this path a symbolic link to target.
link_to
link_to(target: Any) -> None
Make target a hard link to this path.
.. deprecated::
link_to takes the reverse argument order of
hardlink_to and was removed from pathlib in Python
3.12. Prefer hardlink_to; this is kept, and synthesized
where the driver dropped it, only for backward compatibility.
download_to
Download the path's contents to a local destination.
bagof.paths.Stage
Stage(target: Any, *, copy: bool = False, replace_type: bool = False, keep_on_failure: bool = False)
Bases: _StageBase
A staging area for a new version of a path.
Returned by Path.staged; use it as a
context manager, or call commit or discard yourself. The staging
directory exists from the moment the stage is made.
Attributes
path
property
path: Any
Where to create the new version: named like the target.
A local path, even for a remote target. It does not exist until
your code creates a file or a directory there (or copy=True
started it as a copy of the target).
Methods:
commit
Move what was created at path onto the target.
Nothing created means nothing changes. If the move or the upload
fails, the target is left as it was (or restored), the staged data
is removed, and CommitError is raised.
A stage made with keep_on_failure=True keeps the data instead and
the error says where; commit may then be called again, or
discard removes it.