Skip to content

bagof.core.magic

Attributes

REAL_TYPES module-attribute

REAL_TYPES = (numbers.Real,)

The real-number types eq_safenan recognises.

Note

NumPy registers numpy.floating as a numbers.Real, so NumPy floats are recognised without this package importing NumPy.

UNION_TYPES module-attribute

UNION_TYPES = (tx.Union,) if UnionType is tx.Union else (tx.Union, UnionType)

The union spellings this package understands.

UNSET module-attribute

UNSET = Unset()

A value that indicates that an argument was not set.

Note

This is different from None, which may be a valid value.

Classes

MagicError

MagicError(*args, **kwargs)

Bases: Exception

An exception raised by magic objects (factories, converters).

Other Parameters:

Name Type Description
this MagicHint

The MagicHint instance that raised the error.

value Any

The value that caused the error.

Methods:

MultipleCauses

MultipleCauses(causes: Iterable[Exception])

Bases: Exception

A wrapper exception that contains multiple causes.

MagicHint

MagicHint(hint: Any = UNSET)

Bases: Generic[T]

Base class for magic objects (factories, converters).

Parameters:

Name Type Description Default
hint Any

The type hint to use for this magic object. If not provided, the default hint for the class is used.

UNSET

Attributes

BOUND class-attribute instance-attribute
BOUND = tx.Any

The type hint that this magic object is bound to.

DEFAULT class-attribute instance-attribute
DEFAULT = tx.Any

The default type hint for this magic object.

FALLBACK class-attribute instance-attribute
FALLBACK = UNSET

A concrete fallback type, used when the type hint does not resolve to a concrete class - for example, when it is an abstract class, or a bare typing construct (such as Union or Literal) with no concrete origin of its own.

UNWRAP class-attribute instance-attribute
UNWRAP: Tuple[Any, ...] = (tx.Annotated, tx.TypeVar)

The hints that unwrapped, origin and args transparently unwrap before introspecting [hint][bagof.core.magic._hint.MagicHint.hint].

Note

A TypeVar is resolved to its default, its (union of) constraints, or its bound - in that order - so that a typevar is introspected exactly like the hint it stands for. This matches fallback, which resolves typevars through get_concrete_type.

Set to (tx.Annotated,) to opt out and introspect typevars as-is.

has_explicit_hint property
has_explicit_hint: bool

Whether a hint was passed to the constructor.

False when the object fell back to its DEFAULT - which is not the same as carrying a hint that happens to equal it.

unwrapped property
unwrapped: Any

The unwrapped type hint, with any hint listed in UNWRAP (by default, Annotated wrappers and TypeVars) removed.

origin property
origin: Any

The "safe" origin of the type hint

  • Any hint listed in UNWRAP is removed (by default, Annotated wrappers and TypeVars).
  • If the origin is None, the hint itself is returned.
args property
args: Tuple[Any, ...]

The "safe" arguments of the type hint

  • Any hint listed in UNWRAP is removed (by default, Annotated wrappers and TypeVars).
  • If the origin is None, returns an empty tuple.
fallback property
fallback: Any

A "concrete" fallback type for the type hint, if possible.

Methods:

rebind
rebind(hint: Any) -> tx.Self

Return a copy of this object describing a different hint.

Every other attribute is carried over, so a configured object keeps its configuration - a threshold, a pattern, a length. The memoised properties listed in [_CACHED][bagof.core.magic._hint.MagicHint._CACHED] are recomputed.

Example

>>> validator = IsGreaterThan(0)      # hint defaults to Number
>>> stricter = validator.rebind(int)
>>> stricter.threshold, stricter.hint
(0, <class 'int'>)
__call__
__call__(*args, **kwargs) -> T

Do some magic!

Subclasses must implement this method.

error
error(value: Any = UNSET, message: Optional[str] = None, **kwargs) -> MagicError

Build a MagicError for the given value and message.

The error is returned, not raised, so that the caller keeps the raise and its traceback starts where the failure is:

raise self.error(value, "Not a valid instance.")

Tip

Subclasses override this to build their own error type.

lazy_import

lazy_import(name: str)

Class attribute holding an object that is only imported on first access, so that defining a class for an optional library does not import it.

The name must be fully qualified ("package.module.Name"); it is imported the first time the attribute is read, and kept.

Warning

Any read of the class attribute imports it - including dir-and-getattr walks such as inspect.getmembers, documentation generators, or cls.DEFAULT used as a fallback registry key. Pass explicit (lazy) keys where that matters.

Example

class ToDaskArray(ArrayConverter):
    DEFAULT = lazy_import("dask.array.Array")

Functions:

eq_safenan

eq_safenan(x: Any) -> tx.Any

Map a value to a form that compares equal across NaNs.

Since float("nan") != float("nan"), comparing values that may contain NaN with == is unsafe. Apply this function to both operands before comparing them: real NaN values are all mapped to one sentinel (so that two NaNs compare equal), while every other value is returned unchanged.

Note

Only real numbers are recognised. A complex NaN is returned unchanged, and so still compares unequal to itself.

Example

>>> nan = float("nan")
>>> nan == nan
False
>>> eq_safenan(nan) == eq_safenan(nan)
True

get_args_uw

get_args_uw(hint: Any) -> tx.Tuple[tx.Any, ...]

Safe version of tx.get_args that unwraps Annotated hints.

Returns an empty tuple if the input is not a generic type.

get_concrete_type

get_concrete_type(hint: Any, fallback: type = UNSET) -> tx.Type[tx.Any]

Get a valid concrete type from a type hint.

  • If the hint is annotated, the Annotated wrapper is removed.
  • If the hint has an origin, it is used.
  • If the hint is a TypeVar:
    • its default value is used, if it has one; otherwise
    • the first of its constraints is used, if it has any; otherwise
    • its bound is used, if it has one; otherwise
    • the fallback type is used, if it is provided; otherwise
    • a TypeError is raised.
  • If the (resolved) hint is a concrete, non-abstract type, it is returned as is; otherwise
  • The fallback type is used, if it is provided; otherwise
  • A TypeError is raised.

Note

A constrained typevar has no single concrete type - it stands for the union of its constraints - so the first constraint is taken, the same way get_default takes the first value of a Literal.

Example

>>> get_concrete_type(List[int])
<class 'list'>
>>> get_concrete_type(TypeVar("T", int, str))
<class 'int'>

get_origin_uw

get_origin_uw(hint: Any) -> tx.Any

Safe version of tx.get_origin that unwraps Annotated hints.

Returns the input type, instead of None, if the input is not a generic type.

is_typeddict

is_typeddict(cls: Any) -> bool

Return true if an object is a TypedDict or a subclass of it.

Tip

This function differs from typing.is_typeddict in that it returns True for TypedDict itself.

issubclassable

issubclassable(cls: Any) -> bool

Return true if an object is a type or is TypedDict.

Tip

This function differs from isinstance(cls, type) in that it returns True for TypedDict and its subclasses, even though they are not technically types.

Note

A typing construct - Any, Union, Literal - is never subclassable, on any Python version. Some of them are classes on recent Pythons (Any from 3.11, Union from 3.14), so isinstance(hint, type) answers differently across the versions this package supports.

issubscriptable

issubscriptable(x: Any) -> bool

Check that an object is subscriptable (i.e. can be used with []).

True if the object is a type and has __class_getitem__, or if it is an instance and has __getitem__. Otherwise, returns False.

safe_get_args

safe_get_args(hint: Any, unwrap: Any = ()) -> tx.Tuple[tx.Any, ...]

Safe version of tx.get_args.

Returns an empty tuple if the input is not a generic type. Can also unwrap some hints (e.g. Annotated) if asked.

safe_get_origin

safe_get_origin(hint: Any, unwrap: Any = ()) -> tx.Any

Safe version of tx.get_origin.

Can also unwrap some hints (e.g. Annotated) if asked.

Note

Unlike typing.get_origin, this returns the input hint itself, instead of None, when the hint is not a generic type.

safe_isinstance

safe_isinstance(obj: Any, cls: Any) -> bool

Safe isinstance (does not fail if second argument is not a type).

Warning

A TypedDict cannot be instance-checked. Python refuses isinstance(value, SomeTypedDict) outright, and a TypedDict leaves no trace on the dict it describes, so there is nothing to recognise at runtime. This function therefore answers False for one; validate the shape of the dict instead.

Example

>>> safe_isinstance(1, int)
True
>>> safe_isinstance(1, (str, int))  # a tuple, like `isinstance`
True
>>> safe_isinstance(1, "not a type")  # no error
False

safe_issubclass

safe_issubclass(subcls: Any, cls: Any) -> bool

Safe subclass (does not fail if arguments are not types).

Warning

If cls is a TypedDict, this function looks at subcls's __orig_bases__, instead of its __bases__. A plain dict is not a subclass of a TypedDict - the relation only holds the other way round.

Example

>>> safe_issubclass(bool, int)
True
>>> safe_issubclass(bool, (str, int))  # a tuple, like `issubclass`
True
>>> safe_issubclass(int, "not a type")  # no error
False

type2hint

type2hint(x: Any) -> tx.Any

Convert a type to a (subscriptable) type hint.

  • If the input is a type, and it does not have __class_getitem__, we try to find its corresponding type hint. For example, in python 3.8, type2hint(list) returns typing.List.
  • Otherwise, the value is returned as is.

Example

>>> type2hint(frozenset)
typing.FrozenSet
>>> type2hint(3)  # not a type, so unchanged
3

typeddict_required_keys

typeddict_required_keys(cls: Any) -> tx.FrozenSet[str]

The required keys of a TypedDict.

Reads __required_keys__ where the class has it -- the only source that accounts for Required / NotRequired (in either nesting with Annotated) and for inheriting from bases declared with a different total=.

Falls back to __total__ where it does not: typing.TypedDict gained __required_keys__ only in Python 3.9, and before that a key's requiredness came from the class's total= alone -- per-key Required/NotRequired did not exist.

Warning

On older Pythons, a typing.TypedDict that inherits from a base declared with a different total= reports every inherited key as required. The stdlib does not record which class declared a key, nor a usable link back to the base -- a subclass has no __orig_bases__ and its __mro__ reaches only dict - so the true answer is not recoverable.

The error is in the safe direction: a required key that is really optional makes a valid value fail loudly, rather than letting an invalid one through. Use typing_extensions.TypedDict, which reimplements the class precisely to fix this, when it matters.

Example

>>> class Movie(TypedDict):
...     title: str
...     year: NotRequired[int]
>>> typeddict_required_keys(Movie)
frozenset({'title'})

unwrap

unwrap(hint: Any, origin: Any = (tx.Annotated,)) -> tx.Any

Unwrap a type hint from its origin, if it is in the unwrap list.

If TypeVar is one of the origins to unwrap, it will be unwrapped to its default, its (union of) constraints, or its bound - in that order.

Example

>>> from typing import Annotated
>>> unwrap(Annotated[int, "meta"])
<class 'int'>
>>> unwrap(Annotated[Annotated[str, 1], 2])
<class 'str'>
>>> unwrap(int)  # unchanged
<class 'int'>

clear_pending

clear_pending(registry: Optional[dict] = None) -> None

Drop the pending registrations (of registry, or of every registry) without applying them. Meant for test isolation.

defer

defer(registry: dict, ref: Any, value: Any, context: Optional[str] = None) -> None

Register value in registry under the object ref names, once that object has been imported - see find_name for how it is found.

This is the lazy counterpart of registry[key] = value: if the object can already be found, it is registered at once; otherwise the registration is kept pending, and resolve_pending (which get_from_registry calls before every lookup) moves it into the registry as soon as the object can be found. Nothing is ever imported or evaluated to find it.

Names that only exist for type checkers

A name imported under if TYPE_CHECKING: does not exist at runtime, so a key spelled with it never resolves. Use the real dotted path - ForwardRef("dask.array.Array"), not an alias such as "da.Array" - or ForwardRef("Array", module="dask.array"). Builtins are not looked up either: ForwardRef("int") never resolves (key the registry by int itself). pending lists the keys still waiting.

Ordering

"Last registration wins" holds as for plain keys:

  • a newer registration of the same name in the same registry replaces an older pending one;
  • pending entries are applied oldest first, and an entry that resolves now is registered at once;
  • a pending entry never overrides a key already present in the registry when it resolves. It is pending only because its object could not be reached when it was registered, so a key already there for that object was (almost always) written later - typically a plain registry[obj] = value.

Consequently, two different spellings of one object that resolve in different passes are not ordered by registration: the first one to resolve wins. (Entries resolving in the same pass keep registration order.)

Lifetime

A pending entry holds a weak reference to registry when it supports one (a dict subclass), and is dropped once the registry is collected; a plain dict is held strongly until the entry resolves, or until clear_pending. value is always held strongly.

Parameters:

Name Type Description Default
registry dict

The registry to fill.

required
ref ForwardRef

The forward reference. It must hold a dotted name ("package.module.Name", "Name"), not an expression such as "Optional[Name]". Its module= (Python 3.9.7+), if set, takes precedence over context.

required
value Any

The value to register.

required
context str

The module a relative name is looked up in, usually the __module__ of the class being registered.

None

Raises:

Type Description
TypeError

If ref is not a ForwardRef.

ValueError

If ref does not hold a dotted name.

find_name

find_name(name: str, context: Optional[str] = None) -> tx.Any

The object a dotted name refers to, if it is already imported. Never imports anything, and never evaluates the name.

  1. Relative: the first part of the name is looked up in the globals of the context module (if it is imported), and the rest is walked as attributes. This follows the module's own runtime imports, such as import numpy as np for "np.ndarray".
  2. Absolute: the longest dotted prefix of the name present in sys.modules is taken, and the rest is walked as attributes.

A module is always read through its __dict__, so a module-level __getattr__ (PEP 562) cannot import anything: a submodule that has not been imported yet is not found. Builtins are not looked up.

Returns:

Type Description
Any

The object.

Raises:

Type Description
NameError

If the name cannot be found (yet).

has_module

has_module(name: str) -> bool

Whether the top-level package of name can be imported, without importing it.

Only the top-level name is looked up - finding a submodule's spec would import its parent package - so has_module("dask.array") tells whether dask is installed.

is_forward_ref

is_forward_ref(obj: Any) -> bool

Whether obj is a ForwardRef (a lazy key).

pending

pending(registry: Optional[dict] = None) -> tx.List[tx.Tuple[tx.Optional[str], str, tx.Any]]

The registrations still pending (in registry, or in any registry), oldest first, as (context_module, "dotted.name", value). Meant for debugging a key that never resolves.

resolve_pending

resolve_pending() -> None

Move the pending registrations that can now be resolved into their registry, oldest first. Returns at once when nothing is pending.

get_from_registry calls this before every lookup, so a caller only needs it before reading a registry directly. A resolved entry is not written over a key already present for its object - see the ordering note of defer.

Cost

Every pending entry is looked up on each call, so a lookup costs time linear in the number of entries that never resolve (about a microsecond each - an optional library that is installed but never imported, or a misspelt name).

Note

A name that resolves to an unhashable object cannot key a registry: its entry is dropped with a RuntimeWarning.

get_default

get_default(hint: Any) -> tx.Any

Get a default value from a type hint.

  • If the hint is a Literal, the first value in the literal is returned (None, if None is one of the literal's values).
  • If the hint is a Union that contains NoneType, None is returned.
  • Otherwise, if the hint is a Union, we recurse through its sub-hints and return the first default found.
  • If no default value can be found, a TypeError is raised. A factory should then be used.

get_from_registry

get_from_registry(hint: Any, registry: dict) -> tx.Any

Get the best matching value from a registry whose keys are types or type hints.

The best match is the registry key that is the narrowest superclass (or superhint) of hint, following its MRO; exact matches are always preferred. If hint is Annotated and no match is found for it directly, the search is retried against its unwrapped hint.

Exact matches are by hint equality, so List[int] and list[int] are different keys, but Union[int, str] and Union[str, int] are the same one.

Pending lazy keys (see defer) whose target has been imported since are moved into their registry first, so every caller sees them.

Two keys at the same distance from hint are told apart by specificity (a subclass key beats its superclass), and then by registry order. Distances along an MRO are all different, so a true tie only happens between keys hint is a virtual subclass of (such as two unrelated ABCs it is registered with). A lazy key enters the registry when it resolves, so that is the position it ties at.

Example

>>> registry = {int: "number", object: "any"}
>>> get_from_registry(bool, registry)
'number'
>>> get_from_registry(str, registry)
'any'

ishintstance

ishintstance(obj: Any, hint: Any) -> bool

Like isinstance, but the second argument can be a type hint.

  • If hint is type or Type[...], checks that obj is a type and that it is valid subclass of the hint argument.
  • If hint is a Literal, checks that obj is one of its values. The value must match in type as well: True is not a valid Literal[1], even though True == 1.
  • If hint is a Union, checks obj against each of its members.
  • Otherwise, returns issubhint(type(obj), hint).

Warning

A container's item types are not checked: a value carries its type, and a type carries no arguments, so [1, 2] is a valid List[str] as far as this function is concerned. (Python itself refuses isinstance(x, list[int]) for the same reason.) Checking the items means iterating them, which is the caller's decision to make - bagof.validators does it.

Example

>>> ishintstance(1, int)
True
>>> ishintstance(1, Literal[1, 2])
True
>>> ishintstance(bool, Type[int])
True

issubhint

issubhint(hint: Any, superhint: Any) -> bool

Check that a hint is a sub-hint for another hint.

A hint is a valid subhint if all values that are valid for the hint are also valid for the superhint.

Arguments are compared covariantly, so List[bool] is a subhint of List[int]. A hint with no arguments is not a subhint of one that has them - a bare list may hold anything, so it cannot stand in for a List[int].

Note

An unparametrised Union or Literal asks a different question: is this hint one of those? So issubhint(int, Union) is False (an int is not a union) even though issubhint(int, Union[int, str]) is True. This makes them usable as a BOUND, and it is why the relation is not transitive through a bare Union.

Example

>>> from typing import List, Union
>>> issubhint(bool, int)
True
>>> issubhint(Union[int, str], Union[int, str, bytes])
True
>>> issubhint(List[bool], List[int])
True
>>> issubhint(list, List[int])  # a bare list may hold anything
False
>>> issubhint(int, str)
False