ditto.recorders¶
Recorder protocol, registry, and built-in recorders.
RECORDER_REGISTRY discovers names from installed entry-point metadata and loads
each recorder on its first successful lookup. A recorder's name is also its
persisted identifier. Membership checks, key iteration, len() and problems
do not load plugins. Names retain discovery order, followed by names added with
register() in the order they were added.
Every registration is checked against the plugin contract when the registry is
built, so problems is complete without loading anything, and loading a
recorder never changes it. A pytest run with any problem stops before
collection.
The registry is read-only apart from register(), which adds a recorder under
a new name and raises DittoRecorderConflictError if the name conflicts with
one already registered. Registered recorders cannot be replaced or removed.
Operations that read values, such as get() and iteration over items() or
values(), can load plugins and raise DittoRecorderLoadError. The fallback
argument to recorders.get() applies only to absent names; it does not
suppress errors from an installed recorder that fails to load.
copy.copy(RECORDER_REGISTRY) creates an independent registry without loading
plugins. Already loaded recorder objects are shared, while later registrations
affect only the registry they are made in. For isolated tests of
recorders.get() and recorders.register(), pass a RecorderRegistry([], [])
through their registry argument.
Recorder(dumps, loads)
dataclass
¶
Bases: Generic[T]
Recorder: a pair of functions that serialise a value to bytes and back.
A Recorder specifies how a snapshot value becomes the bytes a backend
stores, and how those bytes become a value again. The bytes are the
snapshot file's contents: a JSON recorder produces ordinary JSON text, a
parquet recorder an ordinary parquet file. The type parameter T
constrains the data type this recorder operates on. Use Recorder[Any] for
generic formats (yaml, json). Use a concrete type for format-specific
recorders (e.g. Recorder[pd.DataFrame]).
A recorder's persisted identifier, which ends its snapshot filenames and is
recorded in ditto.lock, is the name it is registered under (e.g. "yaml",
"pandas.parquet"), not a property of the recorder.
Snapshots are handled whole: the value and its serialised bytes must fit in
memory together. For a library that can only read and write files, wrap its
functions with recorder_from_files.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dumps
|
Callable[[T], bytes]
|
Function that serialises a value of type |
required |
loads
|
Callable[[bytes], T]
|
Function that deserialises bytes produced by |
required |
RecorderRegistry(entry_points=None, marks_entry_points=None)
¶
Bases: Mapping[str, Recorder]
Recorders keyed by name, imported only when first looked up.
A recorder's name is also its persisted identifier: it ends the recorder's
snapshot filenames and is recorded in ditto.lock.
Names come from ditto_recorders entry-point metadata, which needs no
import, so membership tests, iteration and problems never import a
plugin. Every registration is checked against the plugin contract when the
registry is built, so problems is complete from the start and loading a
recorder never changes it. Looking up a name a problem affects raises
DittoRecorderConflictError, so the registry never picks one of the
conflicting registrations.
register adds a recorder under a new name, and rejects a name that breaks
the contract alone or together with the names already registered. A
registration therefore never makes an earlier one unusable. Registrations
are never replaced or removed. Iteration preserves discovery order, followed
by registered names in the order they were added.
Looking up values (including through get, items or values) can load
plugins and raise DittoRecorderLoadError or DittoRecorderConflictError.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
entry_points
|
Iterable[EntryPoint]
|
Entry points to serve. Defaults to the installed |
None
|
marks_entry_points
|
Iterable[EntryPoint]
|
Entry points in the removed |
None
|
Source code in ditto/recorders/_plugins.py
problems
property
¶
Every contract problem among the registrations, found without imports.
register(name, recorder)
¶
Add recorder under name, which is also its persisted identifier.
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
DittoRecorderConflictError
|
If |
Source code in ditto/recorders/_plugins.py
__copy__()
¶
Copy registrations and loaded recorders without loading entry points.
Source code in ditto/recorders/_plugins.py
recorder_from_files(*, save, load, suffix)
¶
Adapt a library that reads and writes files to the byte-based Recorder.
Each dumps and loads call uses a fresh temporary directory holding one
file, snapshot<suffix>, which is removed afterwards whether the call
succeeds or fails. Libraries that can serialise to bytes directly should
build a Recorder from their in-memory functions instead, avoiding the
file I/O.
Only single-file formats are supported. A value returned by load must not
depend on the temporary file after load returns, so lazy or memory-mapped
readers are unsuitable.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
save
|
Callable[[T, Path], None]
|
Function that writes a value to the given path. |
required |
load
|
Callable[[Path], T]
|
Function that reads a value from the given path. |
required |
suffix
|
str
|
Filename suffix the library expects, e.g. |
required |
Returns:
| Type | Description |
|---|---|
Recorder[T]
|
A recorder whose |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in ditto/recorders/_files.py
register(name, recorder, registry=RECORDER_REGISTRY)
¶
Add a recorder to the given registry under a new name.
The name is also the recorder's persisted identifier: it ends the
recorder's snapshot filenames and is recorded in ditto.lock.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Name to register the recorder under, following the recorder name
grammar ( |
required |
recorder
|
Recorder
|
The recorder instance to register. |
required |
registry
|
RecorderRegistry
|
Registry to add to. Defaults to the shared |
RECORDER_REGISTRY
|
Raises:
| Type | Description |
|---|---|
DittoRecorderConflictError
|
If |
Source code in ditto/recorders/__init__.py
get(name, registry=RECORDER_REGISTRY, fallback=_MISSING)
¶
Look up a recorder by name.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Key to look up in the registry. |
required |
registry
|
Mapping[str, Recorder]
|
Registry to query. Defaults to the shared |
RECORDER_REGISTRY
|
fallback
|
Recorder
|
Recorder to return when |
_MISSING
|
Returns:
| Type | Description |
|---|---|
Recorder
|
The registered recorder, or an explicitly supplied |