Runtime, results, and persistence

Solving¶

model.solve(params=..., log_level=...) returns one immutable SolutionResult for every built-in or external solver. Its values store is indexed as period -> regime -> value array; replay and collective-dissolution data stay in the addressed artifact stores instead of changing the return type.

Optional arguments:

• max_compilation_workers caps parallel XLA compilation;

• log_path and log_keep_n_latest control diagnostic snapshots;

• retention selects which post-solve artifacts remain available.

• period_capture records selected production entries and completed references.

Hardware-local controls are declared on the model instead, through Model(execution_config=...), so both phases run under one resolved configuration.

There are no flag-selected tuple returns. Pass the complete result to model.simulate(solution=...); omitting solution asks simulation to solve first.

For models whose callables cannot be fingerprinted, set Model(..., durable_identity=False) to solve and simulate locally with the same model instance in the same process. This skips callable fingerprinting and binding-seal checks. Keep callable code, dependencies, and captured values stable throughout the session. Solutions and simulation archives from this mode cannot be saved; period capture/replay and component jobs require the default durable_identity=True.

Capturing and replaying one production period¶

PeriodCapture from lcm.persistence selects ordinary, JIT-compiled GridSearch periods in a normal full solve. Each target receives an atomic entry.h5 before execution. A separate atomic completed.h5 binds the completed value, nonfinite masks and synchronized dispatch time to that entry. The archives contain numerical arrays and JSON metadata, without a pickled model, callable or executable.

Launch the following example in a fresh process with JAX_ENABLE_COMPILATION_CACHE=false set in its environment before importing JAX or lcm.

from pathlib import Path

from lcm import Model
from lcm.persistence import PeriodCapture, load_period_capture
from lcm_examples.iskhakov_et_al_2017 import RegimeId, get_model, get_params

source = {"model": "my-recorded-model-revision"}
params = get_params(n_periods=3)
capture = PeriodCapture(
    directory=Path("period-captures"),
    periods=(("working_life", 0),),
    source_identity=source,
)
template = get_model(n_periods=3)
solution = template.solve(params=params, log_level="off", period_capture=capture)
directory = capture.directory / "working_life@0"
record = load_period_capture(directory=directory)
fresh = Model(
    regimes=template.user_regimes,
    ages=template.ages,
    regime_id_class=RegimeId,
    edges=template.edges,
    initial_nodes=((template.ages.exact_values[0], "working_life"),),
)
replay = fresh.replay_period(directory=directory, params=params, source_identity=source)
assert record.completed and replay.reference_matches

The fresh model independently binds its model and grid fingerprint and canonical solution-parameter fingerprint. Caller-supplied source identities supplement the installed Python source digest, native build and library hashes, JAX/JAXlib versions, precision, devices and resolved execution configuration. Incompatible identities are refused before compilation. Replay compiles and runs only the selected period, with the recorded widths, mesh, input/output placements and transfer plan. It checks the canonical optimized HLO, compiler reservation and recorded admission budget before dispatch. HLO canonicalization normalizes backend JSON member order while retaining its configuration values, constants and layouts. Existing target directories are refused rather than overwritten.

Backend JSON numbers retain their exact tokens, without floating-point conversion. Duplicate members, nonstandard constants and malformed JSON are rejected. Array order and all configuration values remain part of the HLO identity.

GPU capture and replay qualify nonempty numeric outputs only, and require every compiler memory record to report positive output allocation and positive raw peak. Zero-output records are unsupported, including layouts with empty device outputs. A cached executable can expose its HLO and positive allocation sizes while reporting a zero raw peak. That inconsistent report is refused before entry publication or replay dispatch; the peak is never synthesized from allocation sizes.

GPU capture and replay also require a nonempty memory_analysis().serialized_buffer_assignment_proto from each actual compiled core. Missing analysis or missing/empty serialized metadata is refused before entry publication or selected-period replay dispatch. Positive numeric statistics do not replace this requirement. Missing metadata does not by itself identify a persistent-cache hit: a runtime can omit it even after fresh compilation.

For acquisition and replay, start a fresh process with JAX_ENABLE_COMPILATION_CACHE=false before importing JAX or pylcm, and retain compiler debug/HLO metadata. Those settings do not guarantee that the runtime exposes the required metadata. A runtime that omits it is unsupported for GPU public capture/replay. The library never changes cache settings; the actual executable evidence determines acceptance, together with all strict recorded layout and admission checks.

record.metadata exposes identities, input shapes/dtypes, layouts, widths, optimized HLO text and hashes, and per-core compiler reservation, raw peak and external production residency. record.reference exposes the exact completed host array; its NaN and signed-infinity masks are verified against the archived masks. replay.reference_matches compares exact bytes and those masks. The recorded and replay dispatch times both include output synchronization and exclude serialization, loading and compilation. Their difference remains a measurement to inspect; timing equivalence or a speedup is not implied by numerical equality.

If interrupted after entry publication, record.completed is false. Replay then requires explicit require_reference=False, and returns reference_matches=None and in_context_seconds=None. Those inputs cannot establish reference parity.

Admission preserves the production planner’s represented-allocation charge: the same compiler reservation plus the recorded external residency must fit the same effective budget and headroom. It does not recreate unrelated production buffers, allocator state, cache pressure or physical peak memory. The initial public route excludes invariant blocks, donated inputs, continuation payloads, collective/gated kernels and selected artifacts. Capture never changes the model’s economic support, horizon, precision or result retention to make a route fit. Private environment-selected pickle captures retain their separate legacy format.

Execution configuration¶

ExecutionConfig is a Model(...) argument, so the devices, budget, and widths a model runs under are fixed when it is built and both solve() and simulate() read the same resolved values. None of them enters the model’s durable fingerprint:

from lcm import ExecutionConfig, Model

model = Model(
    regimes=regimes,
    ages=ages,
    regime_id_class=RegimeId,
    edges=edges,
    initial_nodes=initial_nodes,
    execution_config=ExecutionConfig(
        device_memory_bytes=40 * 2**30,
        sharded_states=("type",),
        axis_widths={"action_product": 8},
        devices=(0, 1),
    ),
)

Its fields:

• device_memory_bytes declares a per-device ceiling for compiler peak plus accounted live residency.

  • "device" (the default) derives the ceiling from the selected devices: the smallest reported allocator pool limit less device_memory_headroom_fraction. Where no selected device reports a limit, as on CPU, the model is unbudgeted. Routes that cannot be budgeted refuse it exactly as they refuse an integer. With a pool limit it also requires a preallocated pool, which is JAX’s default: with XLA_PYTHON_CLIENT_PREALLOCATE=false the BFC allocator grows the pool in separate regions, so the limit does not promise one contiguous block, and resolution refuses the default unless XLA_PYTHON_CLIENT_ALLOCATOR=cuda_async is set. Pass an integer or None to run on an on-demand pool. XLA_PYTHON_CLIENT_MEM_FRACTION sets the pool, and with it the default budget, for each process sharing a GPU.

  • None omits memory-budget admission; every omitted width takes its bootstrap width.

  • A positive integer is a ceiling every route must honour, and a route that cannot be budgeted refuses it. It may be the device’s whole allocator pool limit: the model admits against that limit less device_memory_headroom_fraction of it, whichever is smaller.

• device_memory_headroom_fraction (default 0.15) is the share of each selected device’s allocator pool kept out of the ceiling, an operational margin for storage outside the represented accounting — collective buffers, library workspaces, the driver context, fragmentation. It is a policy, not a measured requirement; 0.0 admits against the whole pool. Resolving a budget logs one line naming the request or the device default, the fraction, each selected device’s pool limit with the bytes the fraction keeps free, the effective ceiling, and whether the devices capped an explicit request. The line is a warning when they did and an informational line otherwise, so a cap is visible from log_level="warning" up and any other resolution from "progress". Every admission refusal names the effective ceiling and its source --- the request, the request capped by the headroom (with the requested bytes and the fraction that separates them), or the device default --- followed by the remedies: raise device_memory_bytes or lower the fraction, cap widths with axis_width_ceilings or axis_widths, shard over more devices with sharded_states or devices, or pass device_memory_bytes=None to disable admission at bootstrap widths.

• sharded_states names supported model-level states whose grid axes are spread over the devices their regimes are placed on. Sharding is resolved per regime: a regime whose DAG reads the state carries its axis and runs on the mesh that axis defines, while a regime that never reads it — so reachability prunes it there — publishes a value without the axis and runs on a single device. That holds for a non-terminal regime as much as for a terminal one, so a shock read through working life and dropped in retirement shards the working-life regimes alone. A state every regime prunes leaves no axis to spread and is refused. A continuous grid cannot be sharded merely because its extent divides the number of devices, and the continuous route additionally requires the state in every regime.

• axis_widths fixes the compiled width of one named planner axis, leaving the rest to the planner; see Fix a planner axis width. Every key must be an axis some core program declares.

• devices names the device ids the model may use, ascending; None (the default) means every device JAX reports. Every id must be one JAX reports.

• A device-memory budget, derived or explicit, selects the top-first outer-cohort planner for simulation regardless of whether axis_widths["subject"] is pinned; without a pin the planner resolves its own starting anchor. It changes simulation planning only and does not enter the economic fingerprint.

• donate_buffers is an exact Boolean, defaulting to True. False disables compiled solve input donation without changing the model fingerprint or economic inputs.

• invariant_block_widths names an invariant state to solve one code at a time, and invariant_block_schedule (an InvariantBlockSchedule) the order: PERIOD_MAJOR (the default) solves every code of a period before the next period; BLOCK_MAJOR solves, simulates and releases one code’s whole lifetime at a time, retaining its values on the host. See Solve one invariant code at a time.

NB-EGM can nominate one marginal leaf on an unsharded, self-carry main program. Donation requires exclusive solve ownership and a final, unretained read. A physical alias or unowned input selects an ordinary executable compiled at the same widths. With a memory budget, both variants must fit using their own compiler reservation and live residency. This conservative fallback does not use donation to admit a width that only the donating executable fits. Eager execution never donates. A selected replay program does not donate; standalone NB-EGM can select main even under the default retention when its policy is not applicable. Retained continuation leaves remain ineligible. This feature promises no particular workflow speedup or backend buffer reuse.

The ceiling applies to the selected compute devices. During GPU simulation, completed subject chunks can be transferred to CPU storage for final assembly. That host RAM is outside the GPU ceiling; it must be sized separately for the retained chunks and final result. GPU source buffers remain live until their transfer completes. During CPU simulation, retained chunks, concatenation and trimming occupy the budgeted CPU device and remain part of its accounted payload. CPU0 and GPU0 are distinct devices.

An unknown state, an unknown axis name, or an invisible device id raises ExecutionPlanningError at model build, naming the offender and the legal set. model.execution_devices reports the device ids the model resolved.

Compiler workspace budgets¶

Without a budget (device_memory_bytes=None, or the default on a device reporting no pool limit), every streamed axis is lowered at its bootstrap width — the largest power of two below the axis extent, capped at 64 for a reduced axis (whose block is purely temporary) and at up to 1024 for a tiled output axis (whose tiles concatenate into a full-size resident result) — or at the width ExecutionConfig(axis_widths={...}) fixes for that axis name, and compiler memory reports are not consulted. The whole axis is lowered only when a budget shows it fits or a fixed width asks for it. With a budget, the planner enumerates a deterministic width frontier for each streamed axis (one, the powers of two below the extent, and the full extent; a fixed width is the only candidate) and walks it widest-first — descending width product, ties broken toward the lexicographically largest width tuple in axis declaration order. A position whose resident bytes alone already reach the budget is refused before any candidate compiles, since no width could serve it. Otherwise each candidate is lowered and compiled, its complete compiler memory report is read, and the first candidate whose reservation plus resident bytes fits is dispatched. Solve residency includes retained values and continuation inputs, fixed model arrays, concurrent outputs and planned copies on their actual devices. Compiler-kept input metadata identifies which overlapping buffer spans are already represented in compiler allocation accounting. Eliminated inputs stay charged as external residency. Conservative reservations may count some storage twice.

When simulation solves automatically, its original and normalized inputs remain in the solve’s fixed inventory. Shared input and model buffers count once by physical storage; separate dtype-conversion and padding buffers remain charged alongside their originals. This accounts for their coexistence during solve planning, without admitting earlier template or conversion allocations retroactively.

This is an accounting convention using the backend’s compiler report; it is not a measurement of the allocator’s whole-call high-water mark. Compilation memory, non-pool allocations and remaining unprofiled orchestration allocations are outside a complete bound. Use a budget appropriate to the selected devices and measure the actual run when assessing capacity.

A narrower candidate is compiled only after every wider one exceeded the budget. That selects the same candidate an exhaustive search would, at the cost of one extra lowering per rejected width: a core whose full extent fits compiles exactly one candidate, and a core that fits at no width compiles its whole frontier before the error. Compilation is scheduled in waves across regime-period cells — every cell’s widest candidate first, then the next candidate of only those cells still over budget — so parallel compilation and the deduplication of identical lowerings are unchanged. A dense program has exactly one candidate.

Candidate measurement uses compilation, and admission fails closed:

• no candidate is executed to measure it; compiler-reported requirements are the planning signal, because the runtime high-water mark of a run that dies is a truncated underestimate;

• a budget that no candidate meets raises ExecutionPlanningError before backward induction starts, naming the regime, period, core, resident bytes, and budget — the smallest reservation and its raw peak too, once a width has actually been compiled;

• a budget, explicit or the device default, requires JIT compilation. Supplied solutions are supported; forward programs and profiled host operations recheck their reservations against the current retained solution, inputs and growing outputs before dispatch;

• a forward program without a profiled compiled implementation, including a host-driven route, is refused under any budget;

• the selected widths are execution choices: they enter neither the model nor the parameter fingerprint, and a period capture records them so replay_period lowers the same executable without planning again. The capture also records each array’s sharding and each core’s compiled placements as descriptors, without buffers: replay_period lowers against the restored arrays’ default placement, while replay_period_on_recorded_layout reinstates the recorded layout and lowers against the submesh the solve dispatched the period on.

For each complete device record, compiler admission reserves the larger of the raw reported peak and argument + output - alias + temporary bytes. The latter counts represented argument/output storage and preallocated temporary allocations. Alias bytes remove argument/output overlap once. Counters must be present, nonnegative integers; aliases must fit both categories. Each device is accounted for before taking the maximum. The raw peak stays separately identifiable in profiles and refusal messages.

The selected-device ceiling adds external retained residency to that reservation. Generated code, omitted constants and runtime workspace, thread stacks, executable caches, and allocator overhead remain outside the represented allocation contract. This computed reservation is not a measured peak or a complete runtime upper bound.

Reports with nonzero host allocation counters are refused because the compiler’s peak cannot reliably be split across default and host memory spaces. Generated-code metadata alone does not trigger this refusal. Existing GPU-to-CPU chunk offload continues to profile CPU assembly separately and excludes that CPU storage from the GPU ceiling. CPU execution charges its selected CPU storage. The reported CPU witness and scope are recorded in compiler allocation admission. Width determines the choice among candidates that satisfy the accounting contract.

Buffer release, donation and placement¶

Backward induction keeps a ledger of every array a core reads across a regime-period boundary. After a regime’s kernels have run, the engine:

• releases every input whose last reader has run and which the result does not keep — a gated-edge continuation, a continuation leaf — by deleting its device buffer once every output of the period so far is ready; a buffer two keys name (a continuation leaf that is also the published value) is kept while either key is still read;

• never releases a regime’s value: SolutionResult.values keeps every one on device.

An argument a core program names in donation_candidates is donated when every artifact it carries:

• is not the solve-lifetime template, the one input whose declared period lies beyond the model’s last period;

• has this dispatch as its sole remaining consumer;

• is not retained by the result;

• is not pinned by an undeclared reader;

• shares its buffer with no other key;

• reaches the core on its stored layout rather than as a transferred copy.

The executable is compiled with that argument donated, and the input is unreadable afterwards.

Under log_level="debug" every release and donation is logged with the artifact key and the dispatch that closed it.

On several devices every regime is placed before anything is compiled:

• a regime with a state named in ExecutionConfig.sharded_states runs on a mesh of as many devices as the largest divisor of that state’s extent that fits — a three-valued type on four devices runs on three;

• a regime without one — including a regime that prunes a state other regimes shard — runs on one device, taking a device the meshes leave idle in the periods it is active, or else the device with the smallest planned footprint;

• regimes of one period that read nothing of each other within the period and sit on disjoint devices are dispatched together.

Two placements of one model publish values that name the same real number: each partition is vectorized at its own width, so the two runs agree to within a few units in the last place rather than bit for bit. Simulation retains solution arrays on their stored devices and acquires each period’s declared value inputs on the devices required by its forward programs. Matching layouts pass through; other supported layouts use explicit copies, with source and destination storage admitted when a budget is supplied. A solution produced on a proper submesh can therefore be supplied to simulation. The original solution remains owned while the forward period tracks its acquired copies.

Solution results¶

model.solve(...) keeps values, metadata, replay artifacts, diagnostics, and explicit omission reasons in one SolutionResult:

from lcm.solver_api import ResultRetention

solution = model.solve(
    params=params,
    log_level="debug",
    retention=ResultRetention.VALUES_AND_REPLAY,
)

V_working = solution.value(period=0, regime="working")
values = solution.value_array(period=0, regime="working")
curve = values.sel(education="college")
frame = curve.to_dataframe().reset_index()
full_frame = solution.value_frame(period=0, regime="working")
codes = solution.value_frame(period=0, regime="working", use_categorical_codes=True)

value_array returns an xarray DataArray named V with one dimension per axis in metadata.value_schemas[(period, regime)].axis_names order. Axis names must be unique and cannot be V, which is reserved for the value. Discrete states carry their labels as a categorical coordinate in declaration order, with the ordering matching simulation; a state declared with values carries those numbers as a plain coordinate. use_categorical_codes=True keeps integer codes. Collective regimes also have a stakeholder dimension, and a regime without states yields a zero-dimensional array. Coordinates are the resolved grid nodes used by the solve, including parameter-dependent process grids and period-specialized grids, and remain available after saving and loading. The array wraps the stored value without copying it to the host, so select the slice you need before converting it to a table.

value_frame is the same array in long form: a pandas DataFrame with one column per dimension followed by V. It materializes every grid point, so the frame can be large. For values between nodes, use Model.lookup_policy, which uses the same interpolation as simulation. The coordinate-bearing value schema uses solution schema version 3; archives from a different schema version are refused by the existing compatibility gate.

The result and its supporting types live in the lightweight lcm.solver_api submodule; they are not re-exported from the top-level lcm namespace. The retention modes are:

Values live in a ValueStore; artifacts are addressed by an ArtifactRef(period=..., regime=..., key=...) and kept in immutable ArtifactStore instances. Both stores expose the same eager values after a solve and independently lazy entries after restoration. Inspecting coordinates, metadata, omissions, or load_state(...) does not load numerical data. After restoration, solution.value(...) and solution.values[period][regime] load only the requested value entry and verify its checksum first. Whole-store numerical traversal is explicit through solution.values.materialize(); per-coordinate access preserves independent laziness. For ordinary array artifacts, store[ref] does the same. A plugin-defined PyTree requires store.materialize(ref, template=...) (as used by model replay) so its non-executable archive leaves can be rebuilt safely. LoadState.UNLOADED is storage state, not an omission reason.

Both stores, and the omissions mapping of SolutionResult, admit a public mapping through exactly one item traversal. Every raw coordinate is checked before it is inserted anywhere: a ValueStore period must be an exact int and a regime name an exact str (so True is refused rather than merged into 1), an artifact or omission address must be an exact ArtifactRef, and a logical address that the traversal emits twice is refused instead of contracted. A mapping’s key view or length is never consulted, so a mapping whose keys disagree with its items cannot smuggle a payload past the checks. Nested period -> regime -> value and flat (period, regime) -> value forms are recognized from the same traversal.

Built-in key identities include SIMULATION_POLICY, EGM_CONTINUATION, and SOLVER_DIAGNOSTICS. The omissions mapping distinguishes an artifact that is not applicable, not requested, unsupported, or not persisted.

Metadata carries a durable SHA-256 model fingerprint, the exact digest of canonical parameters that can affect the solution, solver and replay-route identities, artifact descriptors, and the solution and solver-interface schema versions. The fingerprint covers the mathematical model and the facts needed to interpret stored arrays, including period/regime topology, state and action names, grid support and category order, solver/replay/artifact versions, numerical conventions, and those solution-relevant parameter values. It excludes execution-only details such as devices, JIT selection, tiling, sharding, and compiler versions, as well as parameters and callable semantics used exclusively by simulation-phase transitions. An array a declaration references — through a closure cell, a default, or a solution-relevant parameter — is hashed with the rank and shape it actually has, so a scalar array and a length-one vector holding the same bytes are different identities; its memory order is storage rather than identity and does not enter the digest. A model function wrapped by a beartype guard — pylcm’s own or one a downstream package’s claw installed — is fingerprinted through the guard: the guard is accepted as transparent only when beartype regenerates its code from the bound callee with the guard’s own configuration, and the callee is what enters the identity. A result that nothing references any more releases its value and artifact arrays: no per-call closure inside pylcm keeps them alive, which matters because the package’s beartype claw decorates every function definition and beartype retains each decorated function object for the life of the process. Separately, an in-memory result keeps its model-instance token; that same-instance check is not applied to a restored archive. metadata.source records that distinction as IN_MEMORY or PERSISTED.

Each (period, regime) value has a lightweight ValueArraySchema recording its exact shape, dtype, and canonical named axes. Artifact descriptors play the corresponding descriptive role for retained payloads. Neither authenticates returned data. Simulation rebuilds immutable authority from the canonical model, canonical parameters, and the installed consuming route, then checks values, repeated metadata, and materialized artifacts independently.

Solver numerical diagnostics follow runtime_checks, independently of retention. Each retained diagnostics payload is described by a model-verifiable descriptor the solve generates from the payload itself, so it is saved with the result and reads back from an archive without a model; a consuming model admits the descriptor after checking that it names only the published fields with the dtypes they carry. A continuation is always available to the backward graph that requires it, regardless of result retention. It remains in the returned result only under ALL_PERSISTABLE_ARTIFACTS and only when its model-built authority declares MODEL_VERIFIABLE; otherwise the result records NOT_REQUESTED or NOT_PERSISTED.

A retention also selects what a solve computes. Every built-in kernel publishes its programs with a scope and, for replay or additive artifact programs, exact retained_artifact_keys and an exact retained_artifact_payload_types entry for every key. All programs retaining the same key must agree on its type. The type describes the final artifact in the period kernel’s KernelOutput, after any composite or adapter transformation; it does not assert that the artifact is present in every invocation. This lets even an inapplicable omitted cell carry an exact descriptor. The solve compiles and runs only the programs selected for that period/regime cell. Each replay program explicitly names the values-only program it replaces, so selecting one replay artifact cannot suppress an unrelated values program in the same multi-core graph. VALUES runs the values-only programs everywhere, so DCEGM and NB-EGM publish their values and carries without assembling a policy, and a nested NNBEGM solve folds its candidates without building replay banks or the adaptive nested policy. A replay-retaining solve runs replay programs only where a declared replay route consumes them. ALL_PERSISTABLE_ARTIFACTS selects replay alternatives and additive artifact-only programs per exact model-authoritative artifact address; it does not widen a regime-level boolean. A standalone case-piece NB-EGM regime has no replay consumer, so it runs its values-only program under every retention and its policy is recorded as NOT_APPLICABLE. Values and carries agree across retentions to the working format’s spacing.

A values-only solve is the cheap way to obtain value functions from a case-piece or piecewise-affine budget model, for example to compare solvers or to sweep parameters:

from lcm.solver_api import ResultRetention

values_only = model.solve(
    params=params,
    log_level="warning",
    retention=ResultRetention.VALUES,
)

V_alive = values_only.value(period=0, regime="alive")

Such a result simulates only where every decision is recoverable from values; a model whose simulation reads an NB-EGM or NNBEGM policy needs the default retention before it can be simulated.

ALL_PERSISTABLE_ARTIFACTS keeps only artifacts whose model-built authority declares PersistencePolicy.MODEL_VERIFIABLE. The NNBEGM replay policy of an AdaptiveOuterMesh search is replayed against the exact mesh the solve generated. Those nodes are solution-owned data: the result carries them as the candidate axis of the policy’s descriptor, and a consuming model admits them after checking what a shared mesh must satisfy (exact finite floats, strictly increasing, within the search’s node budget and the outer state’s domain for that period) before it compares the rest of the descriptor against its own authority. The adaptive policy and the finite candidate bank of a FiniteOuterGrid search are therefore both retained under both modes and written to the complete archive. A built-in EGMCarry continuation is also model-verifiable and is retained only by ALL_PERSISTABLE_ARTIFACTS, making that mode strictly broader than replay-only retention for an EGM regime.

save_solution(solution=..., path=...) atomically writes the complete labelled result to a versioned archive; solution.save(path=...) is the equivalent convenience method. The archive contains JSON metadata and independently addressed numerical datasets with SHA-256 checksums. It contains no model, Python class, callable, pickle, or executable code. A result restored by load_solution can be saved again: its payloads are re-read from the archive it came from, verified against their checksums and descriptors, and written to the new archive without a model. See Standalone persistence for loading and version compatibility.

Simulation¶

model.simulate(...) accepts parameters, initial conditions, an optional complete SolutionResult as solution=..., a required log_level, and runtime_checks=True. Omitting solution solves first. Bare value mappings and separate policy or dissolution-flag inputs are not accepted.

How a SolutionResult is consumed follows its provenance. A result the same model instance solved in this process, for the same canonical parameters, is consumed by reference: the engine reads the arrays the solve allocated without copying or re-validating them, and only checks the replay payloads it is about to read. Every other result — restored from an archive, unpickled, or produced by another model instance — is validated in full and materialized into private buffers once per consuming model and parameter vector; a second simulation from the same result reuses that validated view. The consumed result stays reachable as SimulationResult.solution until the simulation result is saved.

Before consuming such a result, simulation checks its durable model and solution-parameter fingerprints, exact solver/plugin and replay-route identities, schema versions, period count, regime order, solver types, and exact active period/regime value coverage. An in-memory result additionally has to come from that model instance. It unconditionally checks every required value and its descriptive schema independently against the model-owned shape, canonical dtype, and named axes, including at log_level="off".

The model’s side of that identity is fixed when the model is built. Model(...) digests the structure once — topology, names, identities, and every declared callable’s semantics — and records each global and closure binding those callables read. Rebinding one of them afterwards, for instance by reassigning a module-level parameter a utility function closes over, would make the model run code its identity no longer describes, so solve() and simulate() refuse with ModelSealError naming the binding. Build a new model instead. A pickled model is resealed against its stored identity when it is loaded.

All artifact stores and omission records must address active result cells with the exact key version and channel; one reference cannot appear in multiple stores or be both present and omitted. Required lazy entries are materialized during preflight and checksum verified. Numerical leaves are copied into private owned buffers; a plugin-defined PyTree is rebuilt as a fresh exact tuple or structurally closed dataclass record from the model authority’s sealed construction plan, without another plugin flatten or unflatten callback. PyTree-represented static metadata is checked against the plan, while callback-injected instance state is replaced by its declared canonical value. The consuming route receives an owned replay snapshot rather than the caller’s container or array objects. Built-in EGM/NNBEGM policies and collective dissolution flags retain their specialized validation. An installed external route additionally validates solver-specific invariants and builds a JAX-transformable replay reader; this does not sandbox installed plugin code. No forward-simulation step runs until all required entries pass.

A values-only result is therefore sufficient for a model whose decisions are fully recoverable from values, but fails closed before forward simulation when a required replay artifact is absent or invalid. Use the default ResultRetention.VALUES_AND_REPLAY when the model may require such artifacts.

ExecutionConfig(axis_widths={"subject": width}) controls the model’s subject chunks and fixes their inner compiled width. The outer chunk is aligned to the selected subject devices; this alignment does not increase the fixed inner width. With a memory budget and no explicit width, complete chunk profiles select the widest fitting candidate. Retained full-population inputs and result assembly remain part of the bound even when chunks shrink. Each simulation takes its population from the initial conditions, so the same model accepts different call-time populations. seed controls random draws independently of those chunk boundaries. The ordinary seed is an integer, which becomes a Threefry key whatever jax_default_prng_impl says, or a scalar typed Threefry key; other implementations are refused, because their batched draws change with the chunk width. The independent taste_shock_seed draws per subject key and accepts an integer or a scalar typed key of any implementation. Caller key buffers join the budgeted entry inventory, and profiles distinguish the taste root’s actual PRNG implementation. A collective model may require an addressed dissolution replay artifact and own_stakeholder; see Collective regimes.

Under a device-memory budget, the top-first outer-cohort planner profiles the whole-population candidate first and descends only on refusal, using either the pinned subject width or an automatically resolved one as its starting anchor. Existing live storage and transfer checks remain active on every call. See Batch forward simulation for the search limits and memory scope.

Initial conditions are a mapping of state names plus regime_id to equal-length arrays, or a DataFrame with a regime_name column.

Validation and logging¶

log_level controls console output: off is silent, warning prints warnings, progress adds timing, and debug adds value statistics. runtime_checks=True independently enables numerical probability, NaN and existing feasibility/replay checks and raises on failure at every verbosity. runtime_checks=False explicitly skips those checks, including during automatic solving. Density derivative accuracy certification exists only as a test oracle; neither runtime-check setting runs it. Construction, parameter schema, initial-condition structure and solution-identity validation always run. Infinity-only value diagnostics remain warnings; an application requiring all stored values finite must retain that stronger invariant.

model.simulate(...) also reports the resolved execution plan it dispatched: the forward route ("legacy" or "subjects"), the ordered subject devices and their backend, the resolved planner axis widths by regime, the outer chunk count and admitted chunk widths, and the budget mode with its effective device-memory bytes. A one-line summary logs at log_level="progress" and "debug"; the complete record logs at "debug" only, and neither line appears at "warning" or "off". This is diagnostic only — it changes no numerical, RNG, ownership or admission decision — and exists so a clean production run carries evidence of which route actually engaged. The same record is available programmatically as SimulationResult.plan_summary.

pylcm enables a persistent JAX compilation cache by default. Set JAX_COMPILATION_CACHE_DIR to choose the full directory or LCM_COMPILATION_CACHE_NAME to choose the project-specific leaf. pylcm leaves device allocation to JAX, which preallocates; with XLA_PYTHON_CLIENT_PREALLOCATE=false pass an explicit device_memory_bytes or None, since the "device" default is refused there (see device_memory_bytes above).

PolicyLookup¶

Model.lookup_policy(params=..., solution=..., period=..., regime_name=..., states=..., action_grids=None, keep_intermediates=()) evaluates the decision that simulate takes at caller-supplied states of one demanded period and regime. It returns a PolicyLookup with actions, a mapping from action name to the chosen grid values, and value, the maximised value at each state. Continuous states off the grid extrapolate as in simulate; discrete states and restricted action_grids values must lie on their grids. Model.state_names gives the axis order of solution.values[period][regime], and Model.state_grid gives a state’s grid nodes after runtime parameters.

A regime that reads a shocks= group on the integrated current-IID route, that is a group no endogenous-grid regime reads, raises UnsupportedOperationError: its decision depends on a current shock, which needs an explicit observation or conditional query. See Combinations not yet composed with current shocks.

keep_intermediates names the intermediates of the decision to keep, in any order: the action values "Q", their feasibility "F", the flow utility "U" and the certainty-equivalent continuation "CE". Each kept one fills the field of that name over the (possibly restricted) action grid, with one leading row per state and one axis per action in the order of actions; the others stay None, as all four do by default. All come from the one kernel evaluation the decision maximizes over, and keeping them leaves actions and value unchanged. Collective regimes refuse the option.

Q, U and CE are raw: an infeasible entry holds what the action would have, so a Q entry can exceed value, and value is the max of Q over the entries F admits. Keep "F" alongside any of them to know which entries the decision admits. U is the regime’s flow utility. CE is the certainty equivalent of next period’s value, undiscounted: the CE argument the regime’s Koopmans aggregator receives, so Q == W(utility=U, CE=CE, ...) for every aggregator, and Q == U + discount_factor * CE under LinearAggregator. A terminal regime has CE == 0 and Q == U. When the aggregator reads a regime function, such as a type-specific discount factor, rebuilding Q from U and CE also needs that function’s value.

Each kept intermediate holds one float or bool per state row and action. Any non-empty keep_intermediates evaluates the whole action product at once, even where the decision alone streams over it, so it costs more memory than the arrays it returns. A lookup over a restricted grid compiles its own program, so its Q and value can differ in the last bits from the matching slice of a full-grid lookup’s Q.

In a regime with ExtremeValueTasteShocks the discrete choice is random, so actions is None and the lookup publishes the distribution of the discrete choice that simulate draws. For each state row and discrete-action combination, Qc is the max of Q over the feasible continuous actions (-inf without one), choice_probabilities is softmax(Qc / scale) over the feasible combinations (exactly 0.0 for an infeasible one), and conditional_actions maps each continuous action to its grid value at that max. All three have one leading row axis and one axis per discrete action of the (possibly restricted) grid. value is the smoothed maximum scale * logsumexp(Qc / scale) of those conditional values. A row without a feasible combination has value == -inf and zero probabilities, and its conditional actions hold the first grid value. These are published on every lookup of such a regime; keep_intermediates adds Q, F, U and CE over the action product as it does without taste shocks.

The lookup applies the solve’s reduction (mask, continuous max, logsum) to the decision simulate runs: current utility and the Koopmans aggregator take their simulate variants, and the continuation is the solved one. When both phases share those primitives and the action grid is the declared one, value equals the stored solve value at grid rows to numerical precision. With Phased current utility the two can differ, and the probabilities are those of the simulated choice.

To evaluate the distribution on the full state grid, take each state’s nodes from Model.state_grid in Model.state_names(regime_name=...) order, form their Cartesian product with numpy.meshgrid(*nodes, indexing="ij"), and pass each flattened mesh as that state’s array. Only GridSearch regimes support this lookup: an EGM-family solver’s simulation recomputes the choice on an action grid that can differ from its solve’s candidates, so the lookup refuses such a regime.

SimulationResult¶

to_dataframe(additional_targets=None, use_categorical_codes=False, terminal_rows="first") materializes a flat DataFrame. additional_targets accepts a sequence of selected DAG output names (including the tuple result.available_targets), "all", or None. Other bare strings raise InvalidAdditionalTargetsError. terminal_rows="all" retains every frozen absorbing row; the default keeps only terminal entry. Discrete columns are pandas Categoricals: a category class’s labels, or, for a grid declared with values, an ordered Categorical over those numbers that .astype(float) turns into numbers. use_categorical_codes=True gives integer codes instead.

Inspection properties include regime_names, state_names, action_names, n_periods, n_subjects, available_targets, raw_results, flat_params, and period_to_regime_to_V_arr. plan_summary carries the resolved execution plan the call reported (route, subject devices, resolved widths, chunking, budget mode); it is diagnostic only and is None for a result read back with load.

SimulationResult.save(directory=..., df_additional_targets=None, df_use_categorical_codes=False) writes array checkpoints, value functions, metadata, and a Feather table. The two df_* keywords are passed through to to_dataframe when projecting that table and select what it holds; they leave the other three artifacts unchanged. SimulationResult.load(directory=...) restores it; plan_summary (like solution) is not part of the saved archive.

Standalone persistence¶

• save_solution(solution=solution, path=path) writes a complete SolutionResult atomically. A failed write leaves no partially published replacement.

• load_solution(path=path) reads metadata and constructs independently lazy value and artifact stores. Every entry is checksum verified when materialized.

• load_solution(path=path, verify_checksums=True) verifies every payload without changing any entry from UNLOADED to LOADED.

• load_legacy_solution(path=path) explicitly reads the old value-only HDF5 format for migration. Legacy values have no durable model fingerprint, descriptors, omissions, or checksums and are not a complete replay result.

• SolveSnapshot and SimulateSnapshot describe diagnostic snapshots.

• load_snapshot(path, exclude=...) loads a snapshot, with optional components omitted.

The solution archive requires exact matches for its format, solution-schema, and public solver-interface versions. Replay additionally requires the matching plugin, route, and artifact-schema versions. pylcm reports incompatible versions rather than guessing a migration. Without an external plugin installed, load_solution can still read standard metadata and omissions, verify all checksums, and inspect values or array payloads lazily. A plugin-defined PyTree cannot be interpreted or replayed until the matching route supplies its model-authoritative template.

Workflow: Solving and simulating, DataFrame interoperability, and Debugging.