Spaces

Space

The abstract supertype of allocation spaces — anywhere an uninitialized array can be materialized from. A space implements the one leaf method alloc(space, T, dims::Dims); everything else — Undef specs, NamedTuple specs, space-first outputs/checkpoints and frame-first scratch — reduces to it.

alloc(space::Space, T, dims)
alloc(space::Space, T, dims...)

Materialize the description of an array into space. A custom space implements the one leaf method alloc(space, T, dims::Dims).

Similar(x)

The GC-owned Space around an exemplar array: alloc against it is similar(x, T, dims) — same array family and device, lifetimes by scope. The exemplar is dispatched on, never read.

Eager-only: a carve goes through the device's allocator, so allocating from a Similar under an active graph capture throws CaptureViolation rather than silently recording a memory node into the graph. Under capture, use an Arena.

Arena(slab::AbstractVector{UInt8}; alignment = 256)

A bump allocator over one preallocated byte slab: alloc carves typed arrays from it, retract! and reset! reclaim them in bulk, and nothing tracks individual allocations. The arena never grows — allocating past the end of the slab throws — and identical allocation sequences yield identical addresses, which is what graph capture requires.

alignment (a power of two) is where each carve starts, relative to the slab's first byte; a carve additionally never starts below its element type's own alignment. Absolute alignment therefore also depends on the slab's base: CUDA allocators return 256-aligned bases (so the default meets cuBLASLt's workspace requirement), a Julia Vector's base is only 32/64, Mmap.mmap's is page-aligned.

Arenas are task-local, so they take no locks.

alloc(space::Arena, T, dims) -> AbstractArray{T}

Carve a dims-shaped array of bitstype T from the arena: round the offset up to the arena's alignment (never below T's own), take the next sizeof(T) * prod(dims) bytes as a typed array (see carve), advance the offset, update the watermark. Throws past the end of the slab. The offset advance is source state, like an RNG's — hence no !.

Mark

A snapshot of an Arena's offset, returned by mark and consumed by retract!. Carries the arena's epoch, so retracting to a mark taken before a reset! throws instead of silently corrupting newer allocations.

mark(arena) -> Mark

Snapshot the current offset. Does not change the arena.

retract!(arena, m::Mark)

Move the offset back to m: everything allocated since the mark was taken is garbage at once, however many carves that is. Retracting to the same mark twice with nothing allocated in between is a no-op. Throws if the mark is stale (taken before a reset!) or above the current offset.

reset!(arena)

Offset back to 0 — every carve is garbage — and the epoch advances, so existing marks become stale. The watermark survives.

watermark(arena) -> Int

The peak offset ever reached — the measured byte requirement of everything run against this arena. Survives reset!, so it accumulates over all passes of a warmup.

carve(slab, offset, T, dims) -> AbstractArray{T}

A dims-shaped T array over slab[offset+1 : offset+sizeof(T)*prod(dims)], aliasing the bytes — never copying. The arena's only function that touches memory, and its per-storage specialization point. The generic method is reshape(reinterpret(T, view(slab, …)), dims); a Vector{UInt8} slab gets a native Array via unsafe_wrap instead — faster to index, not GC-rooted: valid only while the arena holding the slab is alive.

withspace(f, space)

Call f() with space as the ambient space for its dynamic extent. Inside, ambientspace returns space, and every ambient method — alloc(T, dims...), an Allocating verb called without a leading space — uses it. A scratchspace block rebinds the ambient space to its frame, so the ambient space is always the innermost open frame. The explicit space-first forms remain the primitives; the ambient forms forward to them, so this is one semantics with two spellings. An ambient read is dynamically dispatched (scope is runtime data) — thread spaces explicitly where full inference matters.

Scoped values flow into spawned tasks, but an Arena is task-local by design: don't Threads.@spawn under withspace of an arena.

ambientspace() -> Space
ambientspace(default)

The ambient space bound by the nearest enclosing withspace or scratchspace block. The zero-arg form throws when no space is bound; ambientspace(default) returns default instead — the form for kernel space keyword defaults, which must also work unscoped:

space = ambientspace(Similar(X))

alloc(args...)

The ambient form: any alloc call without a leading space forwards to alloc(ambientspace(), args...) — every explicit spelling (T, dims, an Undef, a NamedTuple spec) has its ambient counterpart through this one method.

capturing(x) -> Bool

Whether the device backing x is currently recording a graph capture.

false for anything that cannot capture — every CPU array, and any backend without an extension answering otherwise. A backend extension adds a method on its array type (Pol.capturing(::CuArray)), so this is the one predicate the rest of the stack asks.

CaptureViolation

A buffer was materialized from a garbage-collected Space while the device was recording a graph capture.

The allocation does not fail, and that is the problem. A device allocator is stream-ordered, so allocating under capture is recorded into the graph as a memory node — capture succeeds, instantiation succeeds, and the first launch succeeds. The second launch fails:

ERROR: CUDA error: invalid argument (code 1, ERROR_INVALID_VALUE)

A graph holding an allocation node whose memory is not freed within the graph cannot be relaunched until that memory is freed — and replay is the whole reason the graph exists. So the error lands on exactly the operation the capture was built to perform, names nothing useful, and survives any smoke test that launches once. Freeing inside the captured region does not rescue it either: the free path errors during capture.

(A lesser hazard rides along: if the pool is exhausted, the allocator escalates to reclaim, which device-synchronizes — and synchronizing under capture invalidates the capture outright. That fires only under memory pressure, so it is the same class of bug, deferred further.)

The capture-safe space is an Arena: a carve is offset arithmetic over a slab allocated long before capture began, so it records nothing into the graph, and an identical allocation sequence yields identical addresses — which is what replay requires. Warm up eagerly, read the watermark, size the slab, then capture.

Warmth is not optional for the rest of the step either: a kernel's first launch loads its module, which capture forbids (ERROR_STREAM_CAPTURE_UNSUPPORTED). Run the step once eagerly before capturing it — the same pass that fills the watermark.

Reaching this error means a pass fell back to its eager default — a Similar space, or a scratchspace on one — inside a captured region. Pass an arena (or a Frame on one) instead.