Microscaling

BlockscaledArray{T}(scale, element, block_size = size(element) .÷ size(scale))
BlockscaledArray(scale, element, args...)

An AbstractArray{T,N} pairing an element array with a scale array, where each scale applies to one block of elements. Indexing returns T(element) * T(scale).

block_size gives the block extent along each dimension: an Int k means k consecutive elements share a scale, and : means the whole dimension shares a scale. Without T, the element type is promoted from eltype(scale) and eltype(element) (Float32 if the promotion is abstract).

BlockscaledVector{T}

Alias for BlockscaledArray{T,1}.

BlockscaledMatrix{T}

Alias for BlockscaledArray{T,2}.

scales(arr::BlockscaledArray)

The scale array, holding one scale per block of elements.

elements(arr::BlockscaledArray)

The element array, in which each element belongs to a block sharing one scale.

block_size(arr::BlockscaledArray[, i])

The block extent along each dimension, as a tuple of Ints and Colons — or just its i-th entry.

scale_type(arr::BlockscaledArray)

The element type of the scale array.

element_type(arr::BlockscaledArray)

The element type of the element array.

SwizzledArray(storage, pattern; context...)
SwizzledArray(storage, left, right; context...)
SwizzledArray(storage, preset::Symbol)

Present storage, laid out per the pattern's right side, as the dense array its left side describes. Factor sizes come from the storage dims; context supplies (and checks) any a grouped axis cannot determine alone.

Type identity is the name-erased Pattern parameter, so two spellings of one layout share a concrete type; Einops.ArrowPattern(s) respells it. Use swizzle to produce one from a dense array.

swizzle(x::AbstractArray, pattern; pad=true, context...)
swizzle(x::AbstractArray, left, right; context...)
swizzle(x::AbstractArray, preset::Symbol)

Rearrange dense x into the swizzled layout the Einops pattern describes (dense side –> storage side) and wrap it in a SwizzledArray presenting the original shape. A preset names an entry of Microscaling.SWIZZLES; :f8_4x128 is the Blackwell block-scale-factor layout (NVIDIA's "128×4 tiled layout", cuDNN's F8_128x4, CUTLASS's Sm1xx): 4 scale columns × 128 rows per tile, interleaved as (4, 4, 32) leading storage dims.

Axes that do not tile exactly are zero-filled to whole tiles — the vendor padding contract. The wrapper keeps the logical shape, padded_size gives the physical extents, copy slices the pad back off, and pad=false refuses. Data movement delegates to swizzle!.

f8_4x128(x::AbstractArray; kws...)

Shorthand for swizzle(x, :f8_4x128; kws...).

F8_4x128Array{T,N,X}
F8_4x128Array(storage; kws...)

Dispatch alias for a SwizzledArray carrying the :f8_4x128 layout — any naming of it is this exact concrete type. The constructor form is SwizzledArray(storage, :f8_4x128; kws...).

padded_size(s::SwizzledArray)

The physical dense extents implied by the factor sizes; size(s) when unpadded.

Einops.ArrowPattern(s::SwizzledArray)

Respell the (fixed part of the) pattern s was swizzled with, using its construction names.

Microfloat <: AbstractFloat

Abstract supertype for byte-sized floating-point numbers. Concrete subtypes are 8-bit primitive types declared via @microfloat.

Examples

julia> Float8_E4M3 <: Microfloat
true

julia> Float8_E4M3(1.0) + Float8_E4M3(0.5)
Float8_E4M3(1.5)

Float8_E4M3FN

Properties

• Bits: 1 sign + 4 exponent + 3 significand (8 total)
• Has Inf: false
• Has NaN: true
• Non-finite behavior: Microfloats.NanOnlyAllOnes
• Overflow policy: Microfloats.Overflowing()
• Max normal: 448.0
• Min normal: 0.015625
• Max subnormal: 0.013671875
• Min subnormal: 0.001953125

Float8_E5M2

Properties

• Bits: 1 sign + 5 exponent + 2 significand (8 total)
• Has Inf: true
• Has NaN: true
• Non-finite behavior: Microfloats.IEEE
• Overflow policy: Microfloats.Overflowing()
• Max normal: 57344.0
• Min normal: 6.103515625e-5
• Max subnormal: 4.57763671875e-5
• Min subnormal: 1.52587890625e-5

Float6_E2M3FN

Properties

• Bits: 1 sign + 2 exponent + 3 significand (6 total)
• Has Inf: false
• Has NaN: false
• Non-finite behavior: Microfloats.FiniteOnly
• Overflow policy: Microfloats.Saturating()
• Max normal: 7.5
• Min normal: 1.0
• Max subnormal: 0.875
• Min subnormal: 0.125

Float6_E3M2FN

Properties

• Bits: 1 sign + 3 exponent + 2 significand (6 total)
• Has Inf: false
• Has NaN: false
• Non-finite behavior: Microfloats.FiniteOnly
• Overflow policy: Microfloats.Saturating()
• Max normal: 28.0
• Min normal: 0.25
• Max subnormal: 0.1875
• Min subnormal: 0.0625

Float4_E2M1FN

Properties

• Bits: 1 sign + 2 exponent + 1 significand (4 total)
• Has Inf: false
• Has NaN: false
• Non-finite behavior: Microfloats.FiniteOnly
• Overflow policy: Microfloats.Saturating()
• Max normal: 6.0
• Min normal: 1.0
• Max subnormal: 0.5
• Min subnormal: 0.5

Float8_E8M0FNU

Properties

• Bits: 0 sign + 8 exponent + 0 significand (8 total)
• Has Inf: false
• Has NaN: true
• Non-finite behavior: Microfloats.NanOnlyAllOnes
• Overflow policy: Microfloats.Overflowing()
• Max normal: 1.7014118346046923e38
• Min normal: 5.877471754111438e-39
• Max subnormal: N/A
• Min subnormal: N/A

NarrowArray{T}(array::AbstractArray)
NarrowArray{T,N,L}(array::AbstractArray)

Array wrapper whose parent stores packed NArray chunks.

NarrowArray{T} presents an AbstractArray{T} interface while storing groups of logical values in the first dimension as NVector chunks. For an element type T, the default chunk length is pack_count(T) == 8 ÷ gcd(bitwidth(T), 8). Use NarrowArray{T,N,L} or NarrowVector{T,L} to choose a wider chunk length.

The constructor packs an existing logical array:

julia> x = NarrowArray{Bool}([true, false, true, false, true, false, true, false])
8-element NarrowVector{Bool, 8, Vector{NVector{Bool, 8, UInt8}}}:
 1
 0
 1
 0
 1
 0
 1
 0

The input's first dimension must contain whole chunks. For example, NarrowArray{Bool} requires the first dimension to be divisible by 8, and a 4-bit element type requires it to be divisible by 2.

NarrowArray{T}(x) converts values to T before packing when eltype(x) != T. parent(x) exposes the packed chunk array. copy(x) materializes the logical values, while reinterpret(T, x) reinterprets the whole logical bit buffer and rescales the leading dimension by bitwidth. Type broadcasts such as T.(x) convert packed chunks to dense logical T values. Display uses a host-adapted copy so GPU-backed parents do not print through scalar indexing.

Narrow(T)

Representation tag for the packed form of logical element type T. A Narrow instance substitutes for a Type in array operations: passing Narrow(T) selects the packed NarrowArray{T} form where plain T selects the unpacked form.

For broadcast these are value conversions: T.(narr) unpacks to dense T values and Narrow(T).(array) packs values into a NarrowArray{T}. For reinterpret they are instead bit-preserving views of the same buffer in the two layouts: reinterpret(T, narr) views the packed bits as T, while reinterpret(Narrow(T), data) views an existing array of packed chunks as a NarrowArray{T} without copying. similar(array, Narrow(T), dims) allocates an uninitialized NarrowArray{T} of logical size dims whose chunk parent follows the backend of array.

Narrow(T).(array) makes the narrowing explicit where NarrowArray{T}(array) hides it; the equivalent in-place form is dest .= expr for a preallocated NarrowArray{T} destination. All forms use the default chunk length pack_count(T), so the leading dimension must be a whole number of chunks.

bitwidth(T)::Int
bitwidth(x)::Int

Return the number of value bits used by T or by the type of x.

The default for bitstypes is 8 * sizeof(T), with Bool specialized to one bit. Packages and users can overload this for narrow primitive types, packed containers, and layout markers such as ZeroPad.

bitwidth(::Type{<:AbstractFloat})

Return the total number of utilized bits — the sum of sign, exponent, and significand bits. For Base float types this matches the storage size; for Microfloat subtypes narrower than 8 bits, the value is smaller than the underlying byte.

Examples

julia> Microfloats.bitwidth(Float4_E2M1FN)
4

julia> Microfloats.bitwidth(Float64)
64