MemoryLayouts.jl 🧠⚡

Optimize your memory layout for maximum cache efficiency.

Documentation for MemoryLayouts.

🚀 The Problem vs. The Solution

Standard collections in Julia (Dicts, Arrays of Arrays, structs) often scatter data across memory, causing frequent cache misses. MemoryLayouts.jl packs this data into contiguous blocks.

🔮 How it works

🛠️ Usage

The package provides the exported functions layout, deeplayout, layout!, withlayout, layoutstats, deeplayoutstats, visualizelayout and deepvisualizelayout. The distinction between layout and deeplayout is that layout only applies to top level objects, whereas deeplayout applies to objects at all levels. The two examples below demonstrate their use. As for the stats functions, these just do a dry run and print out some statistics on the degree of contiguity improvement a user can expect to see. The visualize functions provide a graphical representation of the memory layout in the terminal.

💡 Example for layout

The example below demonstrates how to use layout.

using MemoryLayouts, BenchmarkTools, StyledStrings

function original( A = 10_000, L = 100, S = 5000 )
    x = Vector{Vector{Float64}}( undef, A )
    s = Vector{Vector{Float64}}( undef, A )
    for i ∈ 1:A
        x[i] = rand( L )
        s[i] = rand( S )
    end
    return x
end

function computeme( X )
    Σ = 0.0
    for x ∈ X 
        Σ += x[5] 
    end
    return Σ
end

print( styled"{(fg=0xff9999):original}: " ); @btime computeme( X ) setup=( X = original(); );
print( styled"{(fg=0x99ff99):layout}: " ); @btime computeme( X ) setup=( X = layout( original() ); );
;

💡 Example for deeplayout

The example below illustrates the use of deeplayout.

using MemoryLayouts, BenchmarkTools, StyledStrings


struct 𝒮{X,Y,Z}
    x :: X
    y :: Y 
    z :: Z
end


function original( A = 10_000, L = 100, S = 5000 )
    x = Vector{Vector{Float64}}( undef, A )
    s = Vector{Vector{Float64}}( undef, A )
    for i ∈ 1:A
        x[i] = rand( L )
        s[i] = rand( S )
    end
    return 𝒮( [ x[i] for i ∈ 1:div( A, 3 ) ], [ x[i] for i ∈ div( A, 3 )+1:div( 2*A, 3 ) ], [ x[i] for i ∈ div( 2*A, 3 )+1:A ] )
end

function computeme( X )
    Σ = 0.0
    for x ∈ X.x  
        Σ += x[5] 
    end
    for y ∈ X.y 
        Σ += y[37]
    end
    for z ∈ X.z 
        Σ += z[5] 
    end
    return Σ
end

println( layoutstats( original() ) )
println( visualizelayout( original() ) )


println( deeplayoutstats( original() ) )
println( deepvisualizelayout( original() ) )


print( styled"{(fg=0xff9999):original}: " ); @btime computeme( X ) setup=( X = original(); );
print( styled"{(fg=0x99ff99):layout}: " ); @btime computeme( X ) setup=( X = layout( original() ); );
print( styled"{(fg=0x9999ff):deeplayout}: " ); @btime computeme( X ) setup=( X = deeplayout( original() ); );
;

📊 Dry Run / Statistics

You can inspect the potential improvements in memory contiguity without performing the actual allocation using layoutstats and deeplayoutstats or visually by using visualizelayout and deepvisualizelayout.

using MemoryLayouts

data = [ rand( 10 ) for _ in 1:5 ];

layoutstats( data )

visualizelayout( data )

Memory Layout Visualization
  Span: 640 b
  Min : 0x7f918c0a3580 (leftmost point)
  Max : 0x7f918c0a3800 (rightmost point)
  Scale: 8 b / char
██████████░░░░██████████░░░░██████████░░░░░░░░░░░░░░░░░░██████████░░░░██████████

The output indicates:

• packed: The total size (in bytes) of the data if packed.
• blocks: The number of individual arrays identified.
• span: The current distance between the minimum and maximum memory addresses of the data.
• reduction: The potential reduction in memory span.

🤝 Compatibility

• MemoryLayouts.jl is compatible with
  • AxisKeys
  • InlineStrings
  • NamedDims.jl
  • OffsetArrays
• this assumes that those packages are loaded by the user (weak dependences)

⚡ SIMD Alignment

Both layout and deeplayout accept an optional alignment keyword argument (default 1). This allows you to specify the byte alignment for the start of each array in the contiguous memory block.

Proper memory alignment is relevant for maximizing performance with SIMD (Single Instruction, Multiple Data) instructions (e.g., AVX2, AVX-512). On the other hand, such alignment leaves gaps between blocks of memory that are not a multiple of 64 bytes in length.

• AVX2 typically requires 32-byte alignment.
• AVX-512 typically requires 64-byte alignment.

💡 Example

using MemoryLayouts
struct MyData
    a::Vector{Float64}
    b::Vector{Float64}
end

data = MyData( rand( 100 ), rand( 100 ) )

aligneddata = layout( data; alignment = 64 )

pointer( aligneddata.a ) # Will be a multiple of 64
pointer( aligneddata.b ) # Will be a multiple of 64

🔒 Scoped Layout Handles

Use withlayout to automatically manage backing memory. All calls to layout, layout!, and deeplayout inside the block (that do not pass an explicit handle) use a temporary LayoutHandle that is released automatically when the block exits (or throws):

result = withlayout() do
    x = deeplayout( a )
    y = deeplayout( b )
    compute( x, y )
end

This is the preferred way to manage memory: no explicit release! call is needed, and there is no risk of forgetting to free the backing memory.

🏈️ Performance Mode (Live Dangerously)

By default, MemoryLayouts performs checks to ensure robustness:

1. Cycle Detection: Prevents StackOverflowError if your data structure has cycles (e.g. A -> B -> A).
2. Aliasing Warnings: Warns if multiple fields point to the same array (which MemoryLayouts will duplicate, breaking the shared reference).

If you are confident your data is acyclic and you don't care about shared references (or know you don't have them), you can disable these checks for a small performance boost by setting livedangerously = true.

# Faster, but crashes on cycles!
fast_result = deeplayout( huge_tree; livedangerously = true )

⚠️ Things to be mindful of

🔇 Suppressing the Banner

You can suppress the startup banner by setting the environment variable MEMORYLAYOUTS to "false" or "no".

export MEMORYLAYOUTS="false"

📖 Function documentation

layout(s; exclude = Symbol[], alignment :: Int = 1, livedangerously :: Bool = false)

deeplayout( x; exclude = Symbol[], alignment :: Int = 1, livedangerously :: Bool = false )

layout!( s :: AbstractDict; exclude = Symbol[], alignment :: Int = 1, livedangerously :: Bool = false )

withlayout( f :: Function )

result = withlayout() do
    x = deeplayout( a )
    y = deeplayout( b )
    compute( x, y )
end

layoutstats( s; exclude = Symbol[], alignment :: Int = 1 )

deeplayoutstats( x; exclude = Symbol[], alignment :: Int = 1 )

visualizelayout( x; exclude = Symbol[], width = 80 )

deepvisualizelayout( x; exclude = Symbol[], width = 80 )