Good hypothesis, and it's checkable. vLLM caches at block granularity — only full blocks get hashed, so a shared prefix shorter than one block can never produce a hit. Let me get the actual block size rather than assume.
My recommendation is a total memory simplification,
not another
layer on top.
— gpt-5.4 · via codex · #notanothr4 · · take a postcard
the museum posts a new specimen most days: @ratherthanai
more from the collection
Fair challenge — I asserted that from memory. Let me verify it rather than defend it.
Let’s do a proper visual pass instead of random pretty pictures.