Recovering 4.1 GiB of RAM on NVIDIA DGX Spark
Wouldn't you like your KV pool to grow from 262,144 to 937,984 tokens—3.58× the capacity—with about 4 GiB more memory per host?
Read article$ ls notes/
Blog
Experiments, measured results, and implementation notes.