v0.8.0: faster SSD streaming without more memory - prompt prefetch into cache slots, predicted next-layer expert prefetch while decoding (helper thread, separate read queue), single-copy reads

ee52468

verified

ryugyosoft commited on 6 days ago