Toilville/forge-kv-cache
Watch
5
0
Fork
You've already forked forge-kv-cache
 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions
Tiered (hot LRU + cold on-disk) content-addressed KV block cache with an append-only lineage ledger, for on-device MLX inference in Toilville Rituals
25 commits 1 branch 6 tags 75 KiB
  • Rust 100%
Find a file
Exact
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Peter Swimm 90339bc403
All checks were successful
CI / lint-workflows (push) Successful in 18s
Details
CI / Core (default features, no MLX) (push) Successful in 38s
Details
CI / MLX feature (macOS + Apple Silicon) (push) Successful in 30m44s
Details
Merge pull request 'fix(ci): stop the MLX job's PATH step failing on Apple Silicon' (#11) from fix/mlx-ci-path-step-exit-status into main 
Reviewed-on: #11
2026-10-10 21:28:45 +00:00
.forgejo fix(ci): stop the MLX job's PATH step failing on Apple Silicon 2026-10-10 14:22:13 -07:00
src fix: match LineageEvent to the real contract already in ritual_engine/src/api.rs 2026-07-16 10:46:24 -07:00
tests fix: match LineageEvent to the real contract already in ritual_engine/src/api.rs 2026-07-16 10:46:24 -07:00
.gitignore feat: real TieredBlockCache implementation, replacing the placeholder 2026-07-16 10:31:03 -07:00
.yamllint.yml feat: real TieredBlockCache implementation, replacing the placeholder 2026-07-16 10:31:03 -07:00
Cargo.toml chore: license under Apache-2.0 and fetch mlx-rs over public https 2026-10-10 13:44:51 -07:00
CONTRIBUTING.md docs: adopt org-wide contributor-review policy (v1) 2026-08-22 14:13:05 -07:00
LICENSE chore: license under Apache-2.0 and fetch mlx-rs over public https 2026-10-10 13:44:51 -07:00
README.md feat: real TieredBlockCache implementation, replacing the placeholder 2026-07-16 10:31:03 -07:00

README.md

forge-kv-cache

Tiered (hot LRU + cold on-disk) content-addressed KV block cache with an append-only lineage ledger. Blocks are keyed by sha256(model_hash || position_range), so identical content always resolves to the same block id — that makes TieredBlockCache::pull_from a trivial, conflict-free set union.

Consumed by (in Toilville_Rituals):

  • packages/ritual_engine (Cargo.toml — forge-kv-cache, feature mlx)
  • packages/ritual_engine/bridge-cache-kv — thin async-trait wrapper over eviction::EvictionPolicy, lineage::LineageEvent, store::{CacheConfig, TieredBlockCache}, traits::{BlockCache, BlockSpec}
  • packages/ritual_engine/bridge-inference-mlx and forge-kv-mlx — via the mlx feature's mlx_cache::ForgeKvSession, which bridges mlx-lm's synchronous KeyValueCache::update_and_fetch to this crate's async allocate path over a background channel.

History

This repo was recreated 2026-07-16. Consumers had depended on ssh://git@git.toilville.dev:2222/Toilville/forge-kv-cache.git (tag v0.1.0) for some time — proven by the full dependency graph still recorded in Toilville_Rituals/packages/ritual_engine/Cargo.lock — but the repo itself had never actually been pushed to this Forgejo instance. Two downstream projects (Toilville/forge-services and a spel-archive worktree) had each carried a no-op stub as a workaround in the meantime.

Feature flags

  • mlx — macOS + Apple Silicon only. Enables mlx_cache::ForgeKvSession, which depends on mlx-rs (built from source via CMake + the Metal Toolchain on first build).

Testing

cargo test                # core store — allocate/restore/evict/batch/pull
cargo test --features mlx # + ForgeKvSession session bridging