bugfix avail_size() overestimation and storage_cache_evicted_req#25

Open

shermanjlim wants to merge 1 commit intocasys-kaist:mainfrom

shermanjlim:bugfix-prefixcache

shermanjlim commented Apr 10, 2026

Some bugfixes:

In avail_size(): RadixCache's avail_size() already returns the result in bytes, so there's no need to multiply by self._bytes_per_token.
In storage_cache_evicted_req(): we should be using second_tier_prefix_cache instead of npu_prefix_cache


          bugfix avail_size() overestimation and storage_cache_evicted_req

c48e322

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet