Multi-part file cache rewrite

For a couple of years I worked inside the ONTAP filesystem on Linux, in C and C++, on the parts of the storage stack that run close enough to the hardware to be worth instrumenting carefully.

The change

ONTAP moved files toward a multi-part layout: instead of one contiguous extent for a whole file, a file is described by its metadata and its data lives in parts. That layout is what makes large files and high-concurrency access over them tractable, and it is not a change you can make without touching the cache.

I rewrote the filesystem caching layer to support multi-part files. Not adjusted it — rewrote it, because the existing cache assumed a one-to-one relationship between a file and the region being cached, and that assumption does not survive the new layout.

Why a cache rewrite is the hard part

Changing an on-disk format is the visible risk. The cache is where it actually bites, for reasons that are easy to miss until you hit them:

• The cache key is no longer the file. If a part can be cached and evicted independently of the file that references it, the key has to include enough of the part's identity to prevent two different parts colliding. Get the key wrong and you serve one file's data for another, which is silent corruption.
• Invalidation gets harder. Evicting a file used to evict its data. With parts, you need to know which cached entries a given file owns, or you leak cache entries that can never be invalidated.
• Concurrency multiplies. Concurrent read and write across 10K+ files simultaneously means the cache is being asked to stay coherent under concurrent access to a structure that is already shared. Lock ordering becomes a correctness concern, not a performance one.

The performance win was better data retrieval and better resource utilization across storage operations. But the part I would put in an interview is the correctness argument: what the cache key must include, and why getting it slightly wrong produces a system that passes its tests and returns the wrong bytes under concurrency.

What I want to be precise about

I contributed to the multi-part file architecture, which supports concurrent read and write across 10K+ files. I owned the cache design and implementation within it. I did not own the overall architecture — other engineers did, and the scope of a filesystem format migration is not something one person designs end to end.

I am being explicit because "designed the multi-part file system" is a claim that invites "walk me through the RFC," and I would rather state the part I can defend in detail than the part that sounds better.