Adaptive Prefetcher on top of different cache-types

[raj-prince]

Hi @martindurant,

Creating this issues to take an alignment before making any changes to move prefetcher logic (from gcsfs) to filesystem_spec repo:

1. Do you want to keep prefetcher as a new cache-type or want to keep over the existing caches?
2. If on top of the caches, do you have any idea about the end state or how you want to go about that?

[martindurant]

I like the clear demarcation of a separate cache_type; but since the refetcher is supposed to work well with cache types other than readahead (or none!), we do not want a whole set of new types. That leaves two options:

• alter the caches themselves to accept a new parameter in which the prefetcher backend can be used (this is essentially what gcsfs does now)
• make a "prefetcher" type, in which the type of the regular cache is a further option

The first option may be tricky to implement without intrusion into the s3fs/adlfs code. The latter option is sort of wrong-way-around from the gcsfs mental model ("readahead with prefetcher"), but I don't think that's important. If readahead is the only option we want to support, at least at first, then the distinction doesn't matter.

[raj-prince]

Thanks Martin, was thinking more about this integration, and I feel a separate cache-type is better design than keeping prefetcher as data backend. While I am discussion internally, would like to know if you have any concern on this? Please find the detailed proposal below.

Problem

While keeping the prefetcher as a data backend can boost sequential read performance across cache types, separating prefetching and caching layers is not ideal. The prefetcher and the various fsspec caches each maintain their own buffer, leading to double buffering and extra hops from prefetcher buffer to cache buffer in the data serving path. Although this setup may improve sequential reads, it cannot outperform using the prefetcher on its own. A better approach is to integrate caching and prefetching so that prefetching takes place directly on the cache buffer, similar to how the kernel handles readahead on the page cache.

Even so, this integration could still provide performance improvements for existing caches. To evaluate this approach, let’s examine two primary cache candidates that might benefit from using a prefetcher as a backend:

• Mmapcache: Sequential reads using mmapcache show performance gains. However, mmapcache is built for sparse file caching and, in sequential read scenarios, can consume significant system memory if available. When memory is limited and the mmap-backed tmpfile resides on slow storage, sequential read performance can deteriorate, as data flows from the prefetcher to the mapped tmpfile before reaching the application. In these cases, using the prefetcher alone yields better performance. Therefore, combining mmapcache with the prefetcher is not ideal—it’s resource-intensive and may degrade performance on low-memory systems.
• Blockcache: Similarly, we see substantial performance improvements for sequential reads with a prefetcher, but not beyond what the prefetcher alone achieves (IO size >= block size). This limitation stems from additional hops in the data path: prefetcher memory to cache memory to application. While it might seem that combining cache and prefetcher could help with sequential reads involving back-seeks, the separate buffers prevent this—prefetched data is not retained in the cache, so cache hits are unlikely unless applications repeatedly request the same data. Thus, using cache plus prefetcher is not optimal for sequential reads; the prefetcher alone is preferable.

Beyond double-buffering concerns, enabling the prefetcher for all the caches adds design/user complexity and increases maintenance effort to ensure the prefetcher works well with all the caches. Only a few caches—primarily “first” and “blockcache”—can take advantage of the prefetcher as a backend. Making the prefetcher available for all the caches, regardless of use case, risks complicating future development and makes it difficult to simplify or roll back changes later.

Proposal

The proposal is to introduce the prefetcher as a new cache type, “adaptive-readahead”, and set it as the default cache for gcsfs. Intentionally, I do not suggest replacing the current “readahead” cache implementation with the prefetcher one, so that the behavior remains unchanged for other file systems (like s3fs) and for existing users who explicitly enabled the “readahead” cache.

This approach extends the existing fsspec cache design, simplifying both implementation and user experience. By integrating the new cache directly into fsspec and making it the default for gcsfs, users will benefit from prefetcher improvements out of the box:

• Significant boost to sequential read speed.
• Lower egress for fully random-read workloads, since the adaptive-readahead cache only fetches the requested range from GCS instead of extra data. Some users may experience a regression if their application reads random ranges in multiple calls, but they can revert to the previous readahead cache if needed or “adaptive-readahead” cache itself improved to owner those scenarios.

Also, from the usability perspective, It’s better to improve the default experience for most users, rather than complicate the design to address niche scenarios.

Trade-off

• For use cases that require both caching and prefetching, using the prefetcher as a backend to the blockcache offers limited support and is not optimal. Because blockcache and prefetcher do not share a buffer, prefetched data remains inaccessible to the cache. The ideal solution would mimic the kernel’s read algorithm, with readahead occurring directly on the cache buffer to ensure prefetched data is retained and reused. Implementing this in userspace would be a substantial effort that requires implementation similar to VFS caching layer. Incremental improvements to the adaptive-readahead cache can gradually move us toward the optimal approach which can be further enhanced to reduce data duplication by enabling shared url (object) cache within a process or across multiple processes.
• Caches like “first” do not benefit from prefetchers, as their small buffer size avoids double buffering. The intended use case for “first” is uncertain—particularly for applications needing header caching plus sequential reads. In these scenarios, switching to the new default cache is recommended for better performance. With the improved adaptive-readahead implementation (with LRU based caching), frequent headers will remain in the LRU cache, reducing the need for this specialised cache.

Result

fsspec/gcsfs#795 (comment)

[raj-prince]

PR: #2093