Skip to main content

Caching

Every query in gpui-query has a cache policy. The policy determines whether a fetch happens at all, and whether stale data gets served while a refetch runs in the background. Picking the right policy is the main lever you have for tuning how "live" your data feels versus how many network requests your app makes.

Cache policies​

There are three variants of CachePolicy. The default is Ttl { ttl_ms: 60_000 }, a 60-second TTL. You can override it per query or set a different default when constructing the QueryClient.

NoCache​

CachePolicy::NoCache

Every access triggers a fresh fetch. No cached data is stored or returned. Use this for data that changes constantly and where showing a stale value is worse than showing nothing.

The practical effect: begin_request always returns Started, never CacheHit.

Ttl​

CachePolicy::Ttl { ttl_ms: 30_000 }

Data is fresh for ttl_ms milliseconds after it was last fetched. While fresh, accessing the query returns CacheHit and skips the network entirely. After the TTL expires, the next access starts a new fetch.

This is the default policy. The built-in default TTL is 60 seconds (60,000 ms). For most read-heavy UIs this is a reasonable starting point.

StaleWhileRevalidate​

CachePolicy::StaleWhileRevalidate { ttl_ms: 30_000, stale_ms: 60_000 }

This policy splits the cache lifetime into two windows.

Fresh window (0 to ttl_ms): data is served from cache. No fetch happens. Same as Ttl.

Stale window (ttl_ms to ttl_ms + stale_ms): cached data is returned immediately, and a background refetch starts. The caller gets StaleCacheHit with a request_id for the background fetch.

Expired (past ttl_ms + stale_ms): the data is too old to serve. A normal fetch happens, same as a cache miss.

Use this when your UI benefits from showing slightly outdated data rather than flashing a loading spinner. Feed timelines, dashboards, and list views are common candidates.

How freshness is decided​

CachePolicy exposes several methods that control the caching decision internally. You normally do not call these yourself (the library does), but understanding them helps reason about behavior.

  • is_fresh(age_ms) - returns true when age_ms is within the TTL window
  • is_stale_but_serveable(age_ms) - returns true when the age is past TTL but within the stale window (only StaleWhileRevalidate)
  • is_expired(age_ms) - returns true when the age exceeds the total valid window. NoCache always returns true
  • can_short_circuit(age_ms) - whether the policy can skip a fetch entirely (true for Ttl and StaleWhileRevalidate when data is fresh)
  • can_serve_stale(age_ms) - whether stale data can be returned (true only for StaleWhileRevalidate)
  • total_valid_ms() - returns ttl_ms + stale_ms for SWR, ttl_ms for Ttl, None for NoCache

Setting the policy​

Pass the policy when creating a QueryClient to set the app-wide default.

cx.set_global(QueryClient::new(
    CachePolicy::StaleWhileRevalidate { ttl_ms: 30_000, stale_ms: 60_000 },
    RequestPolicy::LatestWins,
));

Override per query through QueryOptions:

let opts = QueryOptions::new(QueryKey::from(["notifications"]))
    .cache_policy(CachePolicy::NoCache);

Or use resource_with_policies on the client directly:

let entity = client.resource_with_policies::<Vec<Notification>, MyError>(
    QueryKey::from(["notifications"]),
    CachePolicy::NoCache,
    RequestPolicy::LatestWins,
    cx,
);

Request deduplication​

When multiple parts of your app request the same key at the same time, the RequestPolicy decides what happens.

LatestWins (default)​

A new request cancels the in-flight one and starts fresh. This is the right choice when you always want the most recent data and do not care about wasting the earlier request.

RequestPolicy::LatestWins

IgnoreWhileLoading​

If a request is already in progress, new requests for the same key are ignored. The original request runs to completion. Use this when fetches are idempotent and you want to avoid redundant work.

RequestPolicy::IgnoreWhileLoading

What happened: QueryBeginResult​

When you call begin_request on a resource, the return value tells you exactly what happened.

pub enum QueryBeginResult {
    Started { request_id, status, replaced_request_id },
    CacheHit,
    StaleCacheHit { request_id, status, replaced_request_id },
    IgnoredWhileLoading { active_request_id },
}
  • Started - a new fetch was kicked off. replaced_request_id is Some if this request cancelled a previous one (under LatestWins).
  • CacheHit - the cache is fresh, no fetch needed. Use the cached data.
  • StaleCacheHit - data is stale but still serveable. Return the stale data to the UI immediately, then use the request_id to run a background fetch.
  • IgnoredWhileLoading - a request was already running and the policy is IgnoreWhileLoading.

Forcing a refetch​

QueryFetchMode controls whether the cache is consulted.

  • Normal (default) - respects the cache policy. Returns CacheHit if fresh.
  • Force - bypasses cache freshness. Always starts a new fetch. Still respects IgnoreWhileLoading if a request is already in flight.

Use Force when the user explicitly requests fresh data (pull-to-refresh, a manual reload button).

client.force_fetch_query::<UserData, MyError>(
    QueryKey::from(["users", "42"]),
    cache_policy,
    request_policy,
    now_ms,
    cx,
);

Bulk operations on QueryClient​

The QueryClient has bulk methods that operate across all resources matching a filter. These use QueryKeyFilter to select which queries to touch.

Invalidation​

invalidate_queries clears cache freshness on matching resources. The next access to those queries will trigger a refetch. It does not cancel in-flight requests.

let client = cx.global::<QueryClient>();
client.invalidate_queries(
    &QueryKeyFilter::Prefix(&QueryKey::from(["users"])),
    cx,
);

This is the standard pattern after a mutation: invalidate the affected query prefix so the next read fetches fresh data.

Removal​

remove_queries drops matching resources from the registry entirely. Unlike invalidation, which keeps the entity and just expires its cache, removal deletes the entity.

client.remove_queries(&QueryKeyFilter::All, cx);

QueryKeyFilter variants​

  • Exact(&key) - matches only the exact key
  • Prefix(&key) - matches all keys that start with the given prefix segments
  • All - matches every key

Prefix matching is hierarchical. Prefix(&QueryKey::from(["users"])) matches ["users"], ["users", "42"], and ["users", "42", "posts"].

Optimistic updates​

Sometimes you want to update the UI before the server confirms the change. set_query_data writes a value directly into the cache, storing the previous value for rollback.

let client = cx.global::<QueryClient>();
client.set_query_data::<Vec<User>, MyError>(
    &key,
    updated_user_list,
    cx,
);

If the mutation later fails, roll back to the previous data:

client.rollback_query_data::<Vec<User>, MyError>(&key, cx);

This restores the data that was in the resource before set_query_data was called. If there was no previous data, the rollback does nothing and returns false.

Garbage collection​

Queries that no component is observing will linger in memory until garbage collection picks them up. The gc method on QueryClient removes resources that have been idle longer than gc_time_ms.

let client = cx.global::<QueryClient>();
client.gc(cx, now_ms);

The default GC time is 5 minutes (300,000 ms). You can change this per query through QueryOptions:

let opts = QueryOptions::new(key).gc_time_ms(10 * 60 * 1_000); // 10 minutes

Or change the client-wide default during construction:

cx.set_global(
    QueryClient::new(cache_policy, request_policy)
        .with_gc_time(10 * 60 * 1_000),
);

Putting it together​

A typical setup looks like this:

// App-wide defaults: 30s TTL with 60s stale window, latest-wins dedup, 10-minute GC
cx.set_global(
    QueryClient::new(
        CachePolicy::StaleWhileRevalidate { ttl_ms: 30_000, stale_ms: 60_000 },
        RequestPolicy::LatestWins,
    )
    .with_gc_time(10 * 60 * 1_000),
);

Then override per query as needed. A real-time notifications feed might use NoCache. A rarely-changing config endpoint might use Ttl { ttl_ms: 300_000 }. Most things fall somewhere in between.

See Queries for how to define queries that use these policies, QueryClient for the full client API, and Error Handling for what happens when a fetch fails.