Persistence and networking · advanced

Caching strategies

The cache hierarchy, LRU eviction, stampede protection — and invalidation, the part that earns its reputation.

13 min read14 min practice0/2 exercises5 recall cards

By the end you will be able to

Design a memory/disk/network cache hierarchy and place each kind of data in it
Implement LRU eviction — the interview classic — and explain NSCache's differences
Prevent cache stampedes with single-flight, and choose honest invalidation policies

Guess firstAnswering before you read makes the explanation stick — even when you get it wrong.

"There are only two hard things in computer science: cache invalidation and naming things." Why does invalidation earn its place in the joke, when adding a cache is twenty minutes' work?

The hierarchy

Mobile caching is three layers, ordered by speed and volatility:

Layer	Speed	Survives	Typical residents
Memory	~ns	until memory pressure or relaunch	decoded images, hot models
Disk	~ms	relaunches; until eviction/uninstall	image files, JSON responses, thumbnails
Network	~100ms–s	— (the source of truth)	everything, freshly

The canonical read path checks them in order — memory, then disk (promoting hits back to memory), then network (populating both). You already own several of these layers without writing them: URLCache honours HTTP cache headers for URLSession responses (and backs AsyncImage's iOS 27 caching), and the OS flushes memory caches under pressure. The custom work is usually one thing: a keyed store for expensive-to-recompute values — decoded images above all, since a 4MB JPEG decodes to ~30MB of bitmap and re-decoding per scroll is the jank factory from the Instruments lesson.

Two placement rules with teeth:

Cache the expensive form. Store the decoded, downsampled image in memory, not the raw data — the decode was the cost you were avoiding.
Secrets never go in caches. Tokens and personal data have a different home (two lessons from now); disk caches are unencrypted, long-lived, and forgotten in backups.

LRU: the eviction interview classic

A bounded cache must choose a victim when full. Least Recently Used evicts the entry untouched the longest — the bet that recent access predicts future access, which holds remarkably well for UI workloads (the rows you scrolled past yesterday are colder than the ones on screen).

LRU is also a genuine interview-coding staple, because the naive version hides a trap: to evict the least-recent, you must track recency on every read — reads mutate. The classic implementation pairs a dictionary (O(1) lookup) with an order structure; here is the working shape:

An LRU cache — reads refresh recency, eviction removes the coldest.

struct LRUCache {
    let capacity: Int
    private var storage: [String: Int] = [:]
    private var order: [String] = []          // index 0 = least recently used

    mutating func get(_ key: String) -> Int? {
        guard let value = storage[key] else {
            print("  miss \(key)")
            return nil
        }
        touch(key)                            // ← reads change recency
        print("  hit \(key) = \(value)")
        return value
    }

    mutating func put(_ key: String, _ value: Int) {
        if storage[key] == nil && storage.count >= capacity {
            let coldest = order.removeFirst()
            storage[coldest] = nil
            print("  evict \(coldest)")
        }
        storage[key] = value
        touch(key)
    }

    private mutating func touch(_ key: String) {
        order.removeAll { $0 == key }
        order.append(key)                     // most recent at the end
    }
}

var cache = LRUCache(capacity: 3)
cache.put("avatar-1", 101)
cache.put("avatar-2", 102)
cache.put("avatar-3", 103)

// avatar-1 is oldest — but a read makes it recent again:
_ = cache.get("avatar-1")

// Inserting a fourth must now evict avatar-2, not avatar-1:
cache.put("avatar-4", 104)

_ = cache.get("avatar-2")
_ = cache.get("avatar-1")

struct LRUCache {
    let capacity: Int
    private var storage: [String: Int] = [:]
    private var order: [String] = []          // index 0 = least recently used

mutating func get(_ key: String) -> Int? {
        guard let value = storage[key] else {
            print("  miss \(key)")
            return nil
        }
        touch(key)                            // ← reads change recency
        print("  hit \(key) = \(value)")
        return value
    }

mutating func put(_ key: String, _ value: Int) {
        if storage[key] == nil && storage.count >= capacity {
            let coldest = order.removeFirst()
            storage[coldest] = nil
            print("  evict \(coldest)")
        }
        storage[key] = value
        touch(key)
    }

private mutating func touch(_ key: String) {
        order.removeAll { $0 == key }
        order.append(key)                     // most recent at the end
    }
}

var cache = LRUCache(capacity: 3)
cache.put("avatar-1", 101)
cache.put("avatar-2", 102)
cache.put("avatar-3", 103)

// avatar-1 is oldest — but a read makes it recent again:
_ = cache.get("avatar-1")

// Inserting a fourth must now evict avatar-2, not avatar-1:
cache.put("avatar-4", 104)

_ = cache.get("avatar-2")
_ = cache.get("avatar-1")

The middle step is the whole idea: reading avatar-1 saved it, and the eviction chose avatar-2. (Production versions replace the O(n) array with a doubly-linked list for O(1) touch — say that sentence in the interview, then implement the simple version first.)

NSCache, Apple's in-memory cache, differs from a dictionary in exactly the ways worth citing: it evicts automatically under memory pressure (your app sheds cache instead of dying in a jetsam), it is thread-safe without your locks, and — the gotcha — it does not copy keys (mutate a key object after inserting and it may never match its entry again; the defensive copy Dictionary makes is exactly what prevents that) and its contents can vanish between any two calls. Design for the miss: an NSCache entry is a hint, never the only copy of anything.

The stampede, and single-flight

The failure mode caches create: a popular entry expires, and every concurrent reader misses at once — twelve cells all requesting the same avatar, twelve identical downloads. The fix is single-flight: the first miss starts the work; every concurrent miss awaits the same task. You met the mechanism in the actors lesson as the reentrancy fix; here it is as infrastructure:

actor ImageLoader {
    private var cache: [URL: Image] = [:]
    private var inFlight: [URL: Task<Image, Error>] = [:]

    func image(for url: URL) async throws -> Image {
        if let cached = cache[url] { return cached }
        if let running = inFlight[url] { return try await running.value }   // join, don't duplicate

        let task = Task { try await download(url) }
        inFlight[url] = task
        defer { inFlight[url] = nil }

        let image = try await task.value
        cache[url] = image
        return image
    }
}

The actor makes the check-and-insert atomic (no await between reading inFlight and writing it — the reentrancy lesson's rule); the Task dictionary is the cache of work rather than of results. Interviewers reaching for "what if 50 cells want the same image" are fishing for exactly this.

Predict, then runA wrong prediction you have committed to is worth more than a right answer you read.

Single-flight arithmetic: a screen with 30 cells across 3 distinct images, twice (a refresh). What does each strategy download?

struct DownloadCounter {
    private(set) var started: [String: Int] = [:]
    private(set) var cache: Set<String> = []
    private(set) var inFlight: Set<String> = []

    mutating func naiveRequest(_ url: String) {
        // No coordination: every concurrent miss downloads...
        if !cache.contains(url) {
            started[url, default: 0] += 1
            inFlight.insert(url)          // ...though results do land in the cache
        }
    }

    mutating func singleFlightRequest(_ url: String) {
        if cache.contains(url) { return }
        if inFlight.contains(url) { return }      // join the running download
        inFlight.insert(url)
        started[url, default: 0] += 1
    }

    mutating func finishAll() {
        for url in inFlight { cache.insert(url) }
        inFlight = []
    }
}

let cells = ["a", "b", "c", "a", "b", "c", "a", "b", "c", "a"]   // 10 visible cells

var naive = DownloadCounter()
for url in cells { naive.naiveRequest(url) }      // first paint: nothing cached yet
naive.finishAll()
for url in cells { naive.naiveRequest(url) }      // refresh: cache warm

var flight = DownloadCounter()
for url in cells { flight.singleFlightRequest(url) }
flight.finishAll()
for url in cells { flight.singleFlightRequest(url) }

let naiveTotal = naive.started.values.reduce(0, +)
let flightTotal = flight.started.values.reduce(0, +)
print("naive: \(naiveTotal) downloads")
print("single-flight: \(flightTotal) downloads")

Invalidation: the honest policies

Four workable answers to "when does this stop being true", in increasing order of coordination:

TTL — entries expire after a duration. Honest for read-mostly data with known tolerance (feeds: minutes; avatars: days). The failure: the tolerance was a guess.
Event-driven — mutations kill exactly the entries they invalidate: posting a comment purges that thread. Precise, and the discipline is organisational: every write path must remember — which is why cache writes and invalidation belong in one repository type (the architecture lesson's seam), not scattered.
Versioned/ETag — ask the server "still current?" with If-None-Match; a 304 costs a round-trip but no body. URLCache speaks this natively when the backend cooperates.
Stale-while-revalidate — serve the stale copy now, refresh behind it, update the UI when fresh lands. The best perceived performance for feeds and profiles; the cost is a UI that must tolerate content shifting a second after appearing.

The lead-level answer to most "how would you cache X" questions is naming the layer, the eviction, and the invalidation policy — with the policy justified by how stale X may honestly be.

Check yourself

Design review: a teammate caches the user's subscription entitlement in UserDefaults with a 24-hour TTL "so paywall checks are instant offline". What is the senior objection?

›

Implement LRU eviction

Write the code

The interview classic, from the lesson's shape: implement get/put with capacity 2 so the trace matches — reads must refresh recency.

struct LRUCache {
    let capacity: Int
    private var storage: [String: String] = [:]
    private var order: [String] = []

    mutating func get(_ key: String) -> String? {
        // Hit: refresh recency, return value. Miss: nil.
        return nil
    }

    mutating func put(_ key: String, _ value: String) {
        // New key at capacity: evict least-recent first. Always refresh recency.
    }

    var contents: [String] { order }
}

var cache = LRUCache(capacity: 2)
cache.put("a", "alpha")
cache.put("b", "beta")
print(cache.get("a") ?? "miss")
cache.put("c", "gamma")
print(cache.get("b") ?? "miss")
print(cache.get("a") ?? "miss")
print(cache.contents)

Solution unlocks after 3 attempts

›

Choose the policy

Write the code

Encode the invalidation decision table: given a data type's properties, return the policy — the reasoning you would say aloud in a design interview.

struct DataKind {
    let name: String
    let stalenessCostHigh: Bool     // is being wrong expensive?
    let mutatedByThisApp: Bool      // do our own writes change it?
    let changeFrequency: String     // "rarely" | "sometimes" | "constantly"
}

func policy(for kind: DataKind) -> String {
    // stalenessCostHigh                -> "no cache — read the source of truth"
    // mutatedByThisApp                 -> "event-driven purge on our writes"
    // changes "rarely"                 -> "long TTL"
    // otherwise                        -> "stale-while-revalidate"
    return ""
}

let kinds = [
    DataKind(name: "entitlement", stalenessCostHigh: true, mutatedByThisApp: false, changeFrequency: "rarely"),
    DataKind(name: "own profile", stalenessCostHigh: false, mutatedByThisApp: true, changeFrequency: "sometimes"),
    DataKind(name: "country list", stalenessCostHigh: false, mutatedByThisApp: false, changeFrequency: "rarely"),
    DataKind(name: "feed", stalenessCostHigh: false, mutatedByThisApp: false, changeFrequency: "constantly"),
]

for kind in kinds {
    print("\(kind.name): \(policy(for: kind))")
}

struct DataKind {
    let name: String
    let stalenessCostHigh: Bool     // is being wrong expensive?
    let mutatedByThisApp: Bool      // do our own writes change it?
    let changeFrequency: String     // "rarely" | "sometimes" | "constantly"
}

func policy(for kind: DataKind) -> String {
    // stalenessCostHigh                -> "no cache — read the source of truth"
    // mutatedByThisApp                 -> "event-driven purge on our writes"
    // changes "rarely"                 -> "long TTL"
    // otherwise                        -> "stale-while-revalidate"
    return ""
}

let kinds = [
    DataKind(name: "entitlement", stalenessCostHigh: true, mutatedByThisApp: false, changeFrequency: "rarely"),
    DataKind(name: "own profile", stalenessCostHigh: false, mutatedByThisApp: true, changeFrequency: "sometimes"),
    DataKind(name: "country list", stalenessCostHigh: false, mutatedByThisApp: false, changeFrequency: "rarely"),
    DataKind(name: "feed", stalenessCostHigh: false, mutatedByThisApp: false, changeFrequency: "constantly"),
]

for kind in kinds {
    print("\(kind.name): \(policy(for: kind))")
}

Solution unlocks after 3 attempts

⌘Cache something real

Build the single-flight ImageLoader actor in a grid app showing the same 3 image URLs in 30 cells. Log downloads; confirm 3, not 30. Comment out the inFlight check and count again.
Inspect URLCache.shared after fetching an API with cache headers (curl -I the endpoint to see them): print currentDiskUsage, then fetch again and note the speed. Try a .reloadIgnoringLocalCacheData request for comparison.
Replace a Dictionary image cache with NSCache, run in the simulator, and trigger Debug ▸ Simulate Memory Warning — log eviction by checking a known key. Design the code path that survives it.
Add a TTL wrapper ((value, storedAt)) to any cached API call, and write the Swift Testing boundary table for it: just-fresh, exactly-at-TTL, just-stale — the testing lesson's discipline applied here.

This one needs the real Xcode — the browser runtime cannot model it.

Design itSaved on this device. Never graded.

Design the full caching story for a podcast app's artwork: sizes needed (list thumbnail vs player), what exactly is stored at each layer, eviction limits and policy, the single-flight shape, and the invalidation answer for "the show changed its artwork". End with the one metric that would tell you the design is working in production.

Checkpoint

You can now:

Place data in the memory/disk/network hierarchy and cache the expensive form
Implement LRU with read-refreshed recency, and quote NSCache's real semantics
Kill stampedes with an actor-guarded in-flight task map
Choose and defend an invalidation policy by cost-of-wrongness

Next up: offline-first sync — the system-design interview, as a lesson.

How well do you know this now? Rating yourself honestly, then being tested on it, is how you find out where your intuition is wrong.