Caching strategies

Where a cached copy can live

Mobile caching has three layers. They are ordered by speed, and in the opposite order by how long they survive:

Two words for the rest of the lesson. A cache hit is a lookup that finds what it asked for. A cache miss is a lookup that does not, and therefore has to fall back to a slower layer.

A read walks the layers in order:

1. Look in the memory cache — a dictionary the running app holds in RAM. A hit here returns immediately and nothing else happens.
2. On a miss, look in the disk cache — files the app wrote into its own container. A hit here is copied back into memory, so the next read of the same thing is a memory hit.
3. On a miss there too, ask the network. What comes back is written into memory and disk on its way to the caller.

Every entry is filed under a cache key: the value you look it up by, usually a string you build yourself. Choosing it is part of the design. Suppose the key for an image is only its URL. The 60-point avatar in a list and the 400-point version on the detail screen then share one key, so whichever was stored last wins. Either the detail screen shows the small thumbnail stretched up and blurry, or the list holds a 400-point bitmap it has no use for. The rule that avoids this is to put everything that changes the stored value into the key — the URL and the pixel size you decoded it at.

You already own two of these layers without writing any code. URLCache is Foundation's HTTP cache: it stores URLSession responses in memory and on disk, and it obeys the caching instructions the server sends in its headers. Those same headers are what AsyncImage began respecting in iOS 27 (U5-01). Separately, when the system runs short of memory it asks apps to give some back, which empties a well-behaved memory cache for you.

So the part you usually write yourself is one thing: a keyed store for values that are expensive to produce again. Decoded images are the main case, and decoding is where the cost hides. A 4 MB JPEG is compressed on disk, but to draw it the system expands it into a bitmap of one entry per pixel. A 3000 × 2500 photo is 7.5 million pixels, and at 4 bytes each that is roughly 30 MB in memory, whatever the file weighed. Doing that expansion again on every scroll is exactly the main-thread work that P1-03 blames for dropped frames.

Two rules about what goes where:

• Cache the expensive form. Store the decoded, downsampled image, not the raw bytes you downloaded. The decode was the cost you were trying to avoid, so caching the compressed data leaves you paying it on every read.
• Secrets never go in a cache. Access tokens and personal data belong somewhere else, and D2-05 is about where. A disk cache does get iOS's default file encryption, but that default leaves the file readable from the first unlock after a restart until the device powers off. On top of that a cache lives a long time, it is easy to forget about, and it can be copied into a device backup.

Eviction, and the LRU rule

A cache with no size limit keeps growing until the system terminates the app for using too much memory. So a real cache is bounded: it holds at most a set number of entries, or at most a set number of megabytes. Once it is full, storing something new means removing something old. That removal is called eviction, and the rule that picks which entry goes is the eviction policy.

The usual policy is LRU, short for least recently used. "Recently used" has an exact, mechanical meaning here: every time an entry is read or written, the cache marks it as the newest one. When room is needed, the entry whose last read or write is furthest in the past is the one removed. The bet behind that rule is that recent use predicts future use, and for user interfaces the bet is a good one — the rows you scrolled past yesterday are far less likely to be wanted again than the ones on screen right now.

LRU is also a standard interview coding question, because the obvious implementation misses one thing: reads count as use too. If only writes updated the ordering, an entry that thousands of readers hit every second would be evicted for being old. So get has to update the ordering, which means get changes the cache.

The classic shape pairs a dictionary, which finds an entry by key in constant time, with a second structure that remembers the order. Here is that shape, small enough to run:

Four lines come out of that run. Follow them one at a time:

1. The three put calls fill the cache to its capacity of three. Oldest first, the order is avatar-1, avatar-2, avatar-3. Nothing is printed yet, because nothing has been evicted.
2. get("avatar-1") prints hit avatar-1 = 101 and moves that entry to the newest position. The order is now avatar-2, avatar-3, avatar-1.
3. put("avatar-4", 104) needs room, so it removes the oldest entry and prints evict avatar-2. Without step 2 the entry removed here would have been avatar-1.
4. get("avatar-2") prints miss avatar-2, because it is gone. get("avatar-1") prints hit avatar-1 = 101, because the read in step 2 saved it.

The second printed line is the whole point: evict avatar-2, not evict avatar-1. A read protected avatar-1.

This version has one honest weakness. touch calls order.removeAll, which scans the entire array, so every read costs time proportional to the number of entries — that is what "O(n)" means. Production implementations store the order in a doubly linked list alongside a dictionary of nodes, which brings the reordering down to constant time, "O(1)". Say that sentence in an interview, and still write the array version first: an interviewer would rather see a correct array than a half-finished linked list.

NSCache, and how it differs from a dictionary

NSCache is Foundation's ready-made in-memory cache. It is used much like a dictionary — setObject(_:forKey:) to store, object(forKey:) to read — and it differs in four ways worth being able to recite.

1. It empties itself when memory runs short. When the system is low on memory it asks apps to release some, and terminates apps that release nothing. An NSCache gives its contents up automatically, so the cache shrinks instead of the app being killed. A Dictionary you built yourself holds on until you write the code that empties it.
2. It is safe to use from several threads at once. You can read and write it from different threads without adding a lock of your own.
3. Its keys are objects, and it does not copy them. NSCache is generic over class types, so it is written NSCache<NSString, UIImage> — a Swift String has to become an NSString, and a struct has to be wrapped in a class, before either can be a key. A Dictionary copies its keys when you insert them. NSCache keeps a reference to yours instead. So if you hand it a mutable key object and then change that object, the entry you stored can become impossible to find again, because the key no longer hashes to the same place. Use keys that cannot change.
4. Entries may disappear between any two calls. Nothing you store is guaranteed to still be there on the next line.

Point 4 decides how you use it. An NSCache entry is a hint, never the only copy of something. Every read path has to handle the miss, and anything you cannot afford to lose must also live somewhere else.

Two limits are worth setting. countLimit caps the number of entries. totalCostLimit caps the total of the cost values you pass to setObject(_:forKey:cost:); for images the natural cost is the bitmap's size in bytes. Both are advisory — the cache treats them as a target rather than a hard ceiling. Apple also does not document which entry NSCache discards first, so do not write code that depends on it being the least recently used one.

The stampede, and single-flight

Caches also create a failure of their own. Here is how one popular image becomes a burst of identical downloads. Follow the sequence:

1. A comment thread shows twelve cells, and twelve of them show the same avatar.
2. The screen appears. Nothing is cached yet, so cell 1 looks the avatar up, misses, and starts a download.
3. A millisecond later cell 2 looks up the same avatar. Cell 1's download has not finished, so the cache is still empty. Cell 2 misses as well, and starts a second download of the same file.
4. Cells 3 to 12 do the same thing. Twelve downloads of one image are now running at once.
5. All twelve finish and all write the same entry. Eleven of them were wasted — paid for in the user's data allowance, the user's battery, and your image host's bill.

That burst has a name: a cache stampede. Its cause is step 3. A cache only helps a reader who arrives after the first reader has finished, and here every reader arrives inside that gap.

The fix is called single-flight: for any one key, only one piece of work is allowed to be running at a time. The first miss starts the download. Every miss that happens while that download is still running waits for the same download instead of starting another. You have seen the mechanism already — C2-01 uses it when two concurrent calls both miss the cache and both download. Here it is again, as a piece of infrastructure:

actor ImageLoader {
    private var cache: [URL: Image] = [:]
    private var inFlight: [URL: Task<Image, Error>] = [:]

    func image(for url: URL) async throws -> Image {
        if let cached = cache[url] { return cached }
        if let running = inFlight[url] { return try await running.value }   // join, don't duplicate

        let task = Task { try await download(url) }
        inFlight[url] = task
        defer { inFlight[url] = nil }

        let image = try await task.value
        cache[url] = image
        return image
    }
}

Three things carry the weight there.

Two dictionaries, not one. cache holds finished results. inFlight holds work that has started and not finished. A Task is a handle to work that is already running, and awaiting the same handle twice does not run the work twice — both callers get the one result. So inFlight is a cache of work, sitting next to the cache of results.

The type is an actor, which is what makes the check-and-insert safe. Between reading inFlight[url] and writing to it there is no await, so no other caller can slip in between those two lines and start a second download. That is C2-01's rule applied exactly: do not suspend in the middle of a read-modify-write.

defer clears the slot when the function returns, so a key never keeps pointing at a task that has already finished — including one that finished by throwing an error.

When an interviewer asks "what happens if fifty cells want the same image", this is the answer they are listening for.

Invalidation: the four honest policies

Invalidation is deciding that a cached copy has stopped being true, and getting rid of it — or refreshing it — before anyone reads it again. A copy that is still sitting there but is now out of date is called stale.

There are four workable answers to "when does this stop being true", in increasing order of how much coordination they need.

1. TTL, short for time to live. Store the time alongside the value, and treat the entry as stale once a fixed duration has passed. This is honest for data that is mostly read and has a tolerance you can name: a feed may be minutes old, an avatar may be days old. Its weakness is that the duration is a guess, and nothing tells you when the guess was wrong.
2. Event-driven purge. A write deletes exactly the entries it made untrue: posting a comment purges the cached copy of that thread. This is the most precise policy, and its difficulty is organisational rather than technical. Every code path that writes has to remember to purge, including the one a colleague adds next year. That is the argument for keeping the cache write and the purge inside one type — the repository from D1-05, which is already the only place in the app that talks to the server.
3. Conditional requests, using an ETag. An ETag is a short string the server sends alongside a response: a version marker for that exact body. You store it with the cached copy, and send it back on the next request in an If-None-Match header. That makes the request conditional — it means "send me the body only if it has changed". If nothing has changed, the server answers 304 Not Modified: headers, no body. You pay one round trip and almost no bytes, and you keep the copy you already had. URLCache does all of this for you when the server sends the headers.
4. Stale-while-revalidate. Show the stale copy immediately, start a refresh behind it, and update the screen when the fresh copy arrives. One call, two paints. It gives the best perceived speed for feeds and profiles, and the price is a screen whose content can change a second after it appeared — so the UI has to survive that without losing the reader's place or their tap. D1-06 wires this exact pattern into Ledger's feed.

The senior answer to "how would you cache X" names three things rather than one: which layer it lives in, what evicts it, and what invalidates it. The last of the three is justified by how out of date X may honestly be.