Chapter 5 of 12 · Core objects and memory

Collections in Disguise: Lists, Hashes, Sets, Sorted Sets

Build a tiny hash, then keep growing it, and watch its encoding change underneath you without a single command changing shape:

Terminal — a hash that outgrows its compact encoding$ redis-cli HSET user:1 name "Ava" age 29
(integer) 2
$ redis-cli OBJECT ENCODING user:1
"listpack"
$ # now push past hash-max-listpack-entries (default 128)
$ for i in $(seq 1 200); do redis-cli HSET user:1 f$i v$i > /dev/null; done
$ redis-cli OBJECT ENCODING user:1
"hashtable"

HSET, HGET, HGETALL all still work exactly the same. What changed is invisible from the API and significant in memory: a small hash is packed tight; a big one gets a real hash table underneath. Every collection type makes this same trade, with its own thresholds and its own compact format.

The Compact Formats: listpack and intset

For small collections, Redis avoids per-element pointers entirely and instead packs every entry — length-prefixed, back-to-back — into one contiguous block of memory:

listpack — the modern, general-purpose compact encoding (introduced for Streams in Redis 5, and since Redis 7 the default small-collection encoding for hashes, sets, sorted sets, and lists, replacing the older ziplist). A flat sequence of length-prefixed entries with no next/prev pointers at all — you walk it by reading each entry's own length header.
intset — used only for a set made entirely of integers. It's a plain sorted array of integers, deduplicated, so membership tests binary-search in O(log n) instead of hashing, and the whole set is about as memory-dense as a C array can be.

A contiguous, pointer-free block is small and cache-friendly to scan linearly — but it's still O(n) to scan. That's fine as long as "n" stays small; the whole strategy depends on collections usually being small enough that the difference between O(1) hashing and O(n) linear scan doesn't matter, while the memory savings always do.

When a Collection Grows: quicklist and skiplist

Past a size or element-length threshold, each type converts to a structure built for scale instead of density:

Type	Compact (small)	Scaled (large)	Config knob
List	listpack	quicklist	list-max-listpack-size
Hash	listpack	hashtable	hash-max-listpack-entries / -value
Set (ints)	intset	listpack / hashtable	set-max-intset-entries
Set (strings)	listpack	hashtable	set-max-listpack-entries
Sorted set	listpack	skiplist	zset-max-listpack-entries / -value

A quicklist is a doubly linked list of nodes, where each node is itself a small listpack — the best of both shapes: cheap inserts/removals at either end like a linked list, but each node still packs several elements densely instead of one pointer-heavy node per element. A skiplist is the classic probabilistic structure with multiple forward-pointer "levels" per node, giving sorted sets O(log n) insert, remove, and range queries by score — something no flat array could do efficiently once the set gets large. Redis pairs the skiplist with a plain dictionary for O(1) score lookups by member, so a big sorted set actually keeps two structures over the same data, each optimized for a different access pattern.

The ratchet only turns one way. Once a collection converts to its scaled encoding, deleting elements back down below the threshold does not convert it back — converting is a one-time cost Redis is willing to pay to grow, not something worth re-paying on every shrink.

Key Idea

Every collection type separates its abstract API (push, pop, add, range, score) from its physical encoding, and every one of them defaults to the cheapest encoding that fits the data, converting up as it grows. This is the same move Chapter 4 made for strings, applied independently, type by type, all the way through the codebase — a pattern worth noticing now because Chapter 12 will ask you to go looking for it elsewhere too.

Warning

Conversion is triggered by either the element count or the size of any individual value — a hash with only three fields still becomes a hashtable if one of those values is a 10 KB blob, because hash-max-listpack-value caps individual entry size, not just entry count. If your memory usage jumps unexpectedly on a collection you assumed was "small," check the value sizes, not just the length.

Try It Yourself

Compare the two set encodings directly: SADD ints 1 2 3 then OBJECT ENCODING ints (expect intset), versus SADD words a b c then OBJECT ENCODING words (expect listpack, since strings can't use the integer-only intset). Then run SADD ints hello — adding one non-integer member to an intset — and check the encoding again. One non-numeric member is enough to convert the whole set away from intset.

Encodings solve memory density for individual values and small collections. But the keyspace itself — the mapping from every key name to its value — is one giant hash table, and it has its own much more interesting scaling story: how do you resize a hash table holding millions of entries without ever pausing to do it all at once? That's next.

Further Reading antirez/listpack — format spec · redis.io — OBJECT ENCODING