Dictionaries & Hash Maps
Master the foundational data structure of Python. From CPython's PEP 468 compact hash table architecture and $O(1)$ amortized lookups to dynamic dictionary views, LIFO eviction with popitem(), and enterprise in-memory cache patterns.
15 mins
Deep architectural dive
Hashable
Immutable keys only
O(1)
Hash table direct index
17 Checks
Interactive assessment
What You Will Master
01 / Deep Architectural Foundations
A Python dictionary (dict) is an associative array mapping unique, immutable keys to arbitrary values. Unlike sequence collections (lists and tuples) where items are accessed via positional indices (0, 1, 2, ...), dictionaries allow instantaneous $O(1)$ access using semantic keys: user["email"], metrics["p99_latency"], or coordinate tuples coords[(40.7128, -74.0060)].
Underneath Python's surface, dictionaries are the engine that runs the language itself. Module namespaces, class attribute tables (__dict__), function keyword arguments (**kwargs), and global variable scopes (globals()) are all implemented directly as Python dictionaries.
Since Python 3.7 (originally an implementation detail in CPython 3.6), dictionaries are guaranteed by the language specification to strictly preserve insertion order. When you iterate over a dictionary, items yield in the exact order they were inserted.
Dictionary Invariant Rules
- Key Immutability: Keys must be hashable (implementing
__hash__()and__eq__()). Strings, numbers, and frozen tuples are allowed; lists and dicts are not. - Key Uniqueness: Inserting a duplicate key silently overwrites the previous value at that key.
- Value Arbitrariness: Values can be any Python object, mutable or immutable, nested to arbitrary depth.
- Dynamic View Objects:
keys(),values(), anditems()return live views that reflect updates without copying memory.
02 / Syntax Anatomy & Construction Paradigms
# 1. Standard literal syntax
user = {"id": 101, "role": "admin", "active": True}
# 2. Keyword constructor (clean identifiers)
params = dict(timeout=30, retries=3, debug=False)
# 3. From iterable of key-value 2-tuples
pairs = dict([("region", "us-east-1"), ("tier", "p2")])Literal syntax {"a": 1} compiles down to the BUILD_MAP CPython bytecode instruction, making it measurably faster than invoking the callable dict() constructor.
# 1. Parallel iterable pairing with zip()
headers = ["Host", "User-Agent", "Accept"]
values = ["api.pylabs.org", "curl/8.1", "*/*"]
req_headers = dict(zip(headers, values))
# 2. Dictionary comprehension with filtering
squares = {x: x**2 for x in range(6) if x % 2 == 0}
# Result: {0: 0, 2: 4, 4: 16}dict(zip(keys, values)) cleanly fuses two equal-length iterables in $O(N)$ linear time without manual indexing.
03 / CPython Architecture: The PEP 468 Compact Hash Table
How PEP 468 Slashed Dict Memory by 25% and Guaranteed Order
Prior to Python 3.6, CPython implemented dictionaries as a classic sparse hash table where each row in an allocated bucket table held a full 24-byte struct (hash, key_ptr, value_ptr). To maintain $O(1)$ lookups without excessive collisions, hash tables maintain a low load factor ($< 66\%$), leaving roughly one-third of the array completely empty (DKIX_EMPTY), wasting massive RAM and causing iteration order to depend on memory address hashes.
Slot Hash Key Value ------------------------------------------ 0 - EMPTY EMPTY <-- 24 bytes wasted! 1 1928471 "name" "Alice" <-- 24 bytes 2 - EMPTY EMPTY <-- 24 bytes wasted! 3 8391024 "role" "Admin" <-- 24 bytes 4 - EMPTY EMPTY <-- 24 bytes wasted! Result: Massive memory waste + Non-deterministic order
# 1. Sparse Indices Array (tiny int8 integers):
indices = [-1, 0, -1, 1, -1, -1, -1, -1] # 1 byte each!
# 2. Dense Entries Array (Contiguous, packed in order):
entries = [
Row 0: [hash("name"), "name", "Alice"],
Row 1: [hash("role"), "role", "Admin"]
]
Result: 20-25% RAM savings + Strict Insertion Order!1. Hash Computation
CPython calls hash(key). SipHash randomized per process prevents HashDoS collision attacks.
2. Sparse Index Lookup
slot = hash & (table_size - 1). Reads the byte at indices[slot].
3. Dense Array Access
Uses the index to directly fetch entries[index]. Checks identity (is) then equality (==).
04 / Collection Matrix: When to Use Dict vs Others
| Data Structure | Key / Index Type | Lookup Cost | Insertion Order | Primary Use Case |
|---|---|---|---|---|
dict | Any Hashable Key | O(1) Avg | Yes (3.7+) | Key-value mappings, entity records, JSON API payloads, fast associative lookup. |
list | Integer Index (0..N-1) | O(1) by index, O(N) by value | Yes | Ordered sequences, stacks, dynamic buffers, queues. |
set | No keys (values only) | O(1) Avg | No | Deduplication, membership testing, Venn set algebra. |
collections.defaultdict | Any Hashable Key | O(1) Avg | Yes (3.7+) | Grouping and accumulation without manual setdefault() guards. |
05 / Interactive Syntax Reference Sandbox
Dictionary Construction Idioms
Five distinct ways to construct dictionaries depending on data source, memory efficiency, and readability:
# 1. Standard literal (fastest CPython bytecode)
d1 = {"host": "localhost", "port": 5432}
# 2. Keyword constructor (clean strings)
d2 = dict(host="localhost", port=5432)
# 3. From key-value 2-tuples
d3 = dict([("status", 200), ("reason", "OK")])
# 4. Uniform default initialization
d4 = dict.fromkeys(["cpu_pct", "mem_pct", "disk_pct"], 0.0)
# 5. Parallel iterable pairing with zip()
keys = ["lat", "lon", "elev"]
vals = [37.7749, -122.4194, 16]
geo = dict(zip(keys, vals))
print("Literal:", d1)
print("Fromkeys:", d4)
print("Zipped Geo:", geo)06 / Tiered Walkthroughs: From Foundation to Mastery
Hierarchical Application Configuration Manager
Problem: Build an enterprise settings resolution engine. An application receives configuration from three tiers with cascading priority: 1) Command-line arguments (highest priority); 2) Environment variables (medium priority); 3) Default internal settings (fallback). Build a resolver that produces the final unified settings dictionary.
def resolve_application_config(
cli_flags: dict[str, any],
env_vars: dict[str, any],
base_defaults: dict[str, any]
) -> dict[str, any]:
"""
Resolves settings with priority: CLI Flags > Environment Variables > Base Defaults.
"""
# 1. Start with a clone of base defaults
resolved = base_defaults.copy()
# 2. Layer environment variables (only for recognized setting keys)
for key in base_defaults.keys():
if key in env_vars and env_vars[key] is not None:
resolved[key] = env_vars[key]
# 3. Layer CLI flags (highest priority; skip None values)
for key, val in cli_flags.items():
if key in resolved and val is not None:
resolved[key] = val
return resolved
# Configuration fixtures:
defaults = {
"HOST": "127.0.0.1",
"PORT": 8000,
"WORKERS": 4,
"LOG_LEVEL": "INFO",
"ENABLE_TELEMETRY": True
}
environment = {
"PORT": 8080,
"LOG_LEVEL": "DEBUG",
"HOST": "0.0.0.0"
}
command_line = {
"PORT": 9000, # Overrides both env and default!
"WORKERS": 16, # Overrides default
"HOST": None # Ignored: CLI flag was not supplied
}
print("=" * 65)
print(f"{'HIERARCHICAL CONFIGURATION RESOLVER':^65}")
print("=" * 65)
final_config = resolve_application_config(command_line, environment, defaults)
for setting, value in final_config.items():
print(f"{setting:<20}: {value}")
print("=" * 65)
# Assert correctness
assert final_config["PORT"] == 9000 # CLI won
assert final_config["HOST"] == "0.0.0.0" # Env won because CLI was None
assert final_config["WORKERS"] == 16 # CLI won
assert final_config["ENABLE_TELEMETRY"] is True # Default preserved
print("Configuration Cascade Hierarchy Verified!")
High-Performance Word Frequency & Inverted Indexer
Problem: Construct a text analysis tokenizer that scans document text, strips standard punctuation, computes word occurrence counts using the dict.get(word, 0) pattern, and calculates the top-N most frequent terms and vocabulary diversity metrics.
def analyze_document_frequencies(corpus: str, top_n: int = 3) -> dict:
"""
Computes token occurrences and extracts top-N vocabulary terms.
"""
# 1. Clean and tokenize text
for char in [",", ".", "!", "?", ";", ":", "\""]:
corpus = corpus.replace(char, " ")
tokens = corpus.lower().split()
# 2. Tally occurrences using dict.get()
frequencies = {}
for token in tokens:
frequencies[token] = frequencies.get(token, 0) + 1
# 3. Sort by occurrence count descending, then alphabetically ascending
sorted_items = sorted(frequencies.items(), key=lambda item: (-item[1], item[0]))
top_terms = sorted_items[:top_n]
# 4. Compute metrics
total_words = len(tokens)
unique_words = len(frequencies)
lexical_diversity = round(unique_words / total_words, 3) if total_words > 0 else 0.0
return {
"frequencies": frequencies,
"top_terms": top_terms,
"total_tokens": total_words,
"unique_tokens": unique_words,
"lexical_diversity": lexical_diversity
}
sample_doc = """
Python is fast. Python is simple, and Python is productive.
Engineers love Python because Python makes development productive and robust!
"""
print("=" * 60)
print(f"{'CORPUS TOKEN FREQUENCY ANALYZER':^60}")
print("=" * 60)
stats = analyze_document_frequencies(sample_doc, top_n=3)
print(f"Total Tokens Processed: {stats['total_tokens']}")
print(f"Unique Vocabulary Terms: {stats['unique_tokens']}")
print(f"Lexical Diversity: {stats['lexical_diversity']}")
print("-" * 60)
print("Top 3 Most Frequent Terms:")
for word, count in stats["top_terms"]:
print(f" '{word:<12}' -> {count} occurrences")
print("=" * 60)
Thread-Safe TTL In-Memory Cache Store with Lazy Eviction
Problem: Implement a lightweight in-memory key-value cache store. The cache must support: 1) setting keys with an expiration timestamp (Time-To-Live); 2) getting keys with lazy on-access expiration eviction; 3) a bulk purge method that removes all expired keys using list comprehension snapshotting to avoid mutation-during-iteration errors; 4) LIFO cache size trimming using .popitem().
class InMemoryTTLCache:
def __init__(self, max_capacity: int = 1000):
self._store = {}
self.max_capacity = max_capacity
def set(self, key: str, value: any, ttl_seconds: int, current_timestamp: int):
"""Stores key with expiration timestamp. Trims oldest LIFO item if at capacity."""
# Evict if capacity reached
if len(self._store) >= self.max_capacity and key not in self._store:
evicted_key, _ = self._store.popitem() # LIFO eviction
print(f"[CAPACITY EVICT] Evicted key '{evicted_key}'")
self._store[key] = {
"val": value,
"expires_at": current_timestamp + ttl_seconds
}
def get(self, key: str, current_timestamp: int) -> any:
"""Retrieves value. Evicts lazily if expired."""
entry = self._store.get(key)
if not entry:
return None
# Check expiration
if current_timestamp >= entry["expires_at"]:
del self._store[key] # Lazy eviction
print(f"[LAZY EVICT] Key '{key}' expired and purged.")
return None
return entry["val"]
def purge_expired(self, current_timestamp: int) -> int:
"""Purges all expired keys safely without mutating during iteration."""
# Snapshot expired keys first to prevent 'RuntimeError: dictionary changed size'
expired_keys = [
k for k, entry in self._store.items()
if current_timestamp >= entry["expires_at"]
]
for k in expired_keys:
del self._store[k]
return len(expired_keys)
def __len__(self):
return len(self._store)
# Test the cache engine:
cache = InMemoryTTLCache(max_capacity=3)
clock = 1000 # Simulated epoch second
print("=" * 65)
print(f"{'IN-MEMORY TTL KEY-VALUE CACHE STORE':^65}")
print("=" * 65)
# Insert 3 keys with varying TTLs
cache.set("session:u101", {"user": "Alice"}, ttl_seconds=30, current_timestamp=clock)
cache.set("session:u102", {"user": "Bob"}, ttl_seconds=60, current_timestamp=clock)
cache.set("session:u103", {"user": "Carol"}, ttl_seconds=10, current_timestamp=clock)
print(f"Cache Size at t={clock}: {len(cache)} entries")
print("-" * 65)
# Advance clock to t=1015 (u103 expired)
clock = 1015
val_u101 = cache.get("session:u101", current_timestamp=clock) # Still valid
val_u103 = cache.get("session:u103", current_timestamp=clock) # Expired!
print(f"Lookup u101 (valid): {val_u101}")
print(f"Lookup u103 (expired): {val_u103}")
print(f"Cache Size at t={clock}: {len(cache)} entries")
print("-" * 65)
# Bulk purge at t=1070 (u101 and u102 now expired)
clock = 1070
purged = cache.purge_expired(current_timestamp=clock)
print(f"Bulk Purge at t={clock}: Purged {purged} expired entries.")
print(f"Final Cache Size: {len(cache)} entries")
print("=" * 65)
07 / Anti-Patterns & Production Pitfalls
The Unhandled KeyError on Missing Key Lookup
user = {"username": "dev_alex"}
email = user["email"] # BUG: Raises KeyError!Error Behavior: KeyError: 'email'
email = user.get("email", "no-reply@pylabs.org") # Safe retrieval with fallbackSearching Values Linearly Instead of Keys in O(1)
user_roles = {"usr1": "admin", "usr2": "viewer"}
if "admin" in user_roles: # BUG: Checks KEYS, not values! Evaluates to False!
grant_access()Error Behavior: Logic bug: Fails to find values because 'in d' queries keys.
if "admin" in user_roles.values(): # Correct value search
grant_access()Mutating a Dictionary While Iterating Over It
counts = {"a": 0, "b": 5, "c": 0}
for k, v in counts.items():
if v == 0:
del counts[k] # BUG: Modifying dict during iteration!Error Behavior: RuntimeError: dictionary changed size during iteration
# Iterate over a list snapshot of keys:
for k in list(counts.keys()):
if counts[k] == 0:
del counts[k]The dict.fromkeys() Shared Mutable Object Reference Trap
grid = dict.fromkeys(["row1", "row2"], []) # BUG: Shared list reference!
grid["row1"].append("X")
print(grid) # {'row1': ['X'], 'row2': ['X']} - row2 was mutated!Error Behavior: All keys point to the exact same list instance in memory.
# Use a dictionary comprehension for independent allocations:
grid = {k: [] for k in ["row1", "row2"]}Confusing dict.get() Default with Dictionary Mutation
scores = {}
val = scores.get("alice", 100)
print(scores) # BUG: {} - scores was NOT updated by get()!Error Behavior: The key remains missing in the dictionary.
# Use setdefault() if you want to insert the fallback:
val = scores.setdefault("alice", 100) # Inserts 'alice': 100Overwriting Variable Names with the Built-in dict
dict = {"name": "Alice"} # BUG: Overwriting built-in type 'dict'!
new_dict = dict(a=1) # TypeError: 'dict' object is not callableError Behavior: TypeError: 'dict' object is not callable
user_dict = {"name": "Alice"} # Use distinct, semantic variable names08 / Production Systems & Industry Architecture
🟢 Simple Script: Local Developer Automation Tool
Individual developers and IT administrators automate everyday repetitive workflows.
user = {"id": 101, "name": "Elena", "role": "admin"}
print("User name:", user["name"])A clean standalone script providing fast, local automation with zero dependencies.
🟡 Scaled Service: Backend API Service Integration
SaaS platforms run backend microservices processing requests concurrently.
# Service integration layer
def handle_request(payload: dict) -> dict:
return {"status": "success", "processed": True}A modular service handler integrating validation and structured responses.
🔴 Enterprise Production: High-Throughput Cloud Architecture
CPython’s compact dictionary hash table uses open addressing and pseudo-random probing to guarantee O(1) lookups at scale.
# High-resilience production pattern
import time
def process_stream(events: list) -> int:
processed = 0
for event in events:
if event:
processed += 1
return processedCPython’s compact dictionary hash table uses open addressing and pseudo-random probing to guarantee O(1) lookups at scale.
09 / Engineering Case Study: High-Throughput In-Memory Key-Value Caching
Architecture Analysis: Redis-Style In-Memory Store with LIFO Trimming
In latency-critical distributed architectures, applications avoid relational database bottlenecks by buffering frequently queried objects inside in-memory key-value stores. Python dictionaries are the ideal backbone because CPython compact hash maps guarantee $O(1)$ amortized read and write performance.
1. Avoiding Mutation Invalidation
When evicting expired keys during batch purges, code must never call del d[k] while directly iterating over d.items(). Snapshotting the target keys into an immutable list with[k for k, v in d.items() if expired] prevents catastrophic RuntimeError: dictionary changed size crashes.
2. Constant-Time Capacity Limiting
When cache memory reaches its limit, dict.popitem() evicts the last inserted element in $O(1)$ constant time by popping directly from the tail of CPython's dense entries array, eliminating costly array shifts.
10 / Guided Labs: Hands-On Implementation
Alphanumeric Character Frequency Counter
Write a function `count_alphanumeric_frequencies(text: str) -> dict[str, int]` that counts occurrences of case-insensitive alphanumeric characters in a string. Ignore all spaces, punctuation, and special symbols.
- Normalize the string to lowercase using text.lower().
- Use ch.isalnum() to filter out punctuation and spaces.
- Increment the count using counts[ch] = counts.get(ch, 0) + 1.
def count_alphanumeric_frequencies(text: str) -> dict[str, int]:
# TODO: Return dictionary of character counts
pass
# Test:
# count_alphanumeric_frequencies("Hello, World!") -> {'h': 1, 'e': 1, 'l': 3, 'o': 2, 'w': 1, 'r': 1, 'd': 1}
Inverted Department Grouping Engine
Write a function `invert_and_group(employee_depts: dict[str, str]) -> dict[str, list[str]]` that inverts an employee-to-department mapping. The output dictionary must map each department name to a sorted list of employee names belonging to that department.
- Iterate over key-value pairs using employee_depts.items().
- Use grouped.setdefault(dept, []).append(emp) to initialize and append to employee lists.
- Sort the list of names for deterministic results before returning.
def invert_and_group(employee_depts: dict[str, str]) -> dict[str, list[str]]:
# TODO: Invert mapping and group employees per department
pass
# Test:
# emp = {"Alice": "Eng", "Bob": "Sales", "Carol": "Eng"}
# invert_and_group(emp) -> {"Eng": ["Alice", "Carol"], "Sales": ["Bob"]}
11 / Independent Practice Tasks
Nested Config Flattener
Write a function flatten_dict(d: dict, sep: str = ".") -> dict that transforms a nested dictionary like {"db": {"host": "localhost", "port": 5432}} into a flat dictionary {"db.host": "localhost", "db.port": 5432}.
Multi-Key Composite Indexer
Build an in-memory indexer for user event records. Given a list of event dictionaries, index them by composite tuple key (user_id, event_type) to enable instant $O(1)$ event history lookups.
LRU Cache with Move-to-End
Implement a Least-Recently-Used (LRU) cache using a standard Python dictionary. When a key is read or updated, delete and reinsert it so that next(iter(d)) always yields the oldest key.
12 / Key Takeaways & Quick Cheat Sheet
Core Invariants & Rules
- Python dictionaries provide O(1) average-time lookups, insertions, and deletions backed by compact CPython hash tables.
- Since Python 3.7, dictionaries strictly preserve insertion order by default.
- Keys must be hashable; mutable containers like lists and sets cannot serve as dictionary keys.
- d.get(key, default) avoids KeyError exceptions, while d.setdefault(key, default) inserts the fallback if missing.
- Dictionary view objects (keys(), values(), items()) dynamically reflect mutations without allocating memory copies.
- d.keys() supports set algebra operations (&, |, -, ^) for high-speed key comparisons.
- Never use dict.fromkeys() with mutable default objects like []; use a dictionary comprehension instead.
Cheat Sheet Reference
13 / Interactive Quiz Assessment (17 Questions)
user = {"name": "Sara", "age": 22}
print(user["name"])info = {}
info["score"] = 100
print(len(info))d = {"a": 1, "b": 2}
print("a" in d)d = {"x": 10}
d["x"] = 20
print(d["x"])counts = {"a": 1, "b": 2}
print(sum(counts.values()))d1 = {'a': 1, 'b': 2}
d2 = {'b': 20, 'c': 30}
print(sorted(d1.keys() & d2.keys()))14 / Study Guide & Technical FAQ
Q1:Why does dict.fromkeys(['a', 'b'], []) create a shared list bug?
The second argument to dict.fromkeys() is evaluated once and passed by reference to every key. If you pass a mutable list [], all keys point to the exact same list instance in memory. Modifying d['a'].append(1) mutates d['b'] as well! To avoid this, use a dict comprehension: {k: [] for k in ['a', 'b']}.
Q2:How does CPython preserve dictionary insertion order?
Starting with PEP 468 in Python 3.6/3.7, CPython splits dictionaries into a dense entries array and a sparse indices array. Because items are appended to the dense entries array in the exact order they are inserted, iterating over the entries array naturally preserves insertion order.
Q3:Why is 'val in d.values()' slow compared to 'key in d'?
Checking 'key in d' hashes the key and checks the sparse index table directly in O(1) time. In contrast, values are not indexed by hash codes; finding a value requires CPython to scan every element in the entries table in O(N) linear time.
Q4:How can I safely delete items while iterating over a dictionary?
You cannot mutate a dictionary while iterating over it directly (doing so raises RuntimeError). Instead, create a static snapshot of the keys to delete: for k in [key for key, val in d.items() if condition]: del d[k] or iterate over list(d.keys()).
Worked example
Store and retrieve API response metadata using a dictionary with instant key lookup.
status = {"code": 200, "status": "OK"}
print(f"{status['code']}: {status['status']}")
How it works, step by step
- Define key-value pairs in curly braces.
- Access values by key in O(1) time.
- Format directly in an f-string.
Dictionaries & Hash Maps practice
Define a dictionary metrics = {"cpu": 45, "memory": 72, "disk": 88}. Increase memory by 5, insert "network": 12, and print Memory: 77, Network: 12.
Coding Practice is Locked
Sign in to access the interactive Python editor, write and run code checks against automated tests, use hints, and earn XP.
Practice quiz
Check yourself
15 questions with explanations after submission, so you can correct course as you go.
Dictionaries & Hash Maps quiz
LockedYou must be signed in to take this quiz. Sign in to test your understanding, track your answers, and earn XP.
Lesson incomplete
Not completedCreate an ID or sign in to save lesson completion across devices.