data/wal/000004
WAL
Every ingested sample hits the write-ahead log before it is acknowledged,
so a crash cannot lose acknowledged data. Segments are numbered files
(000000, 000001, …) capped at 128 MB, written in
32 KB pages, a framing borrowed from LevelDB/RocksDB. A record that
doesn't fit the rest of a page is split into first/middle/last fragments; records never span
segment boundaries.
Inside the framing, each record type has its own encoding. Series records name and number the series; sample records then refer to them by ID. That's why WAL replay must see series records before their samples, and why checkpoints rewrite them.
Open the interactive byte explorer →
Field notes & invariants
- The framing checksum precedes the data (type, len, CRC, data): the reverse of the block formats.
- Record types: 1 series, 2 samples, 3 tombstones, 4 exemplars, 5 memory-mapped-chunk markers (not in the spec doc), 6 metadata, 7/8 native histograms (integer/float), 9/10 native histograms with custom buckets (NHCB).
- Sample rows delta-encode against the first row: the record stores a raw (id, timestamp) pair once, then per-sample deltas. Values are always raw 8-byte IEEE 754.
- Accuracy note: the spec doc labels the deltas
<uvarint>, but the implementation (record.go) writes them withPutVarint64: zigzag-encoded signed varints. This example encodes them the way the code does. - Native histogram samples get their own record types (
RecordHistogramSamples/RecordFloatHistogramSamples, plus NHCB variants) rather than reusing the classic samples record — each row carries a full schema/layout/bucket set, not just a value. Unlike the chunk encoding, WAL histogram samples are not delta-coded against each other: only the leading (id, timestamp) pair delta-codes against the record's first row. - Compression (snappy or zstd, per the flag bits) applies to the record data only, never the framing.
- A checkpoint directory (
checkpoint.00000N) is a compacted WAL: dropped series and pre-checkpoint samples removed, then old segments deleted.