data/<block-ulid>/chunks/000001

Chunks

Compressed sample data of a persisted block lives in numbered segment files under the block's chunks/ directory, each at most 512 MiB. A second segment file (000002) only ever appears once the first crosses that limit, so a small block like this one always has exactly one. The index refers to a chunk with a uint64 reference: the upper 4 bytes select the segment file, the lower 4 bytes are the byte offset inside it.

The payoff of this format is the XOR chunk data: with a steady scrape interval and slowly-changing values, a sample costs a couple of bits instead of 16 bytes. Drill into sample 2 and later below to see single-bit fields.

encoding is a single byte per chunk — that's why XOR and native-histogram chunks coexist in the very same segment file below; there's no separate "histogram block".

Open the interactive byte explorer →

Field notes & invariants

Chunk len counts only the data field, not the encoding byte or checksum.
The checksum is CRC-32C (Castagnoli polynomial) over encoding + data, serialized big-endian.
Timestamps are compressed as delta-of-deltas (varbit_ts): a perfectly steady scrape interval costs 1 bit per sample. Buckets: 0 | 10+14b | 110+17b | 1110+20b | 1111+64b.
Values are XORed with the previous value (varbit_xor): unchanged → 1 bit; changed → either reuse the previous leading/trailing-zero window (10 + sigbits) or define a new one (11 + 5b leading + 6b sigbits + bits).
Compare the chunks: the constant up series spends 1+1 bits per repeated sample, while the float counter process_cpu_seconds_total needs ~50 significant XOR bits per change. Same format, wildly different density.
Trailing padding of 0–7 zero bits byte-aligns the chunk data.
Histogram chunks share the outer framing (len/encoding/data/crc) but their data starts with a layout (zero threshold, schema, bucket spans, and — for NHCB schema -53 — custom bucket bounds), then one dod/xor delta chain per sample. See the encoding toggle for a field-by-field walkthrough. The histogramST/floathistogramST/XOR2 encodings additionally carry optional start-timestamp (ST) data.
Chunk references in this segment (sequence 0, so ref = offset): chunk 1 at offset 8 → 0x8, chunk 2 at offset 38 → 0x26, chunk 3 at offset 68 → 0x44, chunk 4 at offset 122 → 0x7A, chunk 5 at offset 199 → 0xC7, chunk 6 at offset 293 → 0x125. The index example on this site points at exactly these 6 chunks. A chunk at offset 8 of segment 000002 would instead have ref 0x100000008 (sequence 1 in the upper 4 bytes).

← All TSDB formats