9. Skip to content

9. Point Layout

The point layout stores data arrays as-is in parallel columns alongside a repeated index column. The top-level schema node is named point and is a group with an arbitrary number of columns. The entity index column MUST be the first column under point. All rows with the same index MUST be contiguous.

Point-layout schema: a single top-level point group holding a repeated spectrum_index column alongside parallel mz and intensity columns, with one table row per data point.

point
spectrum_indexmzintensity
0213.21002
0506.9500
0758405
………
2329.150
2516.55002
2783.8302

This layout is simple, but carries several advantages:

  • Predicate filtering. Scalar columns are easily filtered along the page-level range index, which makes multi-dimensional queries easy to write and optimise.
  • Transparent compression. Arrays are encoded and compressed by Parquet, so the data is still stored compactly.

The trade-off: data MUST be stored as-is to keep the page index meaningful, so no additional obscuring transformations (delta encoding, Numpress, etc.) may be used. The zero-run stripping and null-marking methods remain available, because they only remove non-meaningful points from the array rather than transforming the values that remain.

9.1 Parallel time index

In addition to the <entity_type>_index column, the writer MAY include a parallel <entity_type>_time column which MUST store the scan start time associated with that entity, stored as a scalar float64. This applies naturally to mass spectra, but is redundant for chromatograms or traces. This column is not part of the array index.

9.2 Reading a single entry from the point layout

To read a single entry (spectrum, chromatogram, …) stored in point layout:

  1. Identify which columns are annotated in the array index. The <entity_type>_index column MUST be the first column in the table, so it always has index 0.
  2. Find the row group containing the entry's index value via the row-group metadata. Optionally, if the page index is available, find the row ranges of the pages that contain that index.
  3. Read the selected row group (or page row range) and filter to rows whose <entity_type>_index equals the entry's index.
  4. Read all rows for all columns where the entry's index equals the entry's index. Apply any transforms to finish decoding the data. They may require first reading from the entity's metadata table.
  5. If the entry has additional auxiliary arrays, read and decode them from the metadata table.