26. Skip to content

26. Spectra

Spectra are described by up to three files: a signal file (spectra_data.parquet), an optional peaks file (spectra_peaks.parquet), and a set of metadata files described below.

26.1 Signal data disposition

When writing signal data, profile data goes in spectra_data.parquet and centroid data goes in spectra_peaks.parquet, always. When a file contains both for the same spectrum, both files are present. The spectrum_metadata.parquet row carries both a number_of_data_points (MS:1003060) and a number_of_peaks (MS:1003059) column so a reader knows which file(s) to read. Absence of either of these columns implies that that modality does not exist. A reader SHOULD allow the caller to indicating which representation the it wants.

For timsTOF-style data that is centroid in m/z but profiled in ion mobility, the consensus is to treat it as centroid for the mass-spectrum dimension and place it in spectra_peaks.parquet. The presence of the metadata column ion_mobility_frame_representation (MS:1003439) SHOULD tell the reader if the ion mobility centroids have been pre-picked or not.

26.2 Spectrum signal data - spectra_data.parquet

{
  "name": "spectra_data.parquet",
  "entity_type": "spectrum",
  "data_kind": "data_arrays"
}

The spectrum signal data is encoded using either point layout or chunked layout. The entity index column MUST be named spectrum_index, and a co-located time column, if written, SHOULD be named spectrum_time. Non-mass spectra (UV, DAD) belong in wavelength_spectra_data.parquet.

When using null marking, follow the null semantics for signal data carefully for profile data.

Profile only

Only profile spectra are written here. Centroid spectra — including centroid views of profile spectra when both modes are stored — MUST instead be written to 26.3 spectra_peaks.parquet. The number of points written here for a spectrum MUST be recorded in the number_of_data_points (MS:1003060) column of spectra_metadata.parquet, to support read planning.

Column Encoding
spectrum_index delta encoding — ideal for repetitive or slowly increasing integers.
spectrum_time byte stream split
m/z arrays byte stream split (byte shuffling), or RLE dictionary when there is ion-mobility data.
ion-mobility arrays RLE dictionary; byte shuffling tends not to help. Consider increasing the dictionary page size.
intensity arrays When stored as integers as some instruments do, byte stream split (byte shuffling) can efficiently reduce size, but shows limited benefit for floating point storage

When grid encoding is used on any array, byte stream split on the grid indices is usually the best choice.

26.3 Spectrum peak data - spectra_peaks.parquet

{
  "name": "spectra_peaks.parquet",
  "entity_type": "spectrum",
  "data_kind": "peaks"
}

The spectrum peak lists, stored separately from the raw signal in spectra_data.parquet. The entity index column MUST be named spectrum_index, and a co-located time column, if written, SHOULD be named spectrum_time. Any centroid spectra MUST be written here, not to spectra_data.parquet. The number of peaks written for a spectrum MUST be recorded in the number_of_peaks (MS:1003059) column of spectra_metadata.parquet, to support read planning.

26.4 Spectrum metadata - spectra_metadata.parquet

{
  "name": "spectra_metadata.parquet",
  "entity_type": "spectrum",
  "data_kind": "metadata"
}

This table uses the metadata table layout.

26.5 Spectrum scan metadata - spectra_metadata_scans.parquet

{
  "name": "spectra_metadata_scans.parquet",
  "entity_type": "spectrum",
  "data_kind": "scans"
}

This table uses the metadata table layout.

A scan or acquisition from the original raw file used to create a spectrum.

26.6 Spectrum precursor metadata - spectra_metadata_precursors.parquet

{
  "name": "spectra_metadata_scans.parquet",
  "entity_type": "spectrum",
  "data_kind": "precursors"
}

This table uses the metadata table layout.

The method of precursor-ion selection and activation. This table describes the spectrum its source_index references and originates from the spectrum its precursor_index references. When higher MSn is done, the writer MUST replicate the chain of selected ions from previous MS levels using the same source_index but each previous MS level's precursor_index. If file does not contain the precursor spectrum, the precursor_index MUST be null. Precursors SHOULD be stored in descending MS level order.

Selected ion to precursor linkage

26.7 Spectrum selected ion metadata - spectra_metadata_selected_ions.parquet

{
  "name": "spectra_metadata_selected_ions.parquet",
  "entity_type": "spectrum",
  "data_kind": "selected_ions"
}

This table uses the metadata table layout.

An ion isolated for dissociation. This table follows the same relationship as the 26.6 precursor table. In addition, there MAY be multiple selected ions from the same precursor selection with co-isolation. If this is the case, writers SHOULD write the selected_ion_index column to allow unique referencing to each selected ion.

  • source_index (uint64) — the spectrum this selected ion belongs to (foreign key).
  • precursor_index (uint64) — the spectrum the selected ion was created from (foreign key). This MAY be null when the precursor spectrum is not included in the written file.
  • selected_ion_index (uint64) — optional index value unique to this selected ion.
  • ion_mobility_value (float64) / ion_mobility_type (CURIE) — See the 26.5 scan.ion_mobility_value for details storing scalar values. If multiple ion mobility values are available for the selected ion that have been combined, but no ion mobility centroid is available as when a ramp has been used, report multiple selected_ion records, one for the ramp start and one for the ramp end.
  • parameters (list) — controlled or uncontrolled parameters; see the parameters list.
  • MUST supply a child of MS:1000455 (ion selection attribute) one or more times — e.g. selected-ion m/z, charge state, intensity.

26.8 Spectrum product selection metadata — spectra_metadata_products.parquet

{
  "name": "spectra_metadata_products.parquet",
  "entity_type": "spectrum",
  "data_kind": "products"
}

This table uses the metadata table layout.

When describing single reaction monitoring (SRM) or multiple reaction monitoring (MRM) experiments, each product ion is isolated separately with a different isolation window. This table is usually empty or absent

  • source_index (integer) — the spectrum this product belongs to (foreign key).
  • product_index (integer) — the ascending 0-based index, incrementing by 1 per entry. This number uniquely identifies each product ion selection across all rows.
  • isolation_window (group) — the isolation/selection window for this product ion, like the Q3 transmission window on a triple-quadrupole instrument.
  • parameters (list) — controlled or uncontrolled parameters; see the parameters list.