Skip to content

TSV tables

DiMA writes four tidy tables: one observation per row, one variable per column, one table per kind of observation (Wickham 2014; Broman & Woo 2018). They hold the same values as the JSON results, arranged for R, pandas, spreadsheets and command-line tools.

TableOne row perWritten by
positionsposition--tsv-dir DIR, or -o FILE.tsv --table positions
variantsvariant at a position--tsv-dir DIR, or -o FILE.tsv --table variants
metadatavariant × header field × value--tsv-dir DIR, or -o FILE.tsv --table metadata
hcsconserved region--tsv-dir DIR, -o FILE.tsv --table hcs, --hcs-output FILE.tsv, dima hcs -O tsv

With --tsv-dir DIR (created if needed), the four files are named <name>.positions.tsv, <name>.variants.tsv, <name>.metadata.tsv and <name>.hcs.tsv, where <name> is the query name with characters other than letters, digits, ., - and _ replaced by _. dima view converts existing results the same way.

  • Tab-separated, UTF-8, one header row, lines ending in a line feed.
  • Quoting follows the CSV specification with a tab delimiter (RFC 4180): a field is quoted with " only when it contains a tab, a quote or a line break, and quotes inside it are doubled. Header field names and values (in the metadata table) are the only fields that can need it.
  • NA marks an undefined value (null in JSON). Only the columns marked “or NA” below can hold it.
  • Numbers are written in their shortest exact form (for example 75.88235294117646); booleans as true and false.
  • Files written with -o can be compressed by name (.tsv.gz, .tsv.zst); --tsv-dir writes plain files.
ColumnTypeMeaning
positioninteger1-based position (first column of the k-mer)
low_supportNS, LS, ELS or NASupport class; NA when eligible (support > T )
supportintegerSequences with a valid k-mer
distinct_variants_countintegerDistinct non-Index variants
entropynumber or NAEntropy in bits by entropy_method; NA without support
observed_entropynumber or NAObserved (plug-in) entropy; NA without support
entropy_methodtextdima, chao_wang_jost, miller_madow, plug_in, not_estimable
entropy_fallbacktext or NAWhy the requested estimator was not used
distinct_variants_incidencenumber or NANon-Index variants per 100 non-Index sequences
total_variants_incidencenumber or NANon-Index sequences per 100 supporting sequences
sample_coveragenumber or NAEstimated sample coverage (NA when support < 2)
below_min_coveragetrue, false or NACoverage below --min-coverage (NA without the option)
ColumnTypeMeaning
positionintegerPosition
sequencetextThe k-mer
countintegerSequences carrying it
incidencenumbercount / support × 100
motif_shortI, Ma, Mi, UMotif class
motif_longIndex, Major, Minor, UniqueMotif class, spelled out

Rows are ordered by position, then as in the JSON (count descending, then sequence).

Empty (header row only) without --header-format.

ColumnTypeMeaning
positionintegerPosition
sequencetextThe k-mer
fieldtextStratified header field, in header-format order
valuetextCleaned header value (never empty: empty values get the fill value)
countintegerSequences with this k-mer at this position and this value

The counts of one variant and one field add up to the variant’s count. This table grows with positions × variants × values: stratifying by an identifier-like field (an accession) adds one row per sequence for every variant; see Stratify by header metadata.

ColumnTypeMeaning
region_idintegerRegion number
start_column, end_column, lengthintegersAlignment columns covered
min_incidencenumberLowest Index incidence among the members (%)
min_supportintegerLowest support among the members
conservation_lower_bound_95numberLowest exact 95% Clopper–Pearson lower bound (proportion)
flaggedtrue, falseThe region includes positions with support ≤ T
low_support_position_countintegerHow many
sequencetextThe conserved sequence

The member positions and the list of low-support positions are in the JSON output. In dima analyze the table follows --hcs-min-incidence and --hcs-eligible-only; dima view always uses the defaults (100%, all positions), and dima hcs any other setting.