This is the documentation of DiMA . The newest release is .
This documentation of the development version includes changes that are not released yet. The newest release is .
Numerical accuracy
DiMA computes in IEEE 754 double precision, where every operation rounds to about 16 significant digits. This page measures how much that rounding adds up to at the sizes DiMA is used for, and compares it with the uncertainty that comes from sampling. Analysing round-off in this way, and showing that it is tolerable, is what the field’s software-quality literature asks of scientific software (Darriba et al. 2018; Eddy 2011).
Summary
Section titled “Summary”- Always present, always negligible. Every result carries some rounding error, as in any
floating-point program. It grows with the number of sequences
at a position, most at highly conserved positions (one dominant k-mer), which are most positions of a viral alignment. - Bundled datasets (up to 2,637 sequences): every value is within about
of the exact result, relative to its size. That is the last one or two of the 17 significant digits written to the JSON. - The DiMA paper’s SARS-CoV-2 scale (about 2.2 million sequences per position): a default
run (the
dimaestimate and the observed entropy) is within. At 17.5 million sequences it is within (from the 11th significant digit on). The opt-in Chao–Wang–Jost estimator, whose second term is evaluated exactly (see Chao–Wang–Jost’s second term), stays within a few units in the last place at any size. In absolute terms no error reaches bits. - Far below the sampling error. The rounding error is
to times smaller than the standard deviation of the entropy estimate itself. The paper reports entropies to two decimals. - Results stay deterministic either way. Identical inputs give byte-identical outputs on every platform and thread count (see Numerical details).
- A fix for the remaining last digits was prototyped and measured, and is deferred. It
brings almost every value to within about
, but it changes stored digits without changing any scientific result (see Why the fix is deferred).
Where the rounding error comes from
Section titled “Where the rounding error comes from”Three computations lose accuracy as
- counts, supports and labels are integers;
- an incidence is a single division;
- the conserved-sequence threshold is compared in exact integer arithmetic;
- Clopper–Pearson bounds are within a few units in the last place (2 measured); most of that
comes from
itself, which is computed as instead of (part of the deferred fix below).
| Computation | What happens | Affects |
|---|---|---|
| The entropy term | observed (plug-in) entropy, Miller–Madow, the expected resampled entropies dima estimate | |
| The binomial weights of | the odds | the dima estimate |
| Sample coverage | the subtraction cancels when nearly every k-mer is a singleton, so that coverage is close to 0 | sample_coverage |
Measurements
Section titled “Measurements”How large N gets
Section titled “How large N gets”The DiMA paper downloaded “All 11,808,363 available Spike protein sequence records” of
SARS-CoV-2 (July 2022) and deduplicated them to 2,106,985 sequences. Its most diverse position
had a support of 2,208,211 sequences with 343 distinct 9-mers, 108 of them seen once
(Tharanga et al. 2025). The measurements use that
Method
Section titled “Method”Each value was compared with the same formula evaluated in 50-digit arithmetic (Python
mpmath; for Chao–Wang–Jost, the reference implementation’s 80-digit decimal sum). The count
profiles model real positions:
- one differing sequence: a dominant k-mer and a single singleton, counts
; the most common shape at a conserved position; - near-conserved: a dominant k-mer, one doubleton and four singletons;
- many doubletons: a dominant k-mer, nine doubletons and two singletons, so that
, the hardest case for a term-by-term summation of Chao–Wang–Jost’s second term; - diverse: modelled on the paper’s most diverse position, with 350 distinct k-mers, 100 of
them singletons (
).
Relative error
Section titled “Relative error”The table gives
| Profile | Plug-in | Miller–Madow | dima | Chao–Wang–Jost | Coverage | |
|---|---|---|---|---|---|---|
| one differing sequence | 2,208,211 | exact | ||||
| near-conserved | 2,208,211 | |||||
| many doubletons | 2,208,211 | |||||
| diverse | 2,213,789 | |||||
| one differing sequence | 17,500,000 | exact | ||||
| near-conserved | 17,500,000 | |||||
| many doubletons | 17,500,000 |
The error grows roughly in proportion to
Chao–Wang–Jost stays within a few units in the last place at every size because its second
term is evaluated exactly. A term-by-term summation of that term loses up to 8 digits near
0.014173273093526107.
Compared with the sampling error
Section titled “Compared with the sampling error”An entropy estimated from
| Profile | Entropy (bits) | Standard deviation (bits) | Largest rounding error (bits) | Ratio | |
|---|---|---|---|---|---|
| one differing sequence | 2,208,211 | ||||
| near-conserved | 2,208,211 | ||||
| many doubletons | 2,208,211 | ||||
| diverse | 2,213,789 | 1.47 | |||
| one differing sequence | 17,500,000 | ||||
| near-conserved | 17,500,000 | ||||
| many doubletons | 17,500,000 |
Drawing another sample of the same size would typically move each of these entropies
Sample coverage in real data
Section titled “Sample coverage in real data”The rounding error of sample coverage is about
- In the four bundled datasets, positions where more than half of the sequences carry a k-mer seen only once occur only with 100 sequences or fewer. Coverage there is at least 0.07.
- Among positions with more than 100 sequences, the largest singleton share is 0.39, so coverage is at least 0.61.
- The paper’s most diverse SARS-CoV-2 position has 108 singletons among 2,208,211 sequences, a
share of
. - The expected singleton share
falls as grows.
The bundled datasets
Section titled “The bundled datasets”At the sizes of the bundled datasets the current code is within about
One position of the small test alignment in tests/golden/tiny/ splits seven sequences 4 to 3. Its exact plug-in
entropy is 0.9852281360342514, and
DiMA writes 0.9852281360342515, one unit in the last place above it.
The one limit
Section titled “The one limit”With more than
Reproducing a measurement
Section titled “Reproducing a measurement”The reference implementation in scripts/reference/ (see
Testing) computes the affected terms more accurately: in 60–80-digit
decimal arithmetic, with exact integer odds and log1p. Comparing the two on an alignment
with one of the profiles above reproduces the table. For one differing sequence at
python3 -c "import sys; n = 2208211; sys.stdout.write('>s\nA\n' * (n - 1) + '>s\nC\n')" > one.fadima analyze -i one.fa -a dna -k 1 -o dima.jsonpython3 scripts/reference/dima_reference.py analyze -i one.fa -a dna -k 1 -o ref.jsonobserved_entropy (results[0] against positions[0]) differs by entropy by --estimator miller-madow or
--estimator chao-wang-jost to the dima run gives the Miller–Madow and Chao–Wang–Jost
values, which the reference reports as miller_madow and chao_wang_jost.
Chao–Wang–Jost’s second term
Section titled “Chao–Wang–Jost’s second term”The second term of the Chao–Wang–Jost estimator (Chao et al. 2013) is
DiMA evaluates the same quantity exactly, in a time independent of
- Series (for
and below about 0.56): the expansion of about (Hankin 2015, eq. 15.3.10) gives as a sum of non-negative terms that shrink at least as fast as ; at most 17 terms are needed. Its digamma values come from an asymptotic series whose remainder is smaller than the first omitted term (Alzer 1997, Theorem 8). - Continued fraction (otherwise): Pfaff’s transformation (Hankin 2015, eq. 15.3.4) and Gauss’s continued fraction, which converges to the ratio of hypergeometric functions (Van Vleck 1901, p. 1 and §9(a)), give a continued fraction with positive coefficients at a positive argument. Its value lies between any two consecutive approximants (Henrici & Pflüger 1966, §2), and evaluating an approximant backwards keeps its rounding error bounded (Jones & Thron 1974, Theorem 5.1(i)). At most 817 steps are needed below 64 sequences (every possible count profile was checked) and at most 373 above (measured).
Both stop once the remaining truncation error is below half a unit in the last place. A cap of 2048 steps turns any unforeseen failure to converge into an internal error, never a wrong value or a hang.
Measured against the exact values, every position of a synthetic spike alignment of 2.2 million
sequences is within
What this means for testing
Section titled “What this means for testing”The differential test against the reference implementation
(crates/dima-core/tests/reference.rs) and the paper test (crates/dima-core/tests/paper.rs,
which compares the four bundled datasets with values the reference implementation computed)
share one rule: every number must agree within a relative
At a million or more sequences per position the default estimator’s own rounding reaches
What the fix would be
Section titled “What the fix would be”The fix would make four changes; the first three were prototyped and measured against the 50-digit evaluations:
- the term
for is computed as , where is exact, so nothing is lost to rounding near 1 (Goldberg 1991, Theorem 4); - the binomial odds come from the exact integers
and instead of the rounded ; - coverage uses a form without the cancellation when singletons dominate;
of the conserved-sequence Clopper–Pearson bound is written so that it is exactly , which moves every bound by 1 or 2 units in the last place.
The reference implementation already uses the first two and the exact
Accuracy. On 36 profiles of conserved positions with 200,000 to 17.5 million sequences,
every value came within dima at 10 million sequences, as today.
Speed. The default estimator ran as fast or slightly faster (0.90 to 1.02 times the time).
Outputs. The last one or two digits of many values change. Nothing printed to four decimals, no conserved sequence and no summary line changes. Per bundled dataset, with the default estimator:
| Dataset | Numbers changed | Largest relative change |
|---|---|---|
| MERS-CoV spike protein | 1,446 of 9,390 | |
| MERS-CoV spike gene | 5,582 of 47,086 | |
| HIV-1 pol protein | 456 of 41,156 | |
| HIV-1 pol gene | 1,048 of 800,270 |
Five of the 13 golden output files of the command-line tests would change, and with the exact
Why the fix is deferred
Section titled “Why the fix is deferred”- No scientific result changes. At every realistic
the rounding error is at least times smaller than the sampling error. A fix would change digits that no analysis can rely on. - The project’s rule for engineering changes. The paper is the specification, and a pure engineering change must leave results unchanged (see Working agreements). This fix changes stored digits, so it needs a reason that matters for results, and at realistic scale there is none.
- What the field asks for is analysis, not last-digit accuracy. Darriba et al. ask developers to analyse round-off. They also show two runs that differed only in round-off giving different trees, which likelihood-based significance tests could not tell apart (Darriba et al. 2018). HMMER justifies even reduced precision by a numerical stability analysis showing that the accumulated error is tolerable (Eddy 2011). This page is that analysis.
- Established practice. Tools fix floating-point precision in ordinary releases, for example scikit-bio 0.2.3, bcftools 1.13 and SciPy 1.7.0 by their release notes. They promise identical numbers only for the same version: HMMER’s user guide points to its archived versions “when exact reproducibility of an analysis is an issue”. This is tool behaviour, with no peer-reviewed source. DiMA’s own guarantee is the same: byte-identical results from the same version (see Make results reproducible). A later fix therefore breaks no guarantee; it needs a “Methodology” entry in the changelog.
- The one case with a practical cost is handled. Chao–Wang–Jost’s second term, which would
cost time proportional to
and lose up to 8 digits if summed as written, is evaluated exactly (see Chao–Wang–Jost’s second term).
The fix would be reconsidered when a test at a million or more sequences per position is added, or when values are needed beyond ten significant digits.
References
Section titled “References”- Alzer H. On some inequalities for the gamma and psi functions. Mathematics of Computation. 1997;66(217):373-389. doi:10.1090/S0025-5718-97-00807-7 ↩︎
- Chao A, Wang YT, Jost L. Entropy and the species accumulation curve: a novel entropy estimator via discovery rates of new species. Methods in Ecology and Evolution. 2013;4(11):1091-1100. doi:10.1111/2041-210X.12108 ↩︎
- Darriba D, Flouri T, Stamatakis A. The State of Software for Evolutionary Biology. Molecular Biology and Evolution. 2018;35(5):1037-1046. doi:10.1093/molbev/msy014 ↩︎ 1 2
- Eddy SR. Accelerated Profile HMM Searches. PLoS Computational Biology. 2011;7(10):e1002195. doi:10.1371/journal.pcbi.1002195 ↩︎ 1 2
- Goldberg D. What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys. 1991;23(1):5-48. doi:10.1145/103162.103163 ↩︎ 1 2
- Hankin RKS. Numerical Evaluation of the Gauss Hypergeometric Function with the hypergeo Package. The R Journal. 2015;7(2):81-88. doi:10.32614/RJ-2015-022 ↩︎ 1 2 3
- Henrici P, Pflüger P. Truncation error estimates for Stieltjes fractions. Numerische Mathematik. 1966;9(2):120-138. doi:10.1007/BF02166031 ↩︎
- Jones WB, Thron WJ. Numerical stability in evaluating continued fractions. Mathematics of Computation. 1974;28(127):795-810. doi:10.1090/S0025-5718-1974-0373265-5 ↩︎
- Paninski L. Estimation of Entropy and Mutual Information. Neural Computation. 2003;15(6):1191-1253. doi:10.1162/089976603321780272 ↩︎
- Taschuk M, Wilson G. Ten simple rules for making research software more robust. PLOS Computational Biology. 2017;13(4):e1005412. doi:10.1371/journal.pcbi.1005412 ↩︎
- Tharanga S, Ünlü ES, Hu Y, Sjaugi MF, Çelik MA, Hekimoğlu H, Miotto O, Öncel MM, Khan AM. DiMA: sequence diversity dynamics analyser for viruses. Briefings in Bioinformatics. 2025;26(1):bbae607. doi:10.1093/bib/bbae607 ↩︎
- Van Vleck EB. On the Convergence of the Continued Fraction of Gauss and Other Continued Fractions. The Annals of Mathematics. 1901;3(1/4):1-18. doi:10.2307/1967627 ↩︎