Skip to content

Numerical accuracy

DiMA computes in IEEE 754 double precision, where every operation rounds to about 16 significant digits. This page measures how much that rounding adds up to at the sizes DiMA is used for, and compares it with the uncertainty that comes from sampling. Analysing round-off in this way, and showing that it is tolerable, is what the field’s software-quality literature asks of scientific software (Darriba et al. 2018; Eddy 2011).

  • Always present, always negligible. Every result carries some rounding error, as in any floating-point program. It grows with the number of sequences N at a position, most at highly conserved positions (one dominant k-mer), which are most positions of a viral alignment.
  • Bundled datasets (up to 2,637 sequences): every value is within about 5 × 10 − 15 of the exact result, relative to its size. That is the last one or two of the 17 significant digits written to the JSON.
  • The DiMA paper’s SARS-CoV-2 scale (about 2.2 million sequences per position): a default run (the dima estimate and the observed entropy) is within 2.2 × 10 − 12 . At 17.5 million sequences it is within 5.1 × 10 − 11 (from the 11th significant digit on). The opt-in Chao–Wang–Jost estimator, whose second term is evaluated exactly (see Chao–Wang–Jost’s second term), stays within a few units in the last place at any size. In absolute terms no error reaches 10 − 14 bits.
  • Far below the sampling error. The rounding error is 10 10 to 10 12 times smaller than the standard deviation of the entropy estimate itself. The paper reports entropies to two decimals.
  • Results stay deterministic either way. Identical inputs give byte-identical outputs on every platform and thread count (see Numerical details).
  • A fix for the remaining last digits was prototyped and measured, and is deferred. It brings almost every value to within about 10 − 15 , but it changes stored digits without changing any scientific result (see Why the fix is deferred).

Three computations lose accuracy as N grows. Chao–Wang–Jost’s second term would be a fourth if it were summed as written; DiMA evaluates it exactly instead (see Chao–Wang–Jost’s second term). Everything else is exact, rounded once, or measured to be accurate:

  • counts, supports and labels are integers;
  • an incidence is a single division;
  • the conserved-sequence threshold is compared in exact integer arithmetic;
  • Clopper–Pearson bounds are within a few units in the last place (2 measured); most of that comes from α / 2 itself, which is computed as ( 1 − 0.95 ) / 2 = 0.025000000000000022 instead of 0.025 (part of the deferred fix below).
ComputationWhat happensAffects
The entropy term − p log 2 ⁡ p of a k-mer whose count c is close to N p = c / N is rounded before the logarithm is taken, and the logarithm of a number close to 1 loses the low-order digits of its distance from 1 (Goldberg 1991, §1.4 and Theorem 4), which magnifies the rounding by about N / ( N − c ) observed (plug-in) entropy, Miller–Madow, the expected resampled entropies E [ H n ] and so the dima estimate
The binomial weights of E [ H n ] the odds p / ( 1 − p ) are formed from the rounded p , and 1 − p loses accuracy in the same waythe dima estimate
Sample coverage 1 − ( f 1 / N ) A ′ the subtraction cancels when nearly every k-mer is a singleton, so that coverage is close to 0sample_coverage

The DiMA paper downloaded “All 11,808,363 available Spike protein sequence records” of SARS-CoV-2 (July 2022) and deduplicated them to 2,106,985 sequences. Its most diverse position had a support of 2,208,211 sequences with 343 distinct 9-mers, 108 of them seen once (Tharanga et al. 2025). The measurements use that N , and 17.5 million sequences: about 1.5 times the paper’s raw records, to allow for growth and for analyses without deduplication.

Each value was compared with the same formula evaluated in 50-digit arithmetic (Python mpmath; for Chao–Wang–Jost, the reference implementation’s 80-digit decimal sum). The count profiles model real positions:

  • one differing sequence: a dominant k-mer and a single singleton, counts ( N − 1 , 1 ) ; the most common shape at a conserved position;
  • near-conserved: a dominant k-mer, one doubleton and four singletons;
  • many doubletons: a dominant k-mer, nine doubletons and two singletons, so that A N ≈ 9 , the hardest case for a term-by-term summation of Chao–Wang–Jost’s second term;
  • diverse: modelled on the paper’s most diverse position, with 350 distinct k-mers, 100 of them singletons ( N = 2,213,789 ).

The table gives | computed − exact | / exact :

Profile N Plug-inMiller–MadowdimaChao–Wang–JostCoverage
one differing sequence2,208,211 2.2 × 10 − 12 2.1 × 10 − 12 1.3 × 10 − 12 1.3 × 10 − 16 exact
near-conserved2,208,211 4.3 × 10 − 13 4.1 × 10 − 13 6.7 × 10 − 13 3.7 × 10 − 17 5.5 × 10 − 17
many doubletons2,208,211 1.6 × 10 − 13 1.6 × 10 − 13 2.5 × 10 − 13 3.1 × 10 − 17 8.4 × 10 − 17
diverse2,213,789 2.1 × 10 − 17 1.0 × 10 − 16 4.2 × 10 − 15 4.7 × 10 − 17 2.7 × 10 − 17
one differing sequence17,500,000 3.0 × 10 − 11 2.9 × 10 − 11 5.1 × 10 − 11 2.3 × 10 − 18 exact
near-conserved17,500,000 7.0 × 10 − 12 6.8 × 10 − 12 6.7 × 10 − 12 6.2 × 10 − 17 4.1 × 10 − 17
many doubletons17,500,000 2.4 × 10 − 12 2.3 × 10 − 12 3.3 × 10 − 12 1.8 × 10 − 16 1.0 × 10 − 17

The error grows roughly in proportion to N . For one differing sequence the plug-in error is 2.0 × 10 − 16 at 1,000 sequences, 3.6 × 10 − 13 at 100,000 and 3.1 × 10 − 11 at 10 million. Positions where no k-mer dominates stay between 10 − 15 and 10 − 14 even at 10 million sequences.

Chao–Wang–Jost stays within a few units in the last place at every size because its second term is evaluated exactly. A term-by-term summation of that term loses up to 8 digits near A N = 9 : 4.6 × 10 − 9 on a profile with 17.5 million sequences (one dominant k-mer, 1,000 singletons and 4,450 doubletons), the worst of 2,427 profiles searched. There the exact value is 0.014173273093526109 bits and DiMA writes 0.014173273093526107.

An entropy estimated from N sequences has a standard deviation of about σ / N , with σ 2 = ∑ i p i ( − log 2 ⁡ p i − H ) 2 ; the plug-in and Miller–Madow estimators share this rate (Paninski 2003, Theorem 2). The last column divides the largest absolute rounding error of any estimator by that standard deviation:

Profile N Entropy (bits)Standard deviation (bits)Largest rounding error (bits)Ratio
one differing sequence2,208,211 1.0 × 10 − 5 9.5 × 10 − 6 2.2 × 10 − 17 2 × 10 − 12
near-conserved2,208,211 6.0 × 10 − 5 2.3 × 10 − 5 4.0 × 10 − 17 2 × 10 − 12
many doubletons2,208,211 2.0 × 10 − 4 4.1 × 10 − 5 5.0 × 10 − 17 1 × 10 − 12
diverse2,213,7891.47 1.1 × 10 − 3 6.2 × 10 − 15 6 × 10 − 12
one differing sequence17,500,000 1.5 × 10 − 6 1.4 × 10 − 6 7.4 × 10 − 17 5 × 10 − 11
near-conserved17,500,000 8.6 × 10 − 6 3.3 × 10 − 6 6.0 × 10 − 17 2 × 10 − 11
many doubletons17,500,000 2.8 × 10 − 5 5.9 × 10 − 6 9.3 × 10 − 17 2 × 10 − 11

Drawing another sample of the same size would typically move each of these entropies 10 10 to 10 12 times further than rounding does.

The rounding error of sample coverage is about 10 − 16 / C ^ , so it matters only when coverage C ^ is close to 0. That requires nearly every k-mer at a position to be a singleton. An error of 10 − 9 , for example, needs a coverage below 10 − 7 , which at 10 million sequences means more than 99.99999% singletons. Real alignments are nowhere near this:

  • In the four bundled datasets, positions where more than half of the sequences carry a k-mer seen only once occur only with 100 sequences or fewer. Coverage there is at least 0.07.
  • Among positions with more than 100 sequences, the largest singleton share is 0.39, so coverage is at least 0.61.
  • The paper’s most diverse SARS-CoV-2 position has 108 singletons among 2,208,211 sequences, a share of 4.9 × 10 − 5 .
  • The expected singleton share ∑ i p i ( 1 − p i ) N − 1 falls as N grows.

At the sizes of the bundled datasets the current code is within about 5 × 10 − 15 of the exact values, and Chao–Wang–Jost within 4.2 × 10 − 16 (at most 3 units in the last place).

One position of the small test alignment in tests/golden/tiny/ splits seven sequences 4 to 3. Its exact plug-in entropy is 0.985228136034251458 … bits. The nearest double is 0.9852281360342514, and DiMA writes 0.9852281360342515, one unit in the last place above it.

With more than 2 53 ≈ 9 × 10 15 sequences at a position, a ratio such as c / N can round to exactly 1, and an expected entropy can become undefined. DiMA checks that every reported entropy is a finite number of at least 0 bits (see Design rationale). Such a run therefore stops with an internal error (exit 1) instead of writing a wrong value. That limit is about 760 million times the 11.8 million spike records of 2022.

The reference implementation in scripts/reference/ (see Testing) computes the affected terms more accurately: in 60–80-digit decimal arithmetic, with exact integer odds and log1p. Comparing the two on an alignment with one of the profiles above reproduces the table. For one differing sequence at N = 2,208,211 :

Terminal window
python3 -c "import sys; n = 2208211; sys.stdout.write('>s\nA\n' * (n - 1) + '>s\nC\n')" > one.fa
dima analyze -i one.fa -a dna -k 1 -o dima.json
python3 scripts/reference/dima_reference.py analyze -i one.fa -a dna -k 1 -o ref.json

observed_entropy (results[0] against positions[0]) differs by 2.2 × 10 − 12 relative, and entropy by 1.3 × 10 − 12 . Adding --estimator miller-madow or --estimator chao-wang-jost to the dima run gives the Miller–Madow and Chao–Wang–Jost values, which the reference reports as miller_madow and chao_wang_jost.

The second term of the Chao–Wang–Jost estimator (Chao et al. 2013) is f 1 N ( 1 − A ) 1 − N ( − ln ⁡ A − ∑ r = 1 N − 1 ( 1 − A ) r / r ) . Evaluated as written in double precision it can fail completely: ( 1 − A ) 1 − N reaches about 10 17 and multiplies a difference of nearly equal numbers. For 1,000 sequences with one singleton and 20 doubletons it gives 0.62 nats instead of 2.4 × 10 − 5 (measured). Summing it term by term avoids that, but costs time proportional to N at every position with singletons (6.4 ms per position at 2.2 million sequences, about 10% of a whole run at any size) and still loses up to 8 digits near A N = 9 .

DiMA evaluates the same quantity exactly, in a time independent of N . Because − ln ⁡ A = ∑ r ≥ 1 ( 1 − A ) r / r , the term equals f 1 N ( 1 − A ) S with S = ∑ j ≥ 0 ( 1 − A ) j / ( N + j ) = 2 F 1 ( 1 , N ; N + 1 ; 1 − A ) / N , which follows from the series of the hypergeometric function (Hankin 2015, eq. 15.1.1). With x = N A :

  • Series (for N ≥ 64 and x below about 0.56): the expansion of 2 F 1 ( a , b ; a + b ; z ) about z = 1 (Hankin 2015, eq. 15.3.10) gives S as a sum of non-negative terms that shrink at least as fast as x k ; at most 17 terms are needed. Its digamma values come from an asymptotic series whose remainder is smaller than the first omitted term (Alzer 1997, Theorem 8).
  • Continued fraction (otherwise): Pfaff’s transformation (Hankin 2015, eq. 15.3.4) and Gauss’s continued fraction, which converges to the ratio of hypergeometric functions (Van Vleck 1901, p. 1 and §9(a)), give a continued fraction with positive coefficients at a positive argument. Its value lies between any two consecutive approximants (Henrici & Pflüger 1966, §2), and evaluating an approximant backwards keeps its rounding error bounded (Jones & Thron 1974, Theorem 5.1(i)). At most 817 steps are needed below 64 sequences (every possible count profile was checked) and at most 373 above (measured).

Both stop once the remaining truncation error is below half a unit in the last place. A cap of 2048 steps turns any unforeseen failure to converge into an internal error, never a wrong value or a hang.

Measured against the exact values, every position of a synthetic spike alignment of 2.2 million sequences is within 3.7 × 10 − 16 (at most 3 units in the last place), and every position of the bundled datasets within 4.2 × 10 − 16 . The term no longer costs anything noticeable: on that alignment the statistics phase went from 7.97 s to 0.01 s of an 84 s single-threaded run. Below 64 sequences a position can take up to about 11 µs, at most 0.26 s on an extreme alignment of 63 sequences with 30,000 positions. Only the last digits of Chao–Wang–Jost results depend on this; every other estimator is unaffected.

The differential test against the reference implementation (crates/dima-core/tests/reference.rs) and the paper test (crates/dima-core/tests/paper.rs, which compares the four bundled datasets with values the reference implementation computed) share one rule: every number must agree within a relative 10 − 13 , so a zero must be exactly zero. Their alignments have at most a few thousand sequences per position; the largest difference measured is 2.6 × 10 − 15 , a last-bit difference in a platform’s logarithm moves the reference by at most 6 × 10 − 16 , and every legitimate code change measured (summation order, a one-pass regression, the deferred fix below) moved the values by at most 8 × 10 − 15 . The bound leaves a wide margin above all of these, yet catches a numerical regression as small as 1.2 × 10 − 13 (a binomial sum stopped too early). Floating-point issues make small output differences unavoidable during development, and where results cannot be made identical the accepted tolerance belongs in the documentation and the tests (Taschuk & Wilson 2017, Rule 10).

At a million or more sequences per position the default estimator’s own rounding reaches 10 − 12 to 5 × 10 − 11 relative, so a test at that scale would need a looser rule or the deferred fix below. No test runs at that scale.

The fix would make four changes; the first three were prototyped and measured against the 50-digit evaluations:

  • the term − p log 2 ⁡ p for p ≥ 1 / 2 is computed as − p log 1 p ( ( c − N ) / N ) / ln ⁡ 2 , where c − N is exact, so nothing is lost to rounding near 1 (Goldberg 1991, Theorem 4);
  • the binomial odds come from the exact integers c and N − c instead of the rounded p ;
  • coverage uses a form without the cancellation when singletons dominate;
  • α / 2 of the conserved-sequence Clopper–Pearson bound is written so that it is exactly 0.025 , which moves every bound by 1 or 2 units in the last place.

The reference implementation already uses the first two and the exact α / 2 .

Accuracy. On 36 profiles of conserved positions with 200,000 to 17.5 million sequences, every value came within 1.7 × 10 − 15 of the exact values; today 11 to 15 of the values per estimator are off by more than 10 − 13 . Positions where no k-mer dominates keep about 10 − 14 for dima at 10 million sequences, as today.

Speed. The default estimator ran as fast or slightly faster (0.90 to 1.02 times the time).

Outputs. The last one or two digits of many values change. Nothing printed to four decimals, no conserved sequence and no summary line changes. Per bundled dataset, with the default estimator:

DatasetNumbers changedLargest relative change
MERS-CoV spike protein1,446 of 9,390 2.0 × 10 − 15
MERS-CoV spike gene5,582 of 47,086 3.4 × 10 − 15
HIV-1 pol protein456 of 41,156 1.1 × 10 − 15
HIV-1 pol gene1,048 of 800,270 6.1 × 10 − 16

Five of the 13 golden output files of the command-line tests would change, and with the exact α / 2 also the conserved-sequence tables.

  1. No scientific result changes. At every realistic N the rounding error is at least 10 10 times smaller than the sampling error. A fix would change digits that no analysis can rely on.
  2. The project’s rule for engineering changes. The paper is the specification, and a pure engineering change must leave results unchanged (see Working agreements). This fix changes stored digits, so it needs a reason that matters for results, and at realistic scale there is none.
  3. What the field asks for is analysis, not last-digit accuracy. Darriba et al. ask developers to analyse round-off. They also show two runs that differed only in round-off giving different trees, which likelihood-based significance tests could not tell apart (Darriba et al. 2018). HMMER justifies even reduced precision by a numerical stability analysis showing that the accumulated error is tolerable (Eddy 2011). This page is that analysis.
  4. Established practice. Tools fix floating-point precision in ordinary releases, for example scikit-bio 0.2.3, bcftools 1.13 and SciPy 1.7.0 by their release notes. They promise identical numbers only for the same version: HMMER’s user guide points to its archived versions “when exact reproducibility of an analysis is an issue”. This is tool behaviour, with no peer-reviewed source. DiMA’s own guarantee is the same: byte-identical results from the same version (see Make results reproducible). A later fix therefore breaks no guarantee; it needs a “Methodology” entry in the changelog.
  5. The one case with a practical cost is handled. Chao–Wang–Jost’s second term, which would cost time proportional to N and lose up to 8 digits if summed as written, is evaluated exactly (see Chao–Wang–Jost’s second term).

The fix would be reconsidered when a test at a million or more sequences per position is added, or when values are needed beyond ten significant digits.

  • Alzer H. On some inequalities for the gamma and psi functions. Mathematics of Computation. 1997;66(217):373-389. doi:10.1090/S0025-5718-97-00807-7 ↩︎
  • Chao A, Wang YT, Jost L. Entropy and the species accumulation curve: a novel entropy estimator via discovery rates of new species. Methods in Ecology and Evolution. 2013;4(11):1091-1100. doi:10.1111/2041-210X.12108 ↩︎
  • Darriba D, Flouri T, Stamatakis A. The State of Software for Evolutionary Biology. Molecular Biology and Evolution. 2018;35(5):1037-1046. doi:10.1093/molbev/msy014 ↩︎ 1 2
  • Eddy SR. Accelerated Profile HMM Searches. PLoS Computational Biology. 2011;7(10):e1002195. doi:10.1371/journal.pcbi.1002195 ↩︎ 1 2
  • Goldberg D. What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys. 1991;23(1):5-48. doi:10.1145/103162.103163 ↩︎ 1 2
  • Hankin RKS. Numerical Evaluation of the Gauss Hypergeometric Function with the hypergeo Package. The R Journal. 2015;7(2):81-88. doi:10.32614/RJ-2015-022 ↩︎ 1 2 3
  • Henrici P, Pflüger P. Truncation error estimates for Stieltjes fractions. Numerische Mathematik. 1966;9(2):120-138. doi:10.1007/BF02166031 ↩︎
  • Jones WB, Thron WJ. Numerical stability in evaluating continued fractions. Mathematics of Computation. 1974;28(127):795-810. doi:10.1090/S0025-5718-1974-0373265-5 ↩︎
  • Paninski L. Estimation of Entropy and Mutual Information. Neural Computation. 2003;15(6):1191-1253. doi:10.1162/089976603321780272 ↩︎
  • Taschuk M, Wilson G. Ten simple rules for making research software more robust. PLOS Computational Biology. 2017;13(4):e1005412. doi:10.1371/journal.pcbi.1005412 ↩︎
  • Tharanga S, Ünlü ES, Hu Y, Sjaugi MF, Çelik MA, Hekimoğlu H, Miotto O, Öncel MM, Khan AM. DiMA: sequence diversity dynamics analyser for viruses. Briefings in Bioinformatics. 2025;26(1):bbae607. doi:10.1093/bib/bbae607 ↩︎
  • Van Vleck EB. On the Convergence of the Continued Fraction of Gauss and Other Continued Fractions. The Annals of Mathematics. 1901;3(1/4):1-18. doi:10.2307/1967627 ↩︎