Skip to main content

Module entropy

Module entropy 

Source
Expand description

Shannon entropy of the k-mer distribution at a position, and its bias-corrected estimators.

All estimators work on the counts of counts of a position (CountProfile): how many distinct k-mers were seen once, twice, …. That is all they need, it is tiny, and it makes the cost independent of the k-mer length.

  • plug_in: the observed (maximum-likelihood) entropy H = −Σ pᵢ log₂ pᵢ (Shannon 1948), biased downwards by about (D − 1)/(2N ln 2) bits (Miller 1955; Paninski 2003).
  • dima_corrected: the paper’s correction (Tharanga et al. 2025, Fig. 2) — entropies of subsamples of R % of the sequences are regressed on 1/n and the intercept (n → ∞) is the estimate. The paper draws one random with-replacement subsample per R; because the OLS intercept is linear in the responses, its expectation is the intercept fitted to the expected subsample entropies, which are computed here exactly (no random numbers). It is the conditional expectation of the random intercept given the data: the same mean and no larger variance (Rao 1945; Blackwell 1947), the limit of averaging many random subsets as DiMA’s predecessors did (Khan et al. 2008, 2017; Miotto et al. 2008), and identical inputs always give identical outputs. The expected intercept is never negative (see dima_corrected), so the paper’s negative-intercept rule never applies to it; DiMA v5.0.9 applies it to each random run instead, which raises its long-run average at near-conserved positions.

Every reported entropy is checked to be a finite number of at least 0 bits (estimate): the estimators guarantee it, so a violation is a bug and stops the run as an internal error instead of being replaced by another value (Wilson et al. 2014, Practice 5a; Hunter-Zinck et al. 2021, Rule 5).

  • miller_madow: H + (D − 1)/(2N ln 2) (Miller 1955).
  • chao_wang_jost: Chao et al. (2013), the least biased estimator in their comparison.
  • sample_coverage: estimated sample coverage (Chao & Jost 2012), the probability that the next sequence carries an already-seen k-mer.

Structs§

CountProfile
Counts of counts of the k-mers at one position.
Estimate
The result of estimating the entropy of one position.

Constants§

RAREFACTION_MIN_POINTS
Minimum number of rarefaction points for the regression: DiMA v5.0.9’s (calculate_entropy); the paper states none.
RAREFACTION_START
Lowest subsample percentage of the rarefaction grid (Tharanga et al. 2025, Fig. 2).
RAREFACTION_STEP
Step of the rarefaction grid, in per cent: Fig. 2 ends “…, 90, 95, 100%”; DiMA v5.0.9 steps by 5 from the start.

Functions§

chao_wang_jost
Chao–Wang–Jost (2013) entropy estimate in bits; None when N < 2.
dima_corrected
The dima estimate in bits: the exact expectation of the paper’s procedure (Fig. 2).
estimate
Estimates the entropy of a position with the requested estimator, applying the documented fallbacks (support ≤ T for dima; N = 1 for Miller–Madow and Chao–Wang–Jost).
expected_subsample_entropy
Expected plug-in entropy (bits) of a with-replacement subsample of size sequences drawn from the position’s empirical k-mer distribution: E[H(Multinomial(size, p̂))].
miller_madow
Miller–Madow estimate in bits, H + (D − 1)/(2N ln 2) (Miller 1955); None when N < 2.
plug_in
Plug-in Shannon entropy in bits, H = −Σ (c/N) log₂(c/N) (Shannon 1948); None when N = 0.
rarefaction_grid
The paper’s rarefaction grid for support n and threshold t (Fig. 2), with the step and start rounding of DiMA v5.0.9: R % from max(50, ⌈100·T/N⌉ rounded up to a multiple of 5) to 95 in steps of 5 (the 100% point is added separately). Empty when ⌈100·T/N⌉ > 95.
sample_coverage
Estimated sample coverage Ĉ = 1 − (f₁/N)·A’ (Chao & Jost 2012, Eq. 4a), where A’ = (N−1)f₁/((N−1)f₁ + 2f₂), or (N−1)(f₁−1)/((N−1)(f₁−1) + 2) when f₂ = 0; 1 when f₁ = 0. None when N < 2.
subsample_size
Subsample size ⌈R·N/100⌉ for R % of n sequences (Fig. 2, “Upperbound”).