pub fn dima_corrected(
profile: &CountProfile,
threshold: u64,
) -> Result<f64, EntropyFallback>Expand description
The dima estimate in bits: the exact expectation of the paper’s procedure (Fig. 2).
The intercept is never negative: the expected subsample entropy never decreases with the
subsample size n (the frequencies of n + 1 draws average those of their n + 1 subsets of n
draws, and entropy is concave) and never exceeds the observed entropy (Jensen’s inequality:
a with-replacement subsample’s expected frequencies are the observed ones), so the points
never rise as 1/n grows, the least-squares slope is at most 0, and the intercept is at
least the mean of the points, which is at least 0. The paper’s fallback for a negative
intercept therefore never applies; estimate checks the result like every other.
§Errors
The reason the paper’s procedure reports the observed entropy instead: support N ≤ T, fewer than five points, or a degenerate regression.