Skip to main content

dima_corrected

Function dima_corrected 

Source
pub fn dima_corrected(
    profile: &CountProfile,
    threshold: u64,
) -> Result<f64, EntropyFallback>
Expand description

The dima estimate in bits: the exact expectation of the paper’s procedure (Fig. 2).

The intercept is never negative: the expected subsample entropy never decreases with the subsample size n (the frequencies of n + 1 draws average those of their n + 1 subsets of n draws, and entropy is concave) and never exceeds the observed entropy (Jensen’s inequality: a with-replacement subsample’s expected frequencies are the observed ones), so the points never rise as 1/n grows, the least-squares slope is at most 0, and the intercept is at least the mean of the points, which is at least 0. The paper’s fallback for a negative intercept therefore never applies; estimate checks the result like every other.

§Errors

The reason the paper’s procedure reports the observed entropy instead: support N ≤ T, fewer than five points, or a degenerate regression.