Skip to content

Design rationale

The methodology pages justify every scientific choice. This page does the same for the engineering and user-experience choices. Sources follow the project’s citation policy: peer-reviewed work in bioinformatics and computational biology first; general human–computer interaction, visualisation, statistics, mathematics (including numerical analysis) or computer-science sources only where they are the primary source of a number, finding or result, marked [general]; standards and specifications, cited for what they define, marked [standard]. Where no peer-reviewed source exists, the page says so (gap) and names the evidence used instead.

ChoiceRationale and sources
Option names follow the paper’s terms (--support-threshold, --kmer-length, --alphabet), every option has a documented default, and -h gives a summary while --help explains each option with its referencesUsable command-line tools print help, use standard switches and need few mandatory parameters (Seemann 2013; List et al. 2017, Rule 5).
All options and outputs are checked before the input is read; conflicting or ambiguous outputs are refused (for example -O against the file extension)Users make mistakes, and a mistake should cost seconds, not an hour of computation (List et al. 2017, Rule 6).
Errors name the file, record, line, column and byte, and end with a hint:Error messages should provide solutions or point to more information (Lee 2018, Rule 9).
Recoverable conditions are reported once per kind, with a count, examples and a hint, in the results (run.diagnostics) and on standard errorLogging should tell users what happened without flooding them (List et al. 2017, Rule 7); recording it with the results keeps the provenance complete (Wilkinson et al. 2016) [general].
Quiet by default (warnings, errors, one summary line); -q errors only; -v progress milestones and a performance report(List et al. 2017, Rule 7; Seemann 2013).
Colour only reinforces text (error:, warning[code]:); NO_COLOR and --color are honouredColour must not be the only visual means of conveying information (WCAG 2.2, SC 1.4.1) [standard].
Progress is one plain ASCII line on a terminal (counting k-mers: 62%, 1652 sequences, 5.6 MB of 9.1 MB), redrawn in place only when its whole percentage changes or a second passes; --progress plain (or DIMA_PROGRESS=plain) appends a line per phase and the progress at most every 10 seconds and at least once a minute, for screen readers and logs (and in logs at -v); --progress off shows nothingProgress display is a valued usability feature of sequence tools (Shen et al. 2024); progress belongs on standard error (Seemann 2013); users rely on progress indicators to see that an operation is proceeding (Harrison et al. 2007) [general]. Screen readers read the characters of graphical bars aloud and every redraw, and every command should still report its status and progress (Sampath et al. 2021, §5.3.2 and Recommendations 5 and 6) [general], hence the appended plain lines (as Snakemake and minimap2 print by default; tool behaviour, no peer-reviewed source); measured with NVDA’s own diffing and eSpeak, a text line redrawn per percent queues about a sixth of the speech of a 10 Hz bar. Content that updates by itself needs a way to stop it (WCAG2ICT, SC 2.2.2) [standard]. No escape sequences and no box-drawing glyphs: they break under TERM=dumb, the C locale and Windows consoles.
The end-of-run line gives the results and the time (analysed 191 sequences, 1345 positions (1345 eligible) in 0.21 s; entropy mean 0.1261, min 0.0000, max 1.5878 bits), adds N warnings (see above) only when there are warnings, and is the last lineThe paper reports exactly these quantities (Tharanga et al. 2025, Results); tools should say what they did (Taschuk & Wilson 2017, Rules 9 and 10; Seemann 2013). A count that is shown even at zero becomes a habituated token that is ignored when it changes (Ancker et al. 2017) [general]; (van der Sijs et al. 2006) [general].
Numbers in messages are plain digits (2637, not 2,637); durations and sizes use unit symbols (s, min, h, kB, MB), rounded before the unit is chosenDigit grouping with commas is ambiguous across locales: the SI allows the comma as the decimal marker, never puts commas or dots between groups of digits, and makes grouping a matter of choice that is not always followed in scripts read by a computer (SI Brochure, §5.4.4) [standard]; speech synthesisers misread 2,637 in most voices (measured with eSpeak NG). A unit symbol follows the number after a space (SI Brochure, §5.4.3) [standard]; min and h are units accepted for use with the SI (SI Brochure, Table 8) [standard].
Messages are ASCII: at least, X to Y, ..., U+200B (3 times); --help uses ASCII words and ASCII formulas (100*T/N, (D - 1)/(2N ln 2)); only authors’ names keep their lettersScreen readers do not read non-ASCII symbols as intended (Sampath et al. 2021, Recommendation 6) [general]; measured: man in the C locale drops the ceiling brackets, less and PowerShell show byte sequences, and screen readers are silent on them. The methodology pages keep typeset formulas.
Percentages are written 62%, without a space, in messages, help, documentation and the desktop appThe paper writes every percentage this way (Tharanga et al. 2025), as do nearly all recent papers in the field’s journals (213 of 214 measured); a spaced % can wrap onto its own line in --help. This deviates from the SI, which separates the number and % by a space (SI Brochure, §5.4.7) [standard].
Distinct exit codes per class of failure (usage, I/O, invalid data), and 128 + signal number after an interruptionLets scripts and workflow managers react without parsing messages. Gap: no peer-reviewed source; this follows the POSIX shell convention for signals.
An interruption removes temporary files, and the process ends by the same signal; a signal that was ignored at start-up (nohup) stays ignored(List et al. 2017, Rule 6). Gap: the grace period (3 s) and the PID 1 behaviour follow the documentation of container runtimes and batch schedulers, not peer-reviewed work.
The default thread count respects the CPUs a batch scheduler granted on the nodeOversubscribing a shared node slows every job on it. k-mer counting parallelises well across threads (Marçais & Kingsford 2011; Manekar & Sathe 2018). Gap: how to read the grant comes from the Slurm, PBS, SGE and LSF documentation.
When the system refuses some worker threads (a limit on processes and threads, a lack of memory), the run continues with those it started, warns and records both numbers in the run report; it stops (exit 3) only when the reader, the checksum thread, the interrupt handler or a single worker cannot start. Those essential threads start before the workersResults do not depend on the number of threads, so a run with fewer gives the same results (Taschuk & Wilson 2017, Rule 10); a problem is reported, never passed over silently (Seemann 2013, Recommendation 5), and the run’s log says what happened (List et al. 2017, Rule 7). Gap: no peer-reviewed source compares stopping with continuing; GNU sort also continues with fewer threads, while samtools and xz stop.
ChoiceRationale and sources
Results JSON with a versioned JSON Schema, keys of DiMA v5.0.9, and the full provenance of the run (parameters with their origin, input checksum, alphabet decision, validation, diagnostics, references)Results should be traceable to how they were produced (Sandve et al. 2013, Rule 1); FAIR provenance (Wilkinson et al. 2016) [general]; the provenance model of QIIME 2 and Galaxy (Bolyen et al. 2019; Goecks et al. 2010). JSON (RFC 8259) [standard].
No timestamps, host names or paths in the results; identical inputs give identical bytesSoftware should produce identical results for identical inputs (Taschuk & Wilson 2017, Rule 10); randomness, if any, must be recorded (Sandve et al. 2013, Rule 6) — DiMA has none.
Execution facts (times, threads, memory, platform) go to a separate, opt-in run report without user or host namesKeeps results byte-identical while recording how a run went (Grüning, Chilton et al. 2018); field names map onto workflow-provenance vocabularies (Khan et al. 2019; Leo et al. 2024).
Tidy tables: one observation per row, one variable per column, NA for undefined values, RFC 4180 quoting(Wickham 2014) [general]; (Broman & Woo 2018) [general]; (RFC 4180) [standard].
Undefined values are null/NA, never 0See positions without support.
Outputs are compressed by file extension (.gz, .zst)Compressors of sequence data trade size against speed (Kryukov et al. 2020).
Outputs are replaced atomically (temporary file, sync, rename); an interrupted run never leaves a partial fileGap: file-system practice, no peer-reviewed source.
dima cite prints the paper and the exact software version; dima cite --from adds the references of the methods a run used; CITATION.cff describes the releaseSoftware is often mentioned informally instead of cited (Howison & Bullard 2016) [general]; citations should be specific to the version (Smith et al. 2016) [general]; (Katz et al. 2021) [general]; tell users how to cite (Lee 2018, Rule 10); FAIR research software (Barker et al. 2022) [general].
ChoiceRationale and sources
One streaming pass; memory grows with the number of distinct k-mers, not with sequences × lengthTools must be ready for data growth (List et al. 2017, Rule 3).
k-mers are interned exactly per position with a rolling hash, every match verified byte by byteO(1) hash updates per position (Karp & Rabin 1987) [general]; verification rules out collisions.
Positions are partitioned among threads, each owning its hash tablesAvoiding locks lets k-mer counting scale with threads (Marçais & Kingsford 2011); the trade-offs of shared and partitioned hash tables on multi-core CPUs (Blanas et al. 2011; Balkesen et al. 2013) [general].
Exact arithmetic where the paper’s method allows it (integer thresholds, exact expectations), compensated summation, platform-independent logarithms(Taschuk & Wilson 2017, Rule 10).
Every reported entropy is checked to be a finite number of at least 0 bits; a value that is not stops the run as an internal error (exit 1, “please report”), never replaced by another valueAssertions halt a program as soon as something goes wrong (Wilson et al. 2014, Practice 5a); postconditions check that output has the value range required downstream (Hunter-Zinck et al. 2021, Rule 5); a failure must give a correct exit code (Seemann 2013, Recommendation 5); faults that change output without a crash are the hard ones in scientific software (Kanewala & Bieman 2014) [general]. The estimators guarantee the range (the paper’s negative-intercept rule can never apply to the exact expectation), so a violation is a bug. Established tools stop on states their own maths rules out (HMMER/Easel, MAFFT, IQ-TREE, RAxML-NG, Nextclade; tool behaviour); the check costs under 1/100,000 of the run time (measured).
Floating-point rounding is measured and documented rather than corrected to the last digit: values carry up to 5 × 10 − 11 relative error at 17.5 million sequences per position, 10 10 times below the sampling error; a prototyped fix is deferred (see Numerical accuracy)Developers of scientific software should analyse round-off error, and two runs that differ only in round-off can give results that significance tests cannot tell apart (Darriba et al. 2018, Advanced Best Practices); accumulated round-off must be shown to be tolerable (Eddy 2011). The mechanism is the loss of the low-order digits of a number close to 1 before its logarithm is taken (Goldberg 1991, §1.4 and Theorem 4) [general]. The sampling error comes from the asymptotic variance of the entropy estimate (Paninski 2003, Theorem 2) [general]. The fix changes stored digits, so it is not a pure engineering change (working agreement 1). Tools such as scikit-bio, bcftools and SciPy fix precision in ordinary releases, and HMMER points users to archived versions for exact reproducibility (tool behaviour, no peer-reviewed source).
Chao–Wang–Jost’s second term is evaluated exactly in a time independent of the number of sequences: a series of non-negative terms when it converges fast, otherwise a continued fraction with positive coefficients evaluated backwards, each stopped when the remaining error is below half a unit in the last place, with a 2048-step cap that stops the run as an internal error (see Numerical accuracy)Summed as written the term costs time proportional to N (about 10% of a run) and loses up to 8 digits; evaluated directly it cancels completely (measured). The identities: the hypergeometric series, Pfaff’s transformation and the expansion about z = 1 (Hankin 2015, eqs. 15.1.1, 15.3.4 and 15.3.10) [general]; Gauss’s continued fraction and its convergence (Van Vleck 1901, p. 1 and §9(a)) [general]; the value between consecutive approximants (Henrici & Pflüger 1966, §2) [general]; stable backward evaluation (Jones & Thron 1974, Theorem 5.1(i)) [general]; the digamma remainder (Alzer 1997, Theorem 8) [general]. No domain source evaluates this term other than by summing it (gap); domain implementations sum it in O ( N ) (tool behaviour).
Static Linux builds use the mimalloc allocator(Leijen et al. 2019) [general]. Gap: the slowness of musl’s allocator under many threads is documented in measurements, not peer-reviewed work; the release workflow measures it.
xz input is refused with a hint to decompress it or recompress it with Zstandard, which is recommended for large inputsCompressors differ widely in decompression speed on FASTA data (Kryukov et al. 2020). An xz file can be decompressed in parallel only when it was written in blocks, which most xz alignments are not, and a damaged file must be refused whatever the number of threads (Taschuk & Wilson 2017, Rule 10); reading xz on one thread would add a decoder for a format that the large public viral datasets have left for Zstandard. Gap: the formats of public datasets and the decompression speeds were checked for this project, not taken from peer-reviewed work.
ChoiceRationale and sources
Linux binaries built against an old C library (glibc 2.17 for x86_64, 2.28 for ARM64) plus static buildsMany omics tools fail to install, which limits their use (Mangul, Mosqueiro et al. 2019). Gap: that many HPC clusters still run old Linux distributions comes from community surveys, not peer-reviewed work.
Bioconda, BioContainers and Apptainer as the preferred channels once the recipe is accepted(Grüning, Dale et al. 2018; da Veiga Leprevost et al. 2017; Kurtzer et al. 2017; Grüning, Chilton et al. 2018).
Release archives with checksums, build-provenance attestations and software bills of materials; CI actions pinned by commitDefences against supply-chain attacks (Ladisa et al. 2023) [general]; pinning alone is not enough (He et al. 2025) [general]; (Zahan et al. 2023) [general].
Windows programs link the C runtime statically, and the release checks every Windows program’s importsSo that they need no other software installed: without it they need the Microsoft Visual C++ Redistributable, which Windows does not include. Gap: an engineering choice; no peer-reviewed source.
Unsigned desktop packages with documented first-run stepsGap: code signing needs certificates the project does not hold; no peer-reviewed source.
ChoiceRationale and sources
On opening a file, only its first records are inspected (well under a second) and the analysis can start at once; the run is the only full validationUsers should get started quickly and be told about mistakes clearly (List et al. 2017, Rules 6–7; Durant et al. 2022, Rule 7; Lee 2018, Rule 9). Response-time limits of about 0.1 s, 1 s and 10 s (Miller 1968) [general]; (Card et al. 1991) [general]; errors shown mid-input tend to be ignored (Bargas-Avila et al. 2007) [general]. Gap: no domain source states response-time limits.
A header editor offering the paper’s field names, with a live preview of values and a “copy as --header-format” actionRecognition rather than recall, visibility of system status (Nielsen & Molich 1990) [general]; (Nielsen 1994) [general].
Last-used settings are remembered, non-default values are marked, one action resets to the defaults, and the run’s provenance records everythingHidden state undermines reproducibility (Pimentel et al. 2019) [general].
The entropy chart draws corrected values as a solid line, uncorrected (support ≤ T) values dashed, and positions without support as hatched bands, never as 0Colour-blind-safe palette (Wong 2011; Hattab et al. 2020); uncertainty encodings (Weiskopf 2022); missing data are not zeros in genome browsers (Kerpedjiev et al. 2018; Kent et al. 2010); position along a common scale is read most accurately (Cleveland & McGill 1984) [general]; labelled gaps are the best-understood treatment of missing data (Eaton et al. 2005) [general]; breaking a line lowers perceived quality, so it is kept for absent data (Song & Szafir 2019) [general]; dashing and lightness read as uncertainty (MacEachren et al. 2012) [general]; colour is never the only cue and graphics reach 3:1 contrast (WCAG 2.2, SC 1.4.1 and 1.4.11) [standard].
Long ranges are drawn with M4 downsampling, which keeps every peak and trough(Jugel et al. 2014) [general].
Native file dialogs with an automatic fallback to a built-in browser, and a path field(Durant et al. 2022, Rule 7; List et al. 2017, Rule 6). Gap: the fallback heuristics follow other desktop toolkits, not peer-reviewed work.
ChoiceRationale and sources
Tutorials, how-to guides, reference and explanation as separate kinds of pages; quick start first(Karimzadeh & Hoffman 2018; Lee 2018).
A short FAQ whose questions are phrased as users ask them and whose answers link to the full page; a Glossary of the terms in the results; Troubleshooting for every messageKarimzadeh & Hoffman recommend an FAQ, which “is easier to link to an answer” than a manual, and a separate chapter that explains error and warning messages; 14 of the 22 highly cited packages they surveyed have an FAQ (Karimzadeh & Hoffman 2018). A missing FAQ and readers unsure where to start are among the documentation issues found in practice (Aghajani et al. 2019) [general].
Questions go to GitHub Discussions, bugs to GitHub Issues through a form that asks for the version, the command and the run reportExpect support requests and give users a channel for them (List et al. 2017, Rule 10); 16 of the 22 packages surveyed offer a forum or mailing list (Karimzadeh & Hoffman 2018).
One published version of the documentation per release, plus the development version (the master branch)Version-control the documentation (Lee 2018, Rule 6); archival stability (Mangul, Mosqueiro et al. 2019).
Search engines index only the current documentation (/latest/): the pages of each release carry noindex and their own canonical address. dima --help, the desktop app and CITATION.cff link the documentation of their own version, and /latest/ says when it describes changes not yet releasedReaders should find documentation that matches the version they run (Lee 2018, Rule 6). Outdated documentation is a common problem: among up-to-dateness issues, outdated references such as links to old versions were the most prevalent (Aghajani et al. 2019) [general], and most obsolete answers on a question-and-answer site are never updated (Zhang et al. 2021) [general]. A canonical address must point to the same content or a superset of it, so a release page cannot name /latest/ (RFC 6596, Section 3) [standard]; a site under a project path cannot serve the /robots.txt file that crawlers read from the root of the host (RFC 9309, Section 2.3) [standard], so each page carries the tag.
Every scientific statement cites its sources, from the same registry as the program’s help(Lee 2018); (Smith et al. 2016) [general].
Citations show author and year, link to the page’s reference list (which links back) and preview the full reference on hover or keyboard focus; the reference texts are those of dima citeOn desktop computers readers mostly hover over a citation to see its reference; on phones they tap it (Piccardi et al. 2020) [general]. Links from citations to the reference list and back let blind and low-vision readers resolve a citation without losing their place (Wang et al. 2021) [general]. The same work has different reference numbers in different documents, which makes citations harder to keep track of (Lo et al. 2024) [general]. Roles for citations, reference lists and back-links (DPUB-ARIA 1.1) [standard]; previews can be dismissed with Escape, hovered and stay until the pointer or focus leaves (WCAG 2.2, SC 1.4.13) [standard]. Gap: no peer-reviewed study compares author-year with numbered citations for readability.
Labels are computed once over the whole registry, so a work has the same label in the help, the desktop app and every page: works that would share a label show more names (Grüning, Dale et al. 2018, Grüning, Chilton et al. 2018), then year letters for identical authors, which appear only in DiMA’s own reference lists, never in exported referencesA label must lead to exactly one entry of the reference list; labels computed page by page would give the same work different labels on different pages. Gap: the order (names first, then letters) follows the author-date style manuals and the Citation Style Language 1.0.2 specification, not peer-reviewed work.
Standards are labelled by the designation their publisher gives them (RFC 8259, UAX #15, WCAG 2.2), without a year, and DiMA’s reference lists start their entries with it; each cites the dated version that DiMA followsRFCs are cited by their numbers (RFC 7322, Section 3.5) [standard], and the W3C and the Unicode Consortium document the same short forms. Designations are unique and searchable, and need no year letters. Gap: articles in bioinformatics journals mostly cite standards by their editors’ names; no study compares how readers recognise either form.
WCAG 2.2 AA, checked on every page by automated scans in both themes; formulas carry MathML for screen readers(WCAG 2.2) [standard].
No analytics, trackers or advertisingReaders’ privacy. Gap: a principle, not a cited finding.
ChoiceRationale and sources
An independent reference implementation as test oracle, golden files compared byte for byte, and the paper’s datasets in the test suiteThe oracle problem in scientific software (Kanewala & Bieman 2014) [general]; (Wilson et al. 2014; Hunter-Zinck et al. 2021).
Property-based tests (for example: results do not depend on line wrapping, case, compression, threads or batch size)(Claessen & Hughes 2000) [general].
Reproducible benchmarks with the exact commands, hardware and versions(Weber et al. 2019; Mangul, Martin et al. 2019).
  • Aghajani E, Nagy C, Vega-Márquez OL, Linares-Vásquez M, Moreno L, Bavota G, Lanza M. Software Documentation Issues Unveiled. In: 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). 2019. p. 1199-1210. doi:10.1109/ICSE.2019.00122 ↩︎ 1 2
  • Alzer H. On some inequalities for the gamma and psi functions. Mathematics of Computation. 1997;66(217):373-389. doi:10.1090/S0025-5718-97-00807-7 ↩︎
  • Ancker JS, Edwards A, Nosal S, Hauser D, Mauer E, Kaushal R, HITEC Investigators. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Medical Informatics and Decision Making. 2017;17(1):36. doi:10.1186/s12911-017-0430-8 ↩︎
  • Balkesen C, Teubner J, Alonso G, Ozsu MT. Main-memory hash joins on multi-core CPUs: Tuning to the underlying hardware. In: 2013 IEEE 29th International Conference on Data Engineering (ICDE). 2013. p. 362-373. doi:10.1109/ICDE.2013.6544839 ↩︎
  • Bargas-Avila JA, Oberholzer G, Schmutz P, de Vito M, Opwis K. Usable error message presentation in the World Wide Web: Do not show errors right away. Interacting with Computers. 2007;19(3):330-341. doi:10.1016/j.intcom.2007.01.003 ↩︎
  • Barker M, Chue Hong NP, Katz DS, Lamprecht AL, Martinez-Ortiz C, Psomopoulos F, Harrow J, Castro LJ, Gruenpeter M, Martinez PA, Honeyman T. Introducing the FAIR Principles for research software. Scientific Data. 2022;9(1):622. doi:10.1038/s41597-022-01710-x ↩︎
  • Blanas S, Li Y, Patel JM. Design and evaluation of main memory hash join algorithms for multi-core CPUs. In: Proceedings of the 2011 ACM SIGMOD International Conference on Management of data. 2011. p. 37-48. doi:10.1145/1989323.1989328 ↩︎
  • Bolyen E, Rideout JR, Dillon MR, Bokulich NA, Abnet CC, Al-Ghalith GA, Alexander H, Alm EJ, Arumugam M, Asnicar F, Bai Y, Bisanz JE, Bittinger K, Brejnrod A, Brislawn CJ, Brown CT, Callahan BJ, Caraballo-Rodríguez AM, Chase J, Cope EK, et al. Reproducible, interactive, scalable and extensible microbiome data science using QIIME 2. Nature Biotechnology. 2019;37(8):852-857. doi:10.1038/s41587-019-0209-9 ↩︎
  • Broman KW, Woo KH. Data Organization in Spreadsheets. The American Statistician. 2018;72(1):2-10. doi:10.1080/00031305.2017.1375989 ↩︎
  • Card SK, Robertson GG, Mackinlay JD. The information visualizer, an information workspace. In: Proceedings of the SIGCHI conference on Human factors in computing systems Reaching through technology - CHI '91. 1991. p. 181-186. doi:10.1145/108844.108874 ↩︎
  • Claessen K, Hughes J. QuickCheck: a lightweight tool for random testing of Haskell programs. In: Proceedings of the fifth ACM SIGPLAN international conference on Functional programming. 2000. p. 268-279. doi:10.1145/351240.351266 ↩︎
  • Cleveland WS, McGill R. Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods. Journal of the American Statistical Association. 1984;79(387):531-554. doi:10.1080/01621459.1984.10478080 ↩︎
  • da Veiga Leprevost F, Grüning BA, Alves Aflitos S, Röst HL, Uszkoreit J, Barsnes H, Vaudel M, Moreno P, Gatto L, Weber J, Bai M, Jimenez RC, Sachsenberg T, Pfeuffer J, Vera Alvarez R, Griss J, Nesvizhskii AI, Perez-Riverol Y. BioContainers: an open-source and community-driven framework for software standardization. Bioinformatics. 2017;33(16):2580-2582. doi:10.1093/bioinformatics/btx192 ↩︎
  • Darriba D, Flouri T, Stamatakis A. The State of Software for Evolutionary Biology. Molecular Biology and Evolution. 2018;35(5):1037-1046. doi:10.1093/molbev/msy014 ↩︎
  • DPUB-ARIA 1.1. Garrish M, Siegman T. Digital Publishing WAI-ARIA Module 1.1. W3C Recommendation; 2025. https://www.w3.org/TR/2025/REC-dpub-aria-1.1-20250612/ ↩︎
  • Durant E, Rouard M, Ganko EW, Muller C, Cleary AM, Farmer AD, Conte M, Sabot F. Ten simple rules for developing visualization tools in genomics. PLOS Computational Biology. 2022;18(11):e1010622. doi:10.1371/journal.pcbi.1010622 ↩︎ 1 2
  • Eaton C, Plaisant C, Drizd T. Visualizing Missing Data: Graph Interpretation User Study. In: Lecture Notes in Computer Science. 2005. p. 861-872. doi:10.1007/11555261_68 ↩︎
  • Eddy SR. Accelerated Profile HMM Searches. PLoS Computational Biology. 2011;7(10):e1002195. doi:10.1371/journal.pcbi.1002195 ↩︎
  • Goecks J, Nekrutenko A, Taylor J, Galaxy Team. Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences. Genome Biology. 2010;11(8):R86. doi:10.1186/gb-2010-11-8-r86 ↩︎
  • Goldberg D. What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys. 1991;23(1):5-48. doi:10.1145/103162.103163 ↩︎
  • Grüning B, Chilton J, Köster J, Dale R, Soranzo N, van den Beek M, Goecks J, Backofen R, Nekrutenko A, Taylor J. Practical Computational Reproducibility in the Life Sciences. Cell Systems. 2018;6(6):631-635. doi:10.1016/j.cels.2018.03.014 ↩︎ 1 2
  • Grüning B, Dale R, Sjödin A, Chapman BA, Rowe J, Tomkins-Tinch CH, Valieris R, Köster J, Bioconda Team. Bioconda: sustainable and comprehensive software distribution for the life sciences. Nature Methods. 2018;15(7):475-476. doi:10.1038/s41592-018-0046-7 ↩︎
  • Hankin RKS. Numerical Evaluation of the Gauss Hypergeometric Function with the hypergeo Package. The R Journal. 2015;7(2):81-88. doi:10.32614/RJ-2015-022 ↩︎
  • Harrison C, Amento B, Kuznetsov S, Bell R. Rethinking the progress bar. In: Proceedings of the 20th annual ACM symposium on User interface software and technology. 2007. p. 115-118. doi:10.1145/1294211.1294231 ↩︎
  • Hattab G, Rhyne TM, Heider D. Ten simple rules to colorize biological data visualization. PLOS Computational Biology. 2020;16(10):e1008259. doi:10.1371/journal.pcbi.1008259 ↩︎
  • He H, Vasilescu B, Kästner C. Pinning Is Futile: You Need More Than Local Dependency Versioning to Defend against Supply Chain Attacks. Proceedings of the ACM on Software Engineering. 2025;2(FSE):266-289. doi:10.1145/3715728 ↩︎
  • Henrici P, Pflüger P. Truncation error estimates for Stieltjes fractions. Numerische Mathematik. 1966;9(2):120-138. doi:10.1007/BF02166031 ↩︎
  • Howison J, Bullard J. Software in the scientific literature: Problems with seeing, finding, and using software mentioned in the biology literature. Journal of the Association for Information Science and Technology. 2016;67(9):2137-2155. doi:10.1002/asi.23538 ↩︎
  • Hunter-Zinck H, de Siqueira AF, Vásquez VN, Barnes R, Martinez CC. Ten simple rules on writing clean and reliable open-source scientific software. PLOS Computational Biology. 2021;17(11):e1009481. doi:10.1371/journal.pcbi.1009481 ↩︎ 1 2
  • Jones WB, Thron WJ. Numerical stability in evaluating continued fractions. Mathematics of Computation. 1974;28(127):795-810. doi:10.1090/S0025-5718-1974-0373265-5 ↩︎
  • Jugel U, Jerzak Z, Hackenbroich G, Markl V. M4. Proceedings of the VLDB Endowment. 2014;7(10):797-808. doi:10.14778/2732951.2732953 ↩︎
  • Kanewala U, Bieman JM. Testing scientific software: A systematic literature review. Information and Software Technology. 2014;56(10):1219-1232. doi:10.1016/j.infsof.2014.05.006 ↩︎ 1 2
  • Karimzadeh M, Hoffman MM. Top considerations for creating bioinformatics software documentation. Briefings in Bioinformatics. 2018;19(4):693-699. doi:10.1093/bib/bbw134 ↩︎ 1 2 3
  • Karp RM, Rabin MO. Efficient randomized pattern-matching algorithms. IBM Journal of Research and Development. 1987;31(2):249-260. doi:10.1147/rd.312.0249 ↩︎
  • Katz DS, Chue Hong NP, Clark T, Muench A, Stall S, Bouquin D, Cannon M, Edmunds S, Faez T, Feeney P, Fenner M, Friedman M, Grenier G, Harrison M, Heber J, Leary A, MacCallum C, Murray H, Pastrana E, Perry K, et al. Recognizing the value of software: a software citation guide. F1000Research. 2021;9:1257. doi:10.12688/f1000research.26932.2 ↩︎
  • Kent WJ, Zweig AS, Barber G, Hinrichs AS, Karolchik D. BigWig and BigBed: enabling browsing of large distributed datasets. Bioinformatics. 2010;26(17):2204-2207. doi:10.1093/bioinformatics/btq351 ↩︎
  • Kerpedjiev P, Abdennur N, Lekschas F, McCallum C, Dinkla K, Strobelt H, Luber JM, Ouellette SB, Azhir A, Kumar N, Hwang J, Lee S, Alver BH, Pfister H, Mirny LA, Park PJ, Gehlenborg N. HiGlass: web-based visual exploration and analysis of genome interaction maps. Genome Biology. 2018;19(1):125. doi:10.1186/s13059-018-1486-1 ↩︎
  • Khan FZ, Soiland-Reyes S, Sinnott RO, Lonie A, Goble C, Crusoe MR. Sharing interoperable workflow provenance: A review of best practices and their practical application in CWLProv. GigaScience. 2019;8(11):giz095. doi:10.1093/gigascience/giz095 ↩︎
  • Kryukov K, Ueda MT, Nakagawa S, Imanishi T. Sequence Compression Benchmark (SCB) database—A comprehensive evaluation of reference-free compressors for FASTA-formatted sequences. GigaScience. 2020;9(7):giaa072. doi:10.1093/gigascience/giaa072 ↩︎ 1 2
  • Kurtzer GM, Sochat V, Bauer MW. Singularity: Scientific containers for mobility of compute. PLOS ONE. 2017;12(5):e0177459. doi:10.1371/journal.pone.0177459 ↩︎
  • Ladisa P, Plate H, Martinez M, Barais O. SoK: Taxonomy of Attacks on Open-Source Software Supply Chains. In: 2023 IEEE Symposium on Security and Privacy (SP). 2023. p. 1509-1526. doi:10.1109/SP46215.2023.10179304 ↩︎
  • Lee BD. Ten simple rules for documenting scientific software. PLOS Computational Biology. 2018;14(12):e1006561. doi:10.1371/journal.pcbi.1006561 ↩︎ 1 2 3 4 5 6 7
  • Leijen D, Zorn B, de Moura L. Mimalloc: Free List Sharding in Action. In: Lecture Notes in Computer Science. 2019. p. 244-265. doi:10.1007/978-3-030-34175-6_13 ↩︎
  • Leo S, Crusoe MR, Rodríguez-Navas L, Sirvent R, Kanitz A, De Geest P, Wittner R, Pireddu L, Garijo D, Fernández JM, Colonnelli I, Gallo M, Ohta T, Suetake H, Capella-Gutierrez S, de Wit R, Kinoshita BP, Soiland-Reyes S. Recording provenance of workflow runs with RO-Crate. PLOS ONE. 2024;19(9):e0309210. doi:10.1371/journal.pone.0309210 ↩︎
  • List M, Ebert P, Albrecht F. Ten Simple Rules for Developing Usable Software in Computational Biology. PLOS Computational Biology. 2017;13(1):e1005265. doi:10.1371/journal.pcbi.1005265 ↩︎ 1 2 3 4 5 6 7 8 9 10
  • Lo K, Chang JC, Head A, Bragg J, Zhang AX, Trier C, Anastasiades C, August T, Authur R, Bragg D, Bransom E, Cachola I, Candra S, Chandrasekhar Y, Chen YS, Cheng EYY, Chou Y, Downey D, Evans R, Fok R, et al. The Semantic Reader Project. Communications of the ACM. 2024;67(10):50-61. doi:10.1145/3659096 ↩︎
  • MacEachren AM, Roth RE, O'Brien J, Li B, Swingley D, Gahegan M. Visual Semiotics & Uncertainty Visualization: An Empirical Study. IEEE Transactions on Visualization and Computer Graphics. 2012;18(12):2496-2505. doi:10.1109/TVCG.2012.279 ↩︎
  • Manekar SC, Sathe SR. A benchmark study of k-mer counting methods for high-throughput sequencing. GigaScience. 2018;7(12):giy125. doi:10.1093/gigascience/giy125 ↩︎
  • Mangul S, Martin LS, Hill BL, Lam AKM, Distler MG, Zelikovsky A, Eskin E, Flint J. Systematic benchmarking of omics computational tools. Nature Communications. 2019;10(1):1393. doi:10.1038/s41467-019-09406-4 ↩︎
  • Mangul S, Mosqueiro T, Abdill RJ, Duong D, Mitchell K, Sarwal V, Hill B, Brito J, Littman RJ, Statz B, Lam AKM, Dayama G, Grieneisen L, Martin LS, Flint J, Eskin E, Blekhman R. Challenges and recommendations to improve the installability and archival stability of omics computational tools. PLOS Biology. 2019;17(6):e3000333. doi:10.1371/journal.pbio.3000333 ↩︎ 1 2
  • Marçais G, Kingsford C. A fast, lock-free approach for efficient parallel counting of occurrences of k-mers. Bioinformatics. 2011;27(6):764-770. doi:10.1093/bioinformatics/btr011 ↩︎ 1 2
  • Miller RB. Response time in man-computer conversational transactions. In: Proceedings of the December 9-11, 1968, fall joint computer conference, part I on - AFIPS '68 (Fall, part I). 1968. p. 267. doi:10.1145/1476589.1476628 ↩︎
  • Nielsen J, Molich R. Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI conference on Human factors in computing systems Empowering people - CHI '90. 1990. p. 249-256. doi:10.1145/97243.97281 ↩︎
  • Nielsen J. Enhancing the explanatory power of usability heuristics. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1994. p. 152-158. doi:10.1145/191666.191729 ↩︎
  • Paninski L. Estimation of Entropy and Mutual Information. Neural Computation. 2003;15(6):1191-1253. doi:10.1162/089976603321780272 ↩︎
  • Piccardi T, Redi M, Colavizza G, West R. Quantifying Engagement with Citations on Wikipedia. In: Proceedings of The Web Conference 2020. 2020. p. 2365-2376. doi:10.1145/3366423.3380300 ↩︎
  • Pimentel JF, Murta L, Braganholo V, Freire J. A Large-Scale Study About Quality and Reproducibility of Jupyter Notebooks. In: 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR). 2019. p. 507-517. doi:10.1109/MSR.2019.00077 ↩︎
  • RFC 4180. Shafranovich Y. Common Format and MIME Type for Comma-Separated Values (CSV) Files. RFC 4180; 2005. doi:10.17487/RFC4180 ↩︎
  • RFC 6596. Ohye M, Kupke J. The Canonical Link Relation. RFC 6596; 2012. doi:10.17487/RFC6596 ↩︎
  • RFC 7322. Flanagan H, Ginoza S. RFC Style Guide. RFC 7322; 2014. doi:10.17487/RFC7322 ↩︎
  • RFC 8259. Bray T. The JavaScript Object Notation (JSON) Data Interchange Format. RFC 8259; 2017. doi:10.17487/RFC8259 ↩︎
  • RFC 9309. Koster M, Illyes G, Zeller H, Sassman L. Robots Exclusion Protocol. RFC 9309; 2022. doi:10.17487/RFC9309 ↩︎
  • Sampath H, Merrick A, Macvean A. Accessibility of Command Line Interfaces. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 2021. p. 1-10. doi:10.1145/3411764.3445544 ↩︎ 1 2
  • Sandve GK, Nekrutenko A, Taylor J, Hovig E. Ten Simple Rules for Reproducible Computational Research. PLoS Computational Biology. 2013;9(10):e1003285. doi:10.1371/journal.pcbi.1003285 ↩︎ 1 2
  • Seemann T. Ten recommendations for creating usable bioinformatics command line software. GigaScience. 2013;2(1):15. doi:10.1186/2047-217X-2-15 ↩︎ 1 2 3 4 5 6
  • Shen W, Sipos B, Zhao L. SeqKit2: A Swiss army knife for sequence and alignment processing. iMeta. 2024;3(3):e191. doi:10.1002/imt2.191 ↩︎
  • SI Brochure. Bureau International des Poids et Mesures. The International System of Units (SI), 9th edition. BIPM; 2019. doi:10.59161/AUEZ1291 ↩︎ 1 2 3 4
  • Smith AM, Katz DS, Niemeyer KE, FORCE11 Software Citation Working Group. Software citation principles. PeerJ Computer Science. 2016;2:e86. doi:10.7717/peerj-cs.86 ↩︎ 1 2
  • Song H, Szafir DA. Where's My Data? Evaluating Visualizations with Missing Data. IEEE Transactions on Visualization and Computer Graphics. 2019;25(1):914-924. doi:10.1109/TVCG.2018.2864914 ↩︎
  • Taschuk M, Wilson G. Ten simple rules for making research software more robust. PLOS Computational Biology. 2017;13(4):e1005412. doi:10.1371/journal.pcbi.1005412 ↩︎ 1 2 3 4 5
  • Tharanga S, Ünlü ES, Hu Y, Sjaugi MF, Çelik MA, Hekimoğlu H, Miotto O, Öncel MM, Khan AM. DiMA: sequence diversity dynamics analyser for viruses. Briefings in Bioinformatics. 2025;26(1):bbae607. doi:10.1093/bib/bbae607 ↩︎ 1 2
  • van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of Drug Safety Alerts in Computerized Physician Order Entry. Journal of the American Medical Informatics Association. 2006;13(2):138-147. doi:10.1197/jamia.M1809 ↩︎
  • Van Vleck EB. On the Convergence of the Continued Fraction of Gauss and Other Continued Fractions. The Annals of Mathematics. 1901;3(1/4):1-18. doi:10.2307/1967627 ↩︎
  • Wang LL, Cachola I, Bragg J, Cheng EYY, Haupt C, Latzke M, Kuehl B, van Zuylen MN, Wagner L, Weld D. SciA11y: Converting Scientific Papers to Accessible HTML. In: Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility. 2021. p. 1-4. doi:10.1145/3441852.3476545 ↩︎
  • WCAG 2.2. Campbell A, Adams C, Bradley Montgomery R, Cooper M, Kirkpatrick A. Web Content Accessibility Guidelines (WCAG) 2.2. W3C Recommendation; 2024. https://www.w3.org/TR/2024/REC-WCAG22-20241212/ ↩︎ 1 2 3 4
  • WCAG2ICT. W3C Accessibility Guidelines Working Group. Guidance on Applying WCAG 2 to Non-Web Information and Communications Technologies (WCAG2ICT). W3C Group Note; 2025. https://www.w3.org/TR/2025/NOTE-wcag2ict-22-20251211/ ↩︎
  • Weber LM, Saelens W, Cannoodt R, Soneson C, Hapfelmeier A, Gardner PP, Boulesteix AL, Saeys Y, Robinson MD. Essential guidelines for computational method benchmarking. Genome Biology. 2019;20(1):125. doi:10.1186/s13059-019-1738-8 ↩︎
  • Weiskopf D. Uncertainty Visualization: Concepts, Methods, and Applications in Biological Data Visualization. Frontiers in Bioinformatics. 2022;2:793819. doi:10.3389/fbinf.2022.793819 ↩︎
  • Wickham H. Tidy Data. Journal of Statistical Software. 2014;59(10):1-23. doi:10.18637/jss.v059.i10 ↩︎
  • Wilkinson MD, Dumontier M, Aalbersberg IJ, Appleton G, Axton M, Baak A, Blomberg N, Boiten JW, da Silva Santos LB, Bourne PE, Bouwman J, Brookes AJ, Clark T, Crosas M, Dillo I, Dumon O, Edmunds S, Evelo CT, Finkers R, Gonzalez-Beltran A, et al. The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data. 2016;3(1):160018. doi:10.1038/sdata.2016.18 ↩︎ 1 2
  • Wilson G, Aruliah DA, Brown CT, Chue Hong NP, Davis M, Guy RT, Haddock SHD, Huff KD, Mitchell IM, Plumbley MD, Waugh B, White EP, Wilson P. Best Practices for Scientific Computing. PLoS Biology. 2014;12(1):e1001745. doi:10.1371/journal.pbio.1001745 ↩︎ 1 2
  • Wong B. Points of view: Color blindness. Nature Methods. 2011;8(6):441-441. doi:10.1038/nmeth.1618 ↩︎
  • Zahan N, Kanakiya P, Hambleton B, Shohan S, Williams L. OpenSSF Scorecard: On the Path Toward Ecosystem-Wide Automated Security Metrics. IEEE Security & Privacy. 2023;21(6):76-88. doi:10.1109/MSEC.2023.3279773 ↩︎
  • Zhang H, Wang S, Chen TH, Zou Y, Hassan AE. An Empirical Study of Obsolete Answers on Stack Overflow. IEEE Transactions on Software Engineering. 2021;47(4):850-862. doi:10.1109/TSE.2019.2906315 ↩︎