Ancestree: unified likelihood inference of ancestral alleles under supplied or inferred genealogies
- 1Aarhus University
Abstract
Inferring ancestral states--determining, at each polymorphic site, which allele is ancestral and which derived--underpins many downstream population-genetic analyses, from selection scans and the unfolded site-frequency spectrum to demographic inference. However, no existing tool uniformly supports the full range of relevant inputs: plain variant data or ancestral recombination graphs (ARGs), with or without outgroups, while accommodating poly-allelic and recurrently-mutated sites. Here we present Ancestree, a likelihood-based engine that unifies these inputs within a single framework and returns full posteriors over the four nucleotide states at every site. It runs in three modes: a fixed-tree mode that assumes a single topology across sites and co-infers the per-branch substitution rates by maximum likelihood; an ARG mode that reads a different local tree at each site directly from a supplied ancestral recombination graph; and a local-tree mode that instead samples those local trees from the genotype data via a pairwise-coalescent HMM, needing no pre-existing ARG. On simulated data, the genealogy-based modes (ARG and local-tree) are more accurate and scale better, and remain robust under outgroup configurations that violate the fixed-tree assumption. Outgroups themselves remain difficult to replace: per-site inference accuracy on ingroup-polymorphic sites is markedly limited without them, and improves substantially with a single outgroup. The hardest sites are those fixed for the derived allele within the ingroup, which carry no within-ingroup signal and so need several sufficiently deep outgroups to recover, yet these are also highly informative downstream, carrying the high-frequency divergence signal on which selection and adaptation analyses often depend. Ancestree is available at github.com/Sendrowski/Ancestree.
Related
- Statistical inference of virus evolution using ancestral recombination graphs
- Robustness of Ancestral Recombination Graph Inference Tools to Phasing Errors
- Two Routes to Interoceptive Prioritisation: Dissociating Likelihood Precision and Preference Reweighting in Active Inference
- Joint dog and wolf genealogies reveal the evolution of the canine genome
- Migration Genealogies in Multiregional Stable Populations: First-Return Decompositions, Recurrence, and Sensitivity
BibTeX
@misc{sendrowski2026ancestree,
title = {{Ancestree: unified likelihood inference of ancestral alleles under supplied or inferred genealogies}},
author = {J. Sendrowski and T. Bataillon},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.11.750934},
url = {https://www.biorxiv.org/content/10.64898/2026.09.11.750934v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Ancestree: unified likelihood inference of ancestral alleles under supplied or inferred genealogies
AU - J. Sendrowski
AU - T. Bataillon
PY - 2026
DA - 2026/09/11
PB - bioRxiv
DO - 10.64898/2026.09.11.750934
UR - https://www.biorxiv.org/content/10.64898/2026.09.11.750934v1
AB - Inferring ancestral states--determining, at each polymorphic site, which allele is ancestral and which derived--underpins many downstream population-genetic analyses, from selection scans and the unfolded site-frequency spectrum to demographic inference. However, no existing tool uniformly supports the full range of relevant inputs: plain variant data or ancestral recombination graphs (ARGs), with or without outgroups, while accommodating poly-allelic and recurrently-mutated sites. Here we present Ancestree, a likelihood-based engine that unifies these inputs within a single framework and returns full posteriors over the four nucleotide states at every site. It runs in three modes: a fixed-tree mode that assumes a single topology across sites and co-infers the per-branch substitution rates by maximum likelihood; an ARG mode that reads a different local tree at each site directly from a supplied ancestral recombination graph; and a local-tree mode that instead samples those local trees from the genotype data via a pairwise-coalescent HMM, needing no pre-existing ARG. On simulated data, the genealogy-based modes (ARG and local-tree) are more accurate and scale better, and remain robust under outgroup configurations that violate the fixed-tree assumption. Outgroups themselves remain difficult to replace: per-site inference accuracy on ingroup-polymorphic sites is markedly limited without them, and improves substantially with a single outgroup. The hardest sites are those fixed for the derived allele within the ingroup, which carry no within-ingroup signal and so need several sufficiently deep outgroups to recover, yet these are also highly informative downstream, carrying the high-frequency divergence signal on which selection and adaptation analyses often depend. Ancestree is available at github.com/Sendrowski/Ancestree.
ER -