{
  "abstract": "Word counts or relative frequencies (e.g. the number of self-related words as a proportion of total words in a text) are routinely analyzed as dependent variables in the behavioral sciences. We demonstrate that conventional methods of analyzing word counts in this way are likely to lead to false positives, especially when text length varies between levels of the predictor variable. We performed Monte Carlo simulations to assess type I error rates for a wide range of affective and linguistic constructs across 9 datasets as well as known generative models, using 126 candidate statistical methods. We distinguish between three sources of error: distributional misspecification, insufficient control for text length, and biases in frequency estimation. Distributional misspecification occured in models that do not account for the overdispersion of word count distributions, such as Poisson regression. Insufficient control for text length was evident in all models that do not include the logarithm of text length as a covariate, or model it with a spline. Although we report evidence of systematic bias in frequency estimation for shorter texts as predicted by theoretical work, this bias is negligible relative to construct- and dataset-dependent text length effects. Among the methods investigated, we recommend Poisson-lognormal regression with log word count as a covariate, or linear regression with a spline on log word count. We also strongly caution against the use of text truncation and excerpting in analyses of word counts, as these are likely to introduce confounds that cannot be corrected with any statistical method.",
  "abstract_shown": true,
  "affiliations": [],
  "article_type": null,
  "arxiv": null,
  "author_total": 2,
  "authors": [
    {
      "name": "Louis Teitelbaum"
    },
    {
      "name": "Almog Simchon"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/csftyjmsa08v2.bib",
  "cited_by": 0,
  "doi": "10.31234/osf.io/zy6ew_v1",
  "fulltext_url": null,
  "id": "csftyjmsa08v2",
  "issns": [],
  "landing_url": "https://osf.io/preprints/psyarxiv/zy6ew_v1/",
  "language": null,
  "licence": "CC-BY-4.0",
  "licence_label": "CC BY 4.0",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "bepress",
      "Computational Linguistics"
    ],
    [
      "bepress",
      "Linguistics"
    ],
    [
      "bepress",
      "Quantitative Methods"
    ],
    [
      "bepress",
      "Social and Behavioral Sciences"
    ],
    [
      "bepress",
      "Statistical Methods"
    ],
    [
      "openalex",
      "3300|Computational and Text Analysis Methods"
    ]
  ],
  "openalex": "W7223045058",
  "pdf_url": "https://osf.io/zy6ew/download",
  "pmcid": null,
  "pmid": null,
  "published": "2026-10-20",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/csftyjmsa08v2.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:20:17Z",
      "native_id": "zy6ew",
      "source": "osf",
      "url": "https://osf.io/preprints/psyarxiv/zy6ew_v1/"
    }
  ],
  "subjects": [
    {
      "field": "mathematics",
      "origin": "bepress:Statistical Methods",
      "subfield": "statistics"
    }
  ],
  "title": "Modeling Word Counts: Common Practice Risks False Positives",
  "title_text": "Modeling Word Counts: Common Practice Risks False Positives",
  "type": "preprint",
  "updated_at": "2026-10-30T12:20:17Z",
  "url": "https://research.draconitic.com/paper/csftyjmsa08v2",
  "venue": "PsyArXiv",
  "versions": []
}