Skip to content
DraconiticResearch
Search Sources API
197,785 studies31 sourcesUpdated 16 min ago
Peer-reviewedCC BY 4.0Frontiers in Neurology

Inter-rater reliability of needle EMG pattern classification among physicians

  • Ekaterina Seliverstova1,2
  • Artem Pyatkov3
  • Mariya Kovalchuk4
  • Evgenia Druzhinina5
  • Mikhail Sinkin6,5
6 affiliations
  1. 1I.V. Davydovsky City Clinical Hospital of the Moscow Department of Health
  2. 2Burnasyan Federal Medical Biophysical Center of Federal Medical Biological Agency
  3. 3I.M. Sechenov First Moscow State Medical University (Sechenov University)
  4. 4Neuro-muscular Disorders Clinic
  5. 5Pirogov Russian National Research Medical University
  6. 6N.V. Sklifosovsky Research Institute for Emergency Medicine
Read full textPDFDOICite

Abstract

Objective: To evaluate the inter-rater reliability of needle electromyographic (EMG) assessment among 19 physicians with varying clinical experience and specialties using a consensus standard as reference.

Methods: The study followed the GRRAS guidelines. First, four EMG experts agreed on classifying thirty EMG recordings as normal, myogenic, or neurogenic and referred to as a consensus standard (CS). Second, those anonymised recordings were independently classified by 19 non-expert physicians. Thereafter, Cohen’s kappa (κ) was calculated for each physician against the CS. Fleiss’ κ was used for multi-rater agreement. Category-specific (one-vs-rest) κ, sensitivity and specificity were computed. Subgroup analyses by experience and specialty were performed. Post-hoc power analysis was used for the specialty comparison.

Results: The mean κ versus the consensus standard across all physicians was 0.688 (median 0.747, SD 0.172; range 0.287–0.900). Overall multi-rater agreement (Fleiss’ κ) was 0.581. Category-specific κ values were: neurogenic 0.723, normal 0.368, myogenic 0.334. Sensitivity/specificity: neurogenic 94.6%/87.1%, myogenic 69.7%/96.1%, normal 78.2%/88.9%. Agreement with the CS increased with experience: mean κ rose from 0.365 (group 1, ≤1 year) to 0.798 (group 4, >10 years). No significant difference was found between neurologists and clinical physiologists (Mann–Whitney U, p = 0.174); post-hoc power was low (≈45%), thus non-significance should not be interpreted as equivalence. EMG experts showed almost perfect agreement ( κ = 0.85–0.95).

Conclusion: In this exploratory study, physicians demonstrated moderate to substantial agreement with a CS standard in EMG classification. Neurogenic changes are identified most reliably, whereas differentiation between normal and myogenic changes remains challenging. Greater experience was associated with better agreement in our sample, but this finding requires confirmation in larger studies. No statistically significant difference was found between neurologists and clinical physiologists, although the analysis was underpowered. Individual competency varied widely within specialties, suggesting that objective, competency-based qualification may be more relevant than specialty alone.

Related

  1. Inter‐Rater Reliability in Pre‐Anaesthetic Airway Assessment: A Protocol for a Systematic Review
    Constance Thyra Giertz Christiansen, Mathilde Nellemann Jensen, Moritz Kilian German Denneborg +8
    Acta anaesthesiologica ScandinavicaPeer-reviewedCC BYFull textGeneral medicineEurope PMC
  2. Intra- and Inter-Operator Reliability of Internal Jugular Vein Ultrasound Image Acquisition by Ultrasound-Naive Nurses.
    Yukie Tamura, Kazumi Ishii, Miharu Matsumoto +1
    Healthcare (Basel, Switzerland)Peer-reviewedCC BYFull textGeneral medicineEurope PMC
  3. Timing and Context Features in Machine Learning Classification of Inter-Patient ECG Heartbeats on the MIT-BIH Benchmark
    E. V. Alforja, J. C. Aban
    medRxivPreprintAll rights reservedFull textEpidemiology & public healthmedRxiv
  4. Benchmarking Generative AI Chatbots in Preventive Medicine: A Comparative Analysis of Accuracy, Completeness, and Inter-Rater Reliability.
    Nojoud Zoraib Alshahrani, Abdullah Alsabaani, Ali Hassan Almaqsudi +3
    Healthcare (Basel, Switzerland)Peer-reviewedCC BYFull textEpidemiology & public healthEurope PMC
  5. Inter-rater reliability during live pediatric formative Objective Structured Clinical Examination (OSCE) in a French University
    Aurélie Pham, Florian Blanchard, Antoine Monsel +13
    Archives de PédiatriePeer-reviewedCC BY 4.0Full textEpidemiology & public healthHAL

Cite

BibTeXRIS
BibTeX
@article{seliverstova2026interrater,
  title = {{Inter-rater reliability of needle EMG pattern classification among physicians}},
  author = {Ekaterina Seliverstova and Artem Pyatkov and Mariya Kovalchuk and Evgenia Druzhinina and Mikhail Sinkin},
  journal = {Frontiers in Neurology},
  year = {2026},
  month = oct,
  doi = {10.3389/fneur.2026.1982862},
  url = {https://www.frontiersin.org/articles/10.3389/fneur.2026.1982862/full},
  note = {CC BY 4.0},
}
RIS
TY  - JOUR
TI  - Inter-rater reliability of needle EMG pattern classification among physicians
AU  - Ekaterina Seliverstova
AU  - Artem Pyatkov
AU  - Mariya Kovalchuk
AU  - Evgenia Druzhinina
AU  - Mikhail Sinkin
PY  - 2026
DA  - 2026/10/15
JO  - Frontiers in Neurology
DO  - 10.3389/fneur.2026.1982862
UR  - https://www.frontiersin.org/articles/10.3389/fneur.2026.1982862/full
AB  - Objective: To evaluate the inter-rater reliability of needle electromyographic (EMG) assessment among 19 physicians with varying clinical experience and specialties using a consensus standard as reference.  Methods: The study followed the GRRAS guidelines. First, four EMG experts agreed on classifying thirty EMG recordings as normal, myogenic, or neurogenic and referred to as a consensus standard (CS). Second, those anonymised recordings were independently classified by 19 non-expert physicians. Thereafter, Cohen’s kappa (κ) was calculated for each physician against the CS. Fleiss’ κ was used for multi-rater agreement. Category-specific (one-vs-rest) κ, sensitivity and specificity were computed. Subgroup analyses by experience and specialty were performed. Post-hoc power analysis was used for the specialty comparison.  Results: The mean κ versus the consensus standard across all physicians was 0.688 (median 0.747, SD 0.172; range 0.287–0.900). Overall multi-rater agreement (Fleiss’ κ) was 0.581. Category-specific κ values were: neurogenic 0.723, normal 0.368, myogenic 0.334. Sensitivity/specificity: neurogenic 94.6%/87.1%, myogenic 69.7%/96.1%, normal 78.2%/88.9%. Agreement with the CS increased with experience: mean κ rose from 0.365 (group 1, ≤1 year) to 0.798 (group 4, >10 years). No significant difference was found between neurologists and clinical physiologists (Mann–Whitney U, p = 0.174); post-hoc power was low (≈45%), thus non-significance should not be interpreted as equivalence. EMG experts showed almost perfect agreement ( κ = 0.85–0.95).  Conclusion: In this exploratory study, physicians demonstrated moderate to substantial agreement with a CS standard in EMG classification. Neurogenic changes are identified most reliably, whereas differentiation between normal and myogenic changes remains challenging. Greater experience was associated with better agreement in our sample, but this finding requires confirmation in larger studies. No statistically significant difference was found between neurologists and clinical physiologists, although the analysis was underpowered. Individual competency varied widely within specialties, suggesting that objective, competency-based qualification may be more relevant than specialty alone.
SN  - 1664-2295
ER  -