Semantic Context Alignment with Open-World Learning for Memory-Driven Human Activity Recognition
- 1University of Hertfordshire
Abstract
Agentic AI systems that combine perception with memory must repeatedly solve a context alignment problem:
deciding whether a current observation matches something already held in long-term memory, or whether it
is new. The mechanism used to make that decision, symbolic (lexical) or sub-symbolic (embedding-based),
determines whether the system’s memory generalises across vocabulary variation or fragments into redundant,
near-duplicate entries. This study investigates that choice inside an agentic memory architecture, using
Human Activity Recognition as the applied testbed rather than the object of study. Building on the
memory-driven architecture proposed by Shahabian Alashti et al. [1], a Jaccard-and-Bayesian symbolic
context alignment engine is replaced with a semantic engine that computes cosine similarity over Sentence
Transformer embeddings, while every other component of the pipeline, including the Bayesian posterior
and the 60% confidence-gating threshold, is held fixed. Both configurations are evaluated in a closed loop
on seen (Robot House Multi-View, N = 420) and unseen (Toyota Smarthome, N = 930) data, at roughly
three times the scale of the pilot study this project began as. Long-term memory improves accuracy over a
memory-free baseline on both datasets, confirming its functional role in the architecture. On seen data, the
semantic mechanism outperforms the symbolic baseline narrowly (61.60% versus 60.90%). On unseen data
the gap widens substantially (49.80% versus 37.40%), while the supervised classifier alone collapses from
94.00% to 38.30% once evaluated outside its training distribution. These results indicate that the advantage
of semantic over symbolic context alignment is concentrated in generalisation under distribution shift rather
than in-distribution accuracy, a finding with implications for the design of context alignment mechanisms in
memory-driven AI systems beyond activity recognition specifically.
Related
- A Masked Representation Alignment-Based Self-Supervised Learning Method for Radar Emitter Recognition
- Katalyst: knowledge-guided semantic alignment via contrastive learning for enzyme turnover prediction
- Semantic Alignment of EEG and Text: A Contrastive Learning Framework for Decoding Imagined Speech
- Self supervised multi scale learning for WiFi CSI contactless human activity recognition with potential applications in health monitoring
- Enhanced labor activity recognition via wearable physiological signals: a comparative machine learning study
BibTeX
@misc{gideon2026semantic,
title = {{Semantic Context Alignment with Open-World Learning for Memory-Driven Human Activity Recognition}},
author = {Ganah Gideon and Abolfazl Zaraki},
howpublished = {Zenodo},
year = {2026},
month = oct,
doi = {10.5281/zenodo.23598103},
url = {https://zenodo.org/records/23598103},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Semantic Context Alignment with Open-World Learning for Memory-Driven Human Activity Recognition
AU - Ganah Gideon
AU - Abolfazl Zaraki
PY - 2026
DA - 2026/10/23
PB - Zenodo
DO - 10.5281/zenodo.23598103
UR - https://zenodo.org/records/23598103
AB - Agentic AI systems that combine perception with memory must repeatedly solve a context alignment problem: deciding whether a current observation matches something already held in long-term memory, or whether it is new. The mechanism used to make that decision, symbolic (lexical) or sub-symbolic (embedding-based), determines whether the system’s memory generalises across vocabulary variation or fragments into redundant, near-duplicate entries. This study investigates that choice inside an agentic memory architecture, using Human Activity Recognition as the applied testbed rather than the object of study. Building on the memory-driven architecture proposed by Shahabian Alashti et al. [1], a Jaccard-and-Bayesian symbolic context alignment engine is replaced with a semantic engine that computes cosine similarity over Sentence Transformer embeddings, while every other component of the pipeline, including the Bayesian posterior and the 60% confidence-gating threshold, is held fixed. Both configurations are evaluated in a closed loop on seen (Robot House Multi-View, N = 420) and unseen (Toyota Smarthome, N = 930) data, at roughly three times the scale of the pilot study this project began as. Long-term memory improves accuracy over a memory-free baseline on both datasets, confirming its functional role in the architecture. On seen data, the semantic mechanism outperforms the symbolic baseline narrowly (61.60% versus 60.90%). On unseen data the gap widens substantially (49.80% versus 37.40%), while the supervised classifier alone collapses from 94.00% to 38.30% once evaluated outside its training distribution. These results indicate that the advantage of semantic over symbolic context alignment is concentrated in generalisation under distribution shift rather than in-distribution accuracy, a finding with implications for the design of context alignment mechanisms in memory-driven AI systems beyond activity recognition specifically.
ER -