ProjGuard: Safety Monitoring for Computer-Use Agents via Low-Dimensional Projections
Abstract
Computer-use agents are increasingly capable of operating on real operating systems, but this capability has also increased their exposure to prompt injection and other adversarial content that can manipulate their behavior during interaction. Existing defenses typically rely on analyzing the prompt or each potentially malicious input with a second large model at inference time, which can limit coverage or increase deployment cost. We propose ProjGuard, a lightweight runtime monitor that derives a scalar risk signal from the agent's accumulated response history using a supervised linear projection. The monitor evaluates this signal at each step and raises an alert when it exceeds a fitted threshold. When an alert is raised, we selectively activate an auxiliary vision-language model (VLM) to propose the next action and steer execution back toward task completion. Experiments on OS-Harm show that monitoring with on-demand VLM intervention reduces the unsafe rate from 16% to 3% while improving task completion from 59% to 65%. We further evaluate transfer to RiosWorld, where the method remains competitive, reaching 4% unsafe and 64% completion. Overall, these results support a hierarchical safety strategy that combines continuous response monitoring with selective multimodal intervention.
Related
- Finite-dimensional normed spaces without small-norm hyperplane projections
- Multimodal computer vision and machine learning for predictive monitoring of planar-flow casting of amorphous steel foils
- The Silhouette Operator: Identifiability of Low-Rank Measures from One-Dimensional Projections
- A Review of Bayesian Surrogate Modeling for High-Dimensional Computer Simulations
- An exact computer-assisted lower bound for Fisher-information convexity along the two-dimensional heat flow
BibTeX
@misc{contreras2026projguard,
title = {{ProjGuard: Safety Monitoring for Computer-Use Agents via Low-Dimensional Projections}},
author = {Kebin Contreras and Carlos Hinojosa and Jorge Bacca and Bernard Ghanem},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.13631},
eprint = {2605.13631},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.13631},
}
RIS
TY - UNPB
TI - ProjGuard: Safety Monitoring for Computer-Use Agents via Low-Dimensional Projections
AU - Kebin Contreras
AU - Carlos Hinojosa
AU - Jorge Bacca
AU - Bernard Ghanem
PY - 2026
DA - 2026/10/26
PB - arXiv
DO - 10.48550/arxiv.2605.13631
UR - https://arxiv.org/abs/2605.13631
AB - Computer-use agents are increasingly capable of operating on real operating systems, but this capability has also increased their exposure to prompt injection and other adversarial content that can manipulate their behavior during interaction. Existing defenses typically rely on analyzing the prompt or each potentially malicious input with a second large model at inference time, which can limit coverage or increase deployment cost. We propose ProjGuard, a lightweight runtime monitor that derives a scalar risk signal from the agent's accumulated response history using a supervised linear projection. The monitor evaluates this signal at each step and raises an alert when it exceeds a fitted threshold. When an alert is raised, we selectively activate an auxiliary vision-language model (VLM) to propose the next action and steer execution back toward task completion. Experiments on OS-Harm show that monitoring with on-demand VLM intervention reduces the unsafe rate from 16% to 3% while improving task completion from 59% to 65%. We further evaluate transfer to RiosWorld, where the method remains competitive, reaching 4% unsafe and 64% completion. Overall, these results support a hierarchical safety strategy that combines continuous response monitoring with selective multimodal intervention.
ER -