PromptSentinel-X: A Leakage-Aware and Context-Aware Framework for Prompt-Injection Detection in Large Language Model-Powered Web Agents

Large language model (LLM)-powered web agents combine privileged instructions with user requests, webpages, retrieved documents, tool outputs, memory, and conversation history, creating prompt-injection risks that static text classification may not capture. This study presents PromptSentinel-X, a leakage-aware and context-aware screening framework. The primary benchmark contained 1581 English records from the Prompt Injection Malignant dataset; 30,015 external prompt-injection, jailbreak, and benign hard-negative records were reserved for transfer and stress testing.

The framework uses prompt-family-aware partitioning, trusted–untrusted context segmentation, calibrated risk prediction, robustness analysis, and deployment-oriented routing. On a 465-record group-aware test set, PromptSentinel-X achieved 0.9849 accuracy, 0.8887 macro-F1, 0.9851 attack recall, a 0.0050 benign false-positive rate, 0.9971 area under the receiver operating characteristic curve (AUROC), 0.9836 area under the precision-recall curve (AUPRC), and 0.0153 expected calibration error. Random splitting produced a higher baseline macro-F1 but 144 leakage warnings.

Context-aware macro-F1 declined from 0.8301 for static prompts to 0.6306 for multi-turn, 0.3844 for retrieval-augmented generation, and 0.3200 for memory scenarios. PromptSentinel-X is positioned as a calibrated screening and escalation component, not a stand-alone authorization mechanism. Broader tool, memory, multimodal browser, multilingual, and end-to-end agent studies remain necessary.