Publications

* = equal contribution. See Google Scholar for a full list.

2026

SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation

Yibo Peng, Long Lian, David Wagner, Sizhe Chen

EMNLPConference on Empirical Methods in Natural Language Processing

Code

Reference-Based Distillation Detection in LLMs

Rajat Rawat, Sizhe Chen, Akshay Anand, Michael Duan, Bob Rotsted, Sewon Min

PreprintarXiv:2607.09692

Paper Code

Defending Against Prompt Injection with DataFilter

Yizhu Wang, Sizhe Chen, Raghad Alkhudair, Basel Alomair, David Wagner

SaTMLIEEE Conference on Secure and Trustworthy Machine Learning

Paper Model Code

2025

Meta SecAlign: A Secure Foundation LLM Against Prompt Injection Attacks

Sizhe Chen*, Arman Zharmagambetov, David Wagner, Chuan Guo*

PreprintarXiv:2507.02735

Meta-SecAlign models, with an order-of-magnitude lower attack success rate against prompt injections, have been downloaded 10K times in three months. Meta-SecAlign-70B retains commercial-grade utility after defensive fine-tuning and is available for commercial use.

Paper 8B Model 70B Model Poster Slides Code

SecAlign: Defending Against Prompt Injection with Preference Optimization

Sizhe Chen, Arman Zharmagambetov, Saeed Mahloujifar, Kamalika Chaudhuri, David Wagner, Chuan Guo

CCSACM Conference on Computer and Communications Security

SecAlign preference-optimizes an LLM to favor secure responses over responses that follow injected instructions.

DOI Paper Website Poster Slides Media Code

StruQ: Defending Against Prompt Injection with Structured Queries

Sizhe Chen, Julien Piet, Chawin Sitawarin, David Wagner

USENIX SecurityUSENIX Security Symposium

StruQ separates the user instruction and untrusted data into distinct channels to defend against prompt injection.

Paper arXiv Website Poster Slides Code

Defending Against Prompt Injection with a Few Defensive Tokens

Sizhe Chen, Yizhu Wang, Nicholas Carlini, Chawin Sitawarin, David Wagner

AISec SpotlightACM Workshop on Artificial Intelligence and Security

DOI Paper Slides Code

2024

Jatmo: Prompt Injection Defense by Task-Specific Finetuning

Julien Piet, Maha Alrashed, Chawin Sitawarin, Sizhe Chen, Zeming Wei, Elizabeth Sun, Basel Alomair, David Wagner

ESORICSEuropean Symposium on Research in Computer Security

DOI Paper Code

Query Attack by Multi-Identity Surrogates

Sizhe Chen, Zhehao Huang, Qinghua Tao, Xiaolin Huang

IEEE TAIIEEE Transactions on Artificial Intelligence, 5(2):684–697

DOI Paper Code

2023

Can LLMs Follow Simple Rules?

Norman Mu, Sarah Chen, Zifan Wang, Sizhe Chen, David Karamardian, Lulwa Aljeraisy, Basel Alomair, Dan Hendrycks, David Wagner

PreprintarXiv:2311.04235

Paper Code

One-Pixel Shortcut: On the Learning Preference of Deep Neural Networks

Shutong Wu*, Sizhe Chen*, Cihang Xie, Xiaolin Huang

ICLR SpotlightInternational Conference on Learning Representations

Paper Poster Talk Slides Code

Self-Ensemble Protection: Training Checkpoints Are Good Data Protectors

Sizhe Chen, Geng Yuan, Xinwen Cheng, Yifan Gong, Minghai Qin, Yanzhi Wang, Xiaolin Huang

ICLRInternational Conference on Learning Representations

Paper Code

Investigating Catastrophic Overfitting in Fast Adversarial Training: A Self-Fitting Perspective

Zhengbao He, Tao Li, Sizhe Chen, Xiaolin Huang

CVPRWCVPR Workshop on Adversarial Machine Learning

Paper

Measuring the Transferability of ℓ∞ Attacks by the ℓ2 Norm

Sizhe Chen, Qinghua Tao, Zhixing Ye, Xiaolin Huang

ICASSPIEEE International Conference on Acoustics, Speech and Signal Processing

Paper

Unifying Gradients to Improve Real-World Robustness for Deep Networks

Yingwen Wu, Sizhe Chen, Kun Fang, Xiaolin Huang

ACM TISTACM Transactions on Intelligent Systems and Technology, 14(6)

DOI Paper Code

2022

Universal Adversarial Attack on Attention and the Resulting Dataset DAmageNet

Sizhe Chen, Zhengbao He, Chengjin Sun, Jie Yang, Xiaolin Huang

IEEE TPAMIIEEE Transactions on Pattern Analysis and Machine Intelligence

Paper arXiv Slides Code

Subspace Adversarial Training

Tao Li, Yingwen Wu, Sizhe Chen, Kun Fang, Xiaolin Huang

CVPR OralIEEE/CVF Conference on Computer Vision and Pattern Recognition

Paper arXiv Poster Talk Slides Code

Adversarial Attack on Attackers: Post-Process to Mitigate Black-Box Score-Based Query Attacks

Sizhe Chen, Zhehao Huang, Qinghua Tao, Yingwen Wu, Cihang Xie, Xiaolin Huang

NeurIPSConference on Neural Information Processing Systems

Paper Code

Relevance Attack on Detectors

Sizhe Chen, Fan He, Xiaolin Huang, Kun Zhang

Pattern RecognitionVolume 124, Article 108491

DOI Paper Code