Securing On-device Transformer with Hardware Binding and Reversible Obfuscation

Summary

This paper keeps a transformer model usable only on the hardware it was sold for. A model deployed on a phone or sensor can be copied from storage, and watermarks only prove ownership after the theft. The scheme derives a key from a physical unclonable function (PUF, a circuit whose response depends on random manufacturing variation) and uses it to scramble selected weight matrices in a reversible way. Without the key, a ViT-B image classifier falls from 80.33% to 0.10% top-1 accuracy on ImageNet, and on the authorized device it keeps its original accuracy without retraining.

Key numbers

• 80.33% → 0.10%ViT-B top-1 accuracy on ImageNet without the correct key, with 6 layers encrypted; the authorized device keeps 80.33%
• 56.4%Best ViT-B accuracy an attacker reaches by retraining the encrypted model on 20% of the original data
• 1.119×Inference time on CPU with 12 layers encrypted and 3 Arnold's Cat Map iterations (1.58 to 1.67× on GPU)

Method

The scheme protects the multi-head attention and feed-forward weights, which hold most of a transformer's parameters ^[1]. A PUF on the device ^[2] reproduces a key from silicon variation, so the key is never stored in memory where an attacker could read it. A key derivation function expands it into one key per encrypted layer. Each attention weight matrix is scrambled with Arnold's Cat Map (ACM), a chaotic transposition originally used to shuffle image pixels ^[3], whose reversibility follows from a determinant of 1 in its 2 × 2 matrix. The feed-forward matrices are not square, so the scheme permutes their rows with a key-dependent permutation instead.

Authorized hardware decrypts a layer, runs it, and re-encrypts it before the next layer, which limits how long plaintext weights stay in memory. The number of ACM iterations is fixed so that decryption time does not depend on the key, which blocks timing side channels. A selection algorithm encrypts the layers whose obfuscation hurts accuracy most, then adds a few random layers (4 for base-sized models, 6 for large ones) so that guessing one layer's key gives no measurable signal. The method works on the weights alone and needs no retraining or framework-specific features.

Evaluation

The evaluation covers ViT ^[4], Swin, DeiT, and BEiT models, including larger input resolutions and quantized versions, on ImageNet. The decrypted accuracy equals the original in every case. Without the key, encrypting layers 0 and 10 of ViT-B drops accuracy to 0.08%, and the final set of 6 layers gives 0.10%. For ViT-L, encrypting layer 1 alone gives 0.1%.

An attacker who retrains the whole encrypted ViT-B on 20% of the original data reaches 56.4% accuracy after 5 epochs, against 80.33% for the original model. Retraining on 10% of the data reaches 47.9%. Retraining only the encrypted layers gives accuracy close to training from scratch. Incorrect keys at any distance from the true key produce severely degraded accuracy, a strong avalanche effect, and decrypting a single layer correctly changes accuracy by at most 0.04%, so a lucky guess confirms nothing.

On a CPU, encrypting 12 layers with 3 ACM iterations multiplies ViT-B inference time by 1.119 at batch size 64. On a GPU, the forward pass is faster, so the same work is a larger share: 1.671× for 12 iterations and 1.580× for 12 encrypted layers. ACM operations dominate the overhead, and the feed-forward permutations add little.

Limitations

The security argument assumes a robust PUF: tampering destroys it, a fuzzy extractor yields reliable keys across temperature and voltage, and the PUF readout interface is removed after registration. The attacker can read the encrypted model, its architecture, and a subset of the training data, but not runtime memory or PUF keys. An attacker with runtime memory access to an authorized device is outside the model, and plaintext weights exist briefly during each layer's execution.

The evaluation covers vision transformers. ACM is chaotic in the mathematical sense and the proof ignores finite-precision arithmetic. An attacker who holds both the permuted weights and the original public model, for example after fine-tuning with LoRA, might align columns to recover permutations, and the paper lists this as an open attack surface. The GPU overhead is 58 to 67% in the tested configurations, and the authors treat the scheme as a pilot study for transformer protection.

Full paper citation

@inproceedings{hua2025securing,
  title={Securing On-device Transformer with Hardware Binding and Reversible Obfuscation},
  author={Hua, Peichun and Zhang, Hanxiu and Li, Tuo and Zheng, Yue},
  pages={956-972},
  booktitle={2025 IEEE Annual Computer Security Applications Conference (ACSAC)}, 
  year={2025},
  organization={IEEE},
  keywords={Deep learning;Computational modeling;Intellectual property;Transformers;Physical unclonable function;Hardware;Software;Data models;Protection;Software development management;Intellectual Property;Transformer;Model Stealing Attacks;Physical Unclonable Functions},
  doi={10.1109/ACSAC67867.2025.00079},
}

References

1. Ashish Vaswani et al. Attention Is All You Need. NeurIPS, 2017. arxiv.org/abs/1706.03762
2. Blaise Gassend, Dwaine Clarke, Marten van Dijk, and Srinivas Devadas. Silicon physical random functions. ACM CCS, 2002. doi.org/10.1145/586110.586132
3. Jiri Fridrich. Symmetric Ciphers Based on Two-Dimensional Chaotic Maps. International Journal of Bifurcation and Chaos, 1998. doi.org/10.1142/S021812749800098X
4. Alexey Dosovitskiy et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. ICLR, 2021. arxiv.org/abs/2010.11929