Behavioral Skill Reconstruction: Reconstructing Hidden Functionality from LLM Agent Skills

Summary

This paper shows that a closed-source LLM agent skill can be cloned through ordinary use, even when its files stay hidden. An agent skill is a package of instructions, scripts, and data that an agent loads on demand, and providers may sell skills as services without releasing the package. The attack, SKILLCLONE, sends valid task requests, observes the answers, writes code that reproduces them, and repairs that code against the real skill. It recovers 18 of 21 mined skills above a trivial baseline on the strongest victim model, and a disclosure filter flags only 6.7% of its probes.

Key numbers

• 18 of 21Mined skills reconstructed above the trivial-baseline floor on the strongest victim model
• 6.7%Share of SKILLCLONE's task-valid probes flagged by a disclosure-oriented input detector, which catches all four explicit disclosure probes
• 77–87%Median attack success rate on held-out inputs for rule, table, composition, and procedure skills (43% for numeric formulas)

Method

Earlier skill-stealing attacks use prompt injection to make an agent print a skill's files, and defenses target that leakage ^[1]. Behavioral skill reconstruction (BSR) asks a different question: can a user rebuild what a skill does from valid requests and responses alone? The attacker is a black box that queries one skill-enabled agent and sees only its visible messages and files. The skill's own body is never visible. Because the attack stays within normal use, it resembles classical model extraction, in which an attacker learns a model's behavior from its predictions ^[2][3].

SKILLCLONE runs four steps. It forms an interface hypothesis from the skill's public advertisement, the description a customer already sees. It issues structured benign probes, synthesizes an executable replica from the responses, and repairs the replica by differential validation, which compares the clone's outputs with the real skill's on new inputs and fixes the mismatches. Iterative requerying repeats the loop to cover gaps. Recovery is measured by model-relative marginal functionality: a component counts only if the skill-enabled agent applies it reliably and the same base model cannot reproduce it from the public description alone, which screens out what the model already knows.

Evaluation

The suite has 30 skills: 21 mined from a public registry crawl and SkillRet ^[4], and 9 from SkillsBench ^[5]. They cover threshold and decision rules, lookup tables, rule composition, procedures, and numeric formulas. Attack success rate (ASR) is the share of held-out inputs on which the clone's output exactly equals the skill's, and each skill has a floor, the ASR of a trivial baseline. On the strongest victim, SKILLCLONE recovers 18 of the 21 mined skills above floor and produces exact clones for several deterministic procedures.

Median ASR is 83% for threshold and decision rules (4 of 5 skills above floor), 82% for lookup tables (7 of 10), 87% for rule composition (3 of 3), 77% for procedures and algorithms (4 of 6), and 43% for numeric formulas (3 of 6). Controlled in-context experiments show that iterative requerying closes coverage gaps that defeat a single reconstruction round.

Two defenses were measured. A disclosure-oriented filter detects all four explicit disclosure probes, and its input detector flags 6.7% of SKILLCLONE's task-valid probes. Rewriting public skill descriptions to be less detailed gives inconsistent protection. Keeping files secret therefore does not keep the functionality secret, and defenses also need to limit the information that accumulates across legitimate interactions.

Safety and limitations

This paper describes an attack on a class of products, so scope matters. All 30 skill bodies are public artifacts, and no proprietary service, paid marketplace, or deployed agent in the wild was attacked. The paper measures how exposed disclosure-defended deployments are today and does not claim that deterministic functions are learnable in general. It evaluates two defenses from earlier work, disclosure filtering and advertisement rewriting, and it does not present a defense of its own. Cross-session identity controls, cumulative query monitoring, output coarsening, and private routing representations remain untested.

Scoring uses exact equivalence on held-out inputs, which suits deterministic rules, tables, formulas, and procedures. Creative, interactive, and judgment-heavy skills need other metrics, such as human evaluation that separates skill behavior from the base model. The victims are five models (DeepSeek V4 Flash and Pro, GLM-5.1, Kimi-K2.6, and GPT-5.6-Luna), results vary across skill and victim pairs, and other models can behave differently.

Citation

@article{hua2026skillclone,
      title={Behavioral Skill Reconstruction: Reconstructing Hidden Functionality from LLM Agent Skills}, 
      author={Peichun Hua and Haoxuan Xu and Mengyuan Li},
      journal={CoRR},
      volume={abs/2608.04192},
      year={2026},
      url={https://arxiv.org/abs/2608.04192},
      doi={10.48550/arXiv.2608.04192},
      eprinttype={arXiv},
      eprint={2608.04192},
      primaryClass={cs.CR}
}

References

1. Anthropic. Equipping agents for the real world with Agent Skills. Anthropic Engineering Blog, 2025. www.anthropic.com/research/equipping-agents-for-the-real-world-with-agent-skills
2. Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. Stealing Machine Learning Models via Prediction APIs. USENIX Security, 2016. arxiv.org/abs/1609.02943
3. Nicholas Carlini et al. Stealing Part of a Production Language Model. ICML, 2024. arxiv.org/abs/2403.06634
4. Hongcheol Cho, Ryangkyung Kang, and Youngeun Kim. SkillRet: A Large-Scale Benchmark for Skill Retrieval in LLM Agents. arXiv, 2026. arxiv.org/abs/2605.05726
5. Xiangyi Li et al. SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks. arXiv, 2026. arxiv.org/abs/2602.12670