Test whether a skill actually helps

A longer skill file is not evidence of a better agent. In a Google Cloud experiment, a baseline coding skill performed about the same as using no skill. That finding came from one evaluation setup, not a verdict on all skills.

Measure your own workflow

1. Choose a small set of representative tasks and write down what counts as success.
2. Run the same tasks without the new skill. Keep the results as your baseline.
3. Add the skill and compare outcomes under similar conditions. Record failures and timeouts, not just successful examples.
4. Improve the instruction that addresses an observed failure.
5. Keep some tasks out of the editing process, then use them to check the final version.

Google’s team compared candidate skills with a baseline and reserved unseen tasks for a final check. They also found that slow model responses and growing logs could disrupt evaluation.