property-based-testing
by trailofbitsproperty-based-testing skill guide for writing, reviewing, and improving PBT across languages and smart contracts. Use this property-based-testing guide to spot roundtrip, idempotence, invariant, parser, validator, and normalization cases, choose generators, and decide when property-based-testing is stronger than example-based tests.
This skill scores 83/100, which means it is a solid directory listing: it gives agents enough trigger rules and workflow guidance to be meaningfully more useful than a generic prompt, though users should still expect some manual judgment for edge cases and language-specific implementation details.
- Explicit automatic-detection triggers for common PBT patterns like encode/decode pairs, parsers, normalizers, validators, pure functions, and smart contracts.
- Strong operational content across the lifecycle: designing properties, generating tests, reviewing failures, and refactoring for testability.
- Good install decision value from broad language/tool coverage plus seven reference files that deepen the workflow without placeholder content.
- No install command in SKILL.md, so adoption requires users to wire the skill into their environment manually or infer the setup.
- Marked experimental/test signal and some documentation is reference-heavy, so users may need to interpret guidance rather than follow a single end-to-end runnable workflow.
Overview of property-based-testing skill
What this skill is for
The property-based-testing skill helps you write, review, and improve property-based tests when example-based tests stop giving enough confidence. It is especially useful for the property-based-testing skill when you need to validate roundtrips, invariants, parsers, validators, normalizers, sorting logic, or smart contract state rules.
Who benefits most
Best fit: engineers working in languages with mature PBT libraries, reviewers checking weak tests, and developers designing features that should be specified as properties first. If you are deciding whether to install property-based-testing, the key question is whether your code has reusable structure, reversible transforms, or rules that should hold across many inputs.
What makes it different
Unlike a generic testing prompt, this skill is organized around detection, property choice, strategy design, and failure interpretation. That matters because the biggest adoption blocker is not syntax; it is picking a useful property and generating valid inputs without over-filtering. The skill is also broad enough for property-based-testing for Skill Testing across multiple ecosystems, including smart contracts.
How to Use property-based-testing skill
Install and load the right context
Install with npx skills add trailofbits/skills --skill property-based-testing, then open SKILL.md first. For faster decisions, also read README.md and the reference files most relevant to your task: references/generating.md, references/reviewing.md, and references/interpreting-failures.md.
Turn a rough idea into a usable prompt
A good property-based-testing usage prompt should name the target, the operation, and the guarantee you care about. Better input: “Add PBT for decode(encode(x)) on this serializer, keeping inputs within valid UTF-8 and size limits.” Worse input: “Write property tests for this module.” The first gives the skill a real property, valid domain boundaries, and an output shape.
What to provide in your request
Include the language, test library, function signature, known invariants, invalid-input behavior, and any examples or specs. If you are using the property-based-testing guide for a review, paste the current test and explain whether you want bug-finding, refactoring, or a quality audit. If the code has parsing or validation paths, say what should happen on malformed data; that prevents vague tests and bad assume() usage.
Recommended workflow
Start by mapping the code to a property type: roundtrip, idempotence, invariant, ordering, or oracle. Then define strategies that generate valid inputs by construction, not by heavy filtering. Finally, interpret failures with references/interpreting-failures.md before treating them as bugs. For smart contracts, check the EVM-specific notes and keep the state model explicit.
property-based-testing skill FAQ
Is property-based-testing a replacement for normal tests?
No. The skill is most valuable where ordinary examples miss edge cases or where one property can cover many behaviors. Keep example tests for narrow regressions and use property-based-testing where a general rule is clearer than a handful of cases.
Is this property-based-testing guide beginner-friendly?
Yes, if the target behavior is simple and the property is obvious, such as a roundtrip or idempotence check. It is harder when the spec is underspecified, because PBT needs a clear contract before the generator is useful.
When should I not use this skill?
Do not reach for it first when the behavior is mostly UI-driven, highly nondeterministic, or impossible to state as a stable property. It is also a poor fit when the only available oracle is a long chain of reimplementation that just mirrors the production code.
Does it fit my language or framework?
The skill supports many ecosystems, including Hypothesis, fast-check, proptest, jqwik, ScalaCheck, FsCheck, StreamData, QuickCheck, and smart contract fuzzers like Echidna and Medusa. If your stack is outside that list, the underlying method still helps, but you may need to adapt the test syntax and strategy APIs.
How to Improve property-based-testing skill
Give the strongest property, not just the target
The best results come from naming the exact guarantee. For example: “normalize(normalize(x)) == normalize(x) for Unicode strings” is better than “test normalize.” The property-based-testing skill performs better when the property is specific, falsifiable, and tied to a known contract.
Share constraints that shape generators
Common failure modes are invalid inputs, over-filtered strategies, and properties that are too weak to catch real bugs. Improve output quality by stating acceptable ranges, format rules, size limits, and whether exceptions are expected. If a property should only hold after validation, say so explicitly.
Iterate from failing examples
When the first test fails, do not just ask for a fix. Provide the shrunk counterexample, the intended rule, and any documentation that defines edge-case behavior. That lets the skill separate a real bug from a bad property and produce a stronger second-pass test or refactor suggestion.