senior-prompt-engineer
by alirezarezvanisenior-prompt-engineer is a model-agnostic Prompt Writing skill for eval-driven prompt optimization, RAG quality checks, agent workflow validation, and token/cost budgeting. Includes references plus Python tools for prompt analysis, RAG metrics, and agent orchestration.
This skill scores 84/100, which means it is a solid listing candidate for directory users who want an eval-driven prompt engineering workflow rather than a generic prompting checklist. The repository provides clear triggers, concrete operating rules, practical scripts, and supporting references, though adoption would be easier with an explicit quick start and clearer installation guidance.
- Strong triggerability: the frontmatter explicitly covers prompt optimization, eval sets, RAG quality, agent/tool validation, structured outputs, and token/cost budgeting.
- Operational workflow is substantive: SKILL.md includes rules such as baseline-first iteration, eval-set-before-optimization, and preferring platform-native structured outputs over prompt hacks.
- Useful supporting assets: three stdlib Python tools address prompt analysis/optimization, RAG evaluation, and agent workflow validation, backed by reference docs on prompt patterns, evaluation frameworks, and agentic system design.
- No install command or README is present in the skill path, so users may need to infer how to install or invoke the bundled scripts from SKILL.md and script usage blocks.
- The evaluation scripts appear intentionally lightweight and model-agnostic, relying on approximations such as lexical overlap and chars-per-token estimates rather than provider-native evaluators or embeddings.
Overview of senior-prompt-engineer skill
What senior-prompt-engineer is for
senior-prompt-engineer is a model-agnostic Claude skill for serious Prompt Writing work: improving prompts with baselines, building prompt templates, evaluating LLM outputs, checking RAG retrieval quality, validating agent/tool workflows, and estimating token or cost impact. It is best for users who want prompt engineering to behave like an engineering workflow rather than a one-off rewrite request.
Best-fit users and jobs
Install this skill if you are maintaining production prompts, comparing prompt variants, creating structured-output contracts, testing a RAG pipeline, or designing an agent that calls tools. The strongest fit is a developer, AI product builder, prompt engineer, or technical operator who can provide sample inputs, expected outputs, retrieval contexts, or agent configuration files.
Key differentiators
The main difference from ordinary prompt advice is its eval-first operating style. The skill explicitly discourages “optimize by vibes” and pushes you to capture a baseline before changing prompts. It also includes three Python stdlib tools: prompt_optimizer.py for prompt analysis, rag_evaluator.py for retrieval and groundedness checks, and agent_orchestrator.py for workflow validation.
What to know before installing
The skill is practical but not magic. It does not ship a full hosted evaluation platform, does not hardcode provider pricing, and does not replace native platform features like JSON schema enforcement or tool-use APIs. Its value comes from giving your AI assistant a structured workflow, references, and local helper scripts for better prompt decisions.
How to Use senior-prompt-engineer skill
senior-prompt-engineer install and first files to read
Install with:
npx skills add alirezarezvani/claude-skills --skill senior-prompt-engineer
The upstream path is engineering-team/skills/senior-prompt-engineer. After install, read SKILL.md first for the operating rules and trigger conditions. Then inspect:
references/prompt_engineering_patterns.mdfor prompt patterns and examplesreferences/llm_evaluation_frameworks.mdfor metric selection and eval designreferences/agentic_system_design.mdfor ReAct, plan-and-execute, tool use, and multi-agent patternsscripts/prompt_optimizer.pyscripts/rag_evaluator.pyscripts/agent_orchestrator.py
Inputs that make the skill work well
For strong senior-prompt-engineer usage, give the assistant more than “make this prompt better.” Provide the current prompt, the task goal, target model or provider constraints, 10–20 representative test cases if available, expected output format, known failure examples, and any limits on latency, token budget, or cost.
A weak request is:
Improve this support bot prompt.
A stronger request is:
Use senior-prompt-engineer to improve this support triage prompt. Baseline failures: it misclassifies billing refunds as technical issues and often omits escalation reasons. Required output is JSON with
category,priority,escalation_reason, andconfidence. Here are 12 labeled examples and 3 bad outputs from production. Keep the prompt under 900 tokens.
Practical workflow for prompt optimization
Start by asking the assistant to analyze the existing prompt, not rewrite it immediately. The skill’s operating rule is to capture a baseline first. You can run:
python scripts/prompt_optimizer.py prompt.txt --analyze --output baseline.json
Then ask for a revised prompt based on specific failures and compare the new version against the baseline. If you need token or cost awareness, pass your model label and current provider pricing yourself:
python scripts/prompt_optimizer.py prompt.txt --tokens --model claude --price-per-mtok 3.00
Use native structured output features when your platform supports them. The skill’s guidance treats “respond only with JSON” as a fallback, not the preferred contract.
Using it for RAG and agent workflows
For RAG evaluation, prepare JSON files for questions and retrieved contexts, then run:
python scripts/rag_evaluator.py --contexts contexts.json --questions questions.json --output report.json --verbose
Use this when you need relevance, coverage, Precision@K, Recall@K, MRR, NDCG, faithfulness, or groundedness signals. For agents, provide a YAML or JSON configuration describing tools, pattern, max iterations, system prompt, and expected flow. Then validate or visualize:
python scripts/agent_orchestrator.py agent.yaml --validate
python scripts/agent_orchestrator.py agent.yaml --visualize --format mermaid
This helps catch missing tools, unsafe loops, unclear orchestration, and rough token usage before deployment.
senior-prompt-engineer skill FAQ
Is senior-prompt-engineer only for Claude?
No. The skill is written for Claude skill usage, but the prompt engineering methods and scripts are model-agnostic. The tools accept arbitrary model names for informational estimation and avoid hardcoded model IDs or stale pricing tables.
How is it different from asking for a better prompt?
A generic prompt rewrite usually optimizes for readability. The senior-prompt-engineer skill optimizes against evidence: baseline behavior, eval cases, output contracts, retrieval metrics, and workflow constraints. That makes it better for production prompts where consistency matters.
Can beginners use this skill?
Yes, but beginners should start with references/prompt_engineering_patterns.md before running evaluations. The skill assumes you can describe the task, provide examples, and judge whether outputs meet requirements. If you have no examples, first ask the assistant to help design a small eval set.
When should I not use it?
Do not use this skill for casual one-off writing prompts, creative brainstorming where evaluation is subjective, or cases where you cannot provide examples, constraints, or success criteria. It is also not a substitute for provider-native structured output, moderation, tracing, or production observability.
How to Improve senior-prompt-engineer skill
Improve senior-prompt-engineer results with better baselines
The fastest way to improve senior-prompt-engineer results is to supply a real baseline: current prompt, current outputs, failure labels, and desired outputs. Ask the assistant to preserve what already works and target only the failure classes. This prevents broad rewrites that look cleaner but regress important behavior.
Build stronger eval sets
Use 10–20 cases minimum, but make them representative rather than easy. Include normal cases, edge cases, ambiguous cases, adversarial inputs, and examples that previously failed. For extraction or classification, include expected labels. For generation, include scoring criteria such as accuracy, completeness, tone, citation use, or schema validity.
Avoid common failure modes
Common problems include optimizing before defining success, mixing multiple tasks into one prompt, relying on prompt text for guarantees that should be handled by APIs, and measuring only final answer quality while ignoring retrieval or tool selection. If outputs are unstable, separate instruction design, examples, schema, retrieval quality, and agent control flow instead of changing everything at once.
Iterate after the first output
After the first revision, do not immediately ship it. Run the same eval set, compare against baseline.json, inspect regressions, and ask for a targeted second pass. A good follow-up is:
Compare the revised prompt against the baseline. Keep improvements that reduce refund misclassification, but fix the two regressions where technical tickets lost required escalation reasons. Do not increase the prompt by more than 100 tokens.