// Personal project
AI skills, maintained like software
A personal Claude plugin: 14 researched methods as skills, tested and maintained by subagents and a scheduled agent run. It's built like software, with CI on every push, a cost budget, versioned releases and an export to five other AI agent clients.
▌ 1.0 The problem
AI assistants answer confidently, even when they're wrong, and good method gets lost between chats and projects. That means answers someone has to check again, and results that depend on who asked and how. I wanted the methods I rely on to be explicit, sourced and applied the same way every time, without the cost of loading them growing out of hand.
▌ 2.0 Constraints
- 2.1Every skill's description loads into every conversation, so each one costs tokens all the time.
- 2.2The agents read web pages and GitHub issues, and outside text can carry instructions meant to hijack them.
- 2.3No private names may reach a skill.
- 2.4Dated facts and links go stale, so they need rechecking at the source.
- 2.5It has to work in other AI clients too, not only in Claude.
▌ 3.0 My part
My own project. I research and write the methods, and designed how the plugin is built, tested and maintained. I built the test, measurement and release tooling in Python and GitHub Actions, set up the agent workflows around it, and approve every release.
▌ 4.0 Agentic workflows
A routing eval, run by a subagent
In every maintenance run, a read-only subagent on a smaller model sees only each skill's name and description, and picks a skill for each of 54 example requests. A wrong pick points to the description that misled it.
Three reviewers, one brief
Before the latest release, three read-only subagents each reviewed a part of the plugin against the same brief, looking for anything tied to one project or person. Their findings became one plan, and I decided what changed.
A scheduled maintenance agent
Every quarter, an agent run measures the plugin, rechecks dated facts and links at their sources, works through lessons logged as GitHub issues and opens a draft pull request with a report. Nothing merges until I approve it.
From chat to issue to release
A correction in any chat becomes a GitHub issue, one per lesson, so chats running at the same time never overwrite each other. The maintenance run turns them into changes, and only it closes them.
The right model for each step
Helpers that check, review or fetch run on a smaller, cheaper model. The model-choice skill, built on research into model routing and cascades such as FrugalGPT and RouteLLM, applies the same rule to any task: cheap checks first, and a larger model only when a check fails.
Guarded against prompt injection
Text from web pages and issues counts as data, never as instructions, and the helpers that read it can't change anything. The security skill also reviews AI agent setups against OWASP's Top 10 lists for LLM and agentic applications.
▌ 5.0 14 skills
Source evaluation
A 1–5 trust score per claim, built on the Swedish Psychological Defence Agency's “Kolla källan” questions. It scores what a rule says separately from what people actually do.
Decision analysis
Multi-criteria scoring with swing weights. People weigh independently, so the first speaker doesn't anchor everyone. A robustness check runs across hundreds of nearby weightings, and the process ends with a pre-mortem.
Estimates
Time, cost and usage as a range with a P50 and a P80: how similar cases went first, then a simulation of the parts, then a check against what actually happened.
UX and UI
16 principles drawn from Nielsen Norman Group, GOV.UK and Apple's and Google's design guidelines, with a screenshot review checklist.
Accessibility
WCAG 2.2 AA in practice, tested with axe-core plus a keyboard and screen-reader pass.
Plain writing
The plain-language standard ISO 24495-1, applied with Swedish klarspråk and plain English: the most important thing first, one idea per sentence, common words.
Data modelling
Cardinality, keeping data consistent where there are no transactions, concurrency, versioned migrations, and Postgres/Supabase row-level security.
Security review
A threat model with STRIDE, then checks against the OWASP Top 10 and ASVS for web apps, repositories, CI and AI agents. A review aid, not a penetration test.
Plugin engineering
Lessons from building and running plugins, especially ones a team shares: what works, what breaks, and the checks that prevent it.
Model choice
Which model and effort level for each step: cheap checks first, and a larger model only when a check fails. Written for any AI client and for self-hosted models.
Maintenance
Upkeep of this plugin: measuring its token use, testing which skill each request picks, rechecking facts and links, working through logged lessons, and building new versions.
Lesson intake
Corrections and findings from any chat become GitHub issues: made general, checked for duplicates, and filed only after the user says yes.
Proactive help
After the answer, at most three suggestions tied to the user's own plans: collisions, closing windows, single points of failure and missing steps. Nothing is done without a yes.
What I need from you
Every reply that needs an answer or an action from the user ends with one numbered list of questions and actions, with labels that carry over until each one is answered or done.
Most of these methods started in my planning platform, a side project with a trip planner, and an event planner to follow. There they keep the data model, the accessibility tests and the fact-checking consistent.
▌ 6.0 Decisions
Treat skills as software
The source lives in GitHub, and every push runs CI in GitHub Actions: the self-test, the export and the cost measurement. The self-test checks structure, versions, references and valid frontmatter, and blocks private names, which it knows only as salted hashes.
Evidence first
Ten of the skills rest on cited research, standards and official guidance, with every source scored 1–5. Before a fact is stored, shared or relied on, Claude checks it at the original source, looks harder at anything implausible and marks what it couldn't confirm.
Checked from outside
Research shows models are poor at catching their own mistakes without outside feedback. So the checks run against sources, tests and separate reviewers rather than the model's own opinion, and I approve anything that matters. That lowers the risk of made-up facts; it doesn't remove it.
A budget for cost
A script estimates tokens in three tiers: descriptions, loaded in every chat; instructions, loaded when a skill is used; and reference files, loaded only when read. The self-test fails if the descriptions together pass about 1,500 tokens.
Portable, not locked in
An export packages the skills in the open Agent Skills format, with an install guide for Codex, Cursor, Gemini CLI, GitHub Copilot and OpenCode. Claude-only features are marked, each with a fallback for other clients.
Release from clean source
Every build gets a version bump and a changelog entry. A release script checks the packaged files against the source and keeps every version with checksums.
▌ 7.0 Why it matters
Lower running cost
What loads in every conversation is measured and capped, and the checking runs on cheaper models.
Answers you can check
Facts trace back to scored sources, and anything that couldn't be confirmed is marked as such.
Change without surprises
Every change passes the tests and my review before it merges, and ships as a versioned release with a changelog.
Risk kept in check
A person approves anything that matters, private names can't get in, and outside text is treated as data, which makes the agents harder to hijack.
No lock-in
The same skills install in six AI agent clients, Claude among them.
▌ 8.0 Result
Fourteen skills, in daily use across my own projects. The part loaded in every conversation comes to about 1,170 tokens, down from about 1,320 before the latest release, even with a skill added.
Want the details behind this? Email me at filip@heinel.dev.