MPiFY Skill Tester
How it works

What happens when you check a skill

A skill is a folder of instructions, and sometimes scripts, that an AI agent such as Claude loads and follows. Installing one is like adding a colleague who does whatever the folder says, often without asking you first. So it's worth reading before it goes in.

The steps

  • We fetch it. For a GitHub link we look up the exact commit, then download only the skill folders being checked, so a skill inside a huge repository is still quick. Uploads are unpacked into a throwaway folder. Links that point outside a skill are left out and reported.
  • We never run it. Every file is read as text. No script, install command or package from the skill is executed.
  • We run our checks. They look for the problems that turned out to matter when we reviewed 1,582 real skills by hand: instructions to stop asking you, packages that don't exist, code fetched and run from the internet, hidden text, credential reads, persistence, and tools a skill pre-approves. We also ask npm and PyPI whether each package a skill installs actually exists.
  • We run SkillSpector. NVIDIA's open-source scanner matches hundreds of patterns, including YARA rules for known malware. We run its static checks only.
  • You get a report. Each skill gets one of three results. Red flags: read these lines before you install. Worth knowing: normal for many skills, such as sending your data to a service it's built for. No red flags found: nothing matched, which isn't the same as a guarantee.

Why SkillSpector's label isn't the headline

On its own, the scanner flags anything that looks like an attack, including security guides that explain attacks and docs that mention a .env file. When we read every skill it marked "do not install" on one machine, almost all were fine, and one of the real problems had only been marked "caution". So its findings are in each report, with the ones that were nearly always harmless labelled, but they don't decide the result.

Privacy

  • Reports have unlisted links. Anyone you share a link with can read it; nobody else can find it.
  • Uploaded files are deleted as soon as the check finishes. Reports are kept for 90 days.
  • We don't store IP addresses, only a salted hash used for rate limits. No analytics, no trackers, no third-party fonts.
  • Package names are sent to npm and PyPI to check they exist, and SkillSpector sends dependency names to OSV.dev to look up known vulnerabilities. Nothing else leaves the server.

Limits

  • Public GitHub repositories and uploads only. Up to 25 skills per check; for bigger collections, pick one.
  • A script that misbehaves only when it runs, or fetches its real payload later, can get past any static check.
  • Skills that install packages when they run get whatever those packages are on the day.

Credits

Built by MPiFY. Scanning uses SkillSpector by NVIDIA, under the Apache 2.0 licence, version v2.12.0. Skill Tester isn't affiliated with or endorsed by NVIDIA or Anthropic. Found something wrong? Email hello@mpify.com.