Skip to content

Testing Documentation - MCP-SCAN

This directory contains the complete documentation of the mcp-scan testing system, the static security analyzer for MCP servers.

Document Index

Document Description
Test Structure Organization of unit tests, integration, benchmarks
Fixtures Test fixtures by vulnerability class (A-N)
DVMCP Damn Vulnerable MCP Server - vulnerable test server
Run Tests Commands to run tests and coverage
Current Status Current coverage and areas for improvement

Overview

The mcp-scan testing system is designed to ensure:

  1. Accurate detection: Validate that each vulnerability class (A-N) is detected correctly
  2. No false positives: Verify that benign code does not generate alerts
  3. Performance: Benchmarks to ensure acceptable analysis times
  4. Regression: Golden tests to detect changes in behavior

Types of Tests

Unit Tests

Located alongside the source code (*_test.go), they validate individual components: - AST Parser (Python, TypeScript, JavaScript, Go) - Taint analysis engine - Pattern detectors - ML classifiers - Call graph

Integration Tests

Validate the complete scanner flow: - File discovery - MCP surface analysis - Vulnerability detection - Report generation

Golden Tests

Compare outputs against expected snapshots: - JSON format - SARIF format - Evidence

Benchmarks

Measure performance of critical components: - AST parsing - ML feature extraction - Prompt classification

Directory Structure

mcp-scan/
├── internal/
│   ├── parser/parser_test.go       # Parser tests
│   ├── pattern/engine_test.go      # Detector tests (1300+ lines)
│   ├── taint/engine_test.go        # Taint engine tests
│   ├── callgraph/graph_test.go     # Call graph tests
│   ├── ml/classifier_test.go       # ML classifier tests
│   └── ...
├── pkg/
│   └── scanner/
│       ├── scanner_test.go         # Scanner unit tests
│       ├── scanner_integration_test.go  # Integration tests
│       └── scanner_bench_test.go   # Benchmarks
└── testdata/
    ├── fixtures/                   # Vulnerable/benign code
    │   ├── class_a/               # RCE
    │   ├── class_g/               # Tool poisoning
    │   └── ...
    ├── golden/                     # Expected outputs
    └── damn-vulnerable-MCP-server/ # Complete DVMCP

Quick Commands

# Run all tests
make test

# Tests with coverage
make test-coverage

# Golden tests (snapshots)
make test-golden

# Tests against DVMCP
make test-detection

# Benchmarks
go test -bench=. ./...

Contributing Tests

When adding new features or detectors:

  1. Create fixtures in testdata/fixtures/class_X/
  2. Add unit tests in the corresponding package
  3. Update integration tests if it affects the complete flow
  4. Update golden tests if the output format changes
  5. Document in current-status.md

References