Advanced Detectors¶

mcp-scan includes four advanced detection systems that provide deep security analysis beyond pattern matching:

LLM Multi-Provider - Semantic analysis using AI models
CodeQL - Deep semantic analysis with custom queries
LSP - Type-aware analysis via Language Server Protocol
ML - Machine learning-based classification

LLM Multi-Provider Detection¶

The LLM detector uses large language models to semantically analyze text for prompt injection attempts. It supports multiple providers:

Provider	Type	API Key	Default Model
Ollama	Local	Not required	llama3.2:3b
Claude	Cloud (Anthropic)	`ANTHROPIC_API_KEY`	claude-3-haiku-20240307
OpenAI	Cloud	`OPENAI_API_KEY`	gpt-4o-mini

Configuration¶

Using Ollama (Local)¶

# Start Ollama
ollama serve

# Pull a model
ollama pull llama3.2:3b

# Run scan with LLM detection
mcp-scan scan ./my-mcp-server --llm --llm-provider ollama

Using Claude (Anthropic)¶

# Set API key
export ANTHROPIC_API_KEY=sk-ant-...

# Run scan
mcp-scan scan ./my-mcp-server --llm --llm-provider claude

Using OpenAI¶

# Set API key
export OPENAI_API_KEY=sk-...

# Run scan
mcp-scan scan ./my-mcp-server --llm --llm-provider openai

Auto-detection¶

If no provider is specified, mcp-scan will auto-detect available providers in this order: 1. Ollama (if running locally) 2. Claude (if ANTHROPIC_API_KEY is set) 3. OpenAI (if OPENAI_API_KEY is set)

# Auto-detect provider
mcp-scan scan ./my-mcp-server --llm

CLI Flags¶

Flag	Description	Default
`--llm`	Enable LLM detection	false
`--no-llm`	Disable LLM detection	-
`--llm-provider`	Provider: ollama, claude, openai	auto-detect
`--llm-url`	Ollama URL	http://localhost:11434
`--llm-model`	Model name	Provider-specific
`--llm-threshold`	Confidence threshold (0-1)	0.7
`--llm-timeout`	Request timeout	60s

YAML Configuration¶

llm:
  enabled: true
  provider: "ollama"      # ollama, claude, openai
  url: "http://localhost:11434"
  model: "llama3.2:3b"
  threshold: 0.7
  timeout: "60s"
  max_length: 5000

CodeQL Detection¶

CodeQL provides deep semantic analysis using GitHub's query language. It can detect complex vulnerabilities like taint flows across function boundaries.

Prerequisites¶

Install CodeQL CLI:

# macOS
brew install codeql

# Linux
CODEQL_VERSION=2.16.0
curl -L -o codeql.tar.gz \
  "https://github.com/github/codeql-cli-binaries/releases/download/v${CODEQL_VERSION}/codeql-linux64.tar.gz"
tar xzf codeql.tar.gz
sudo mv codeql /opt/codeql
export PATH="/opt/codeql:$PATH"

# Verify installation
codeql version

Usage¶

# Basic CodeQL scan
mcp-scan scan ./my-mcp-server --codeql

# With custom queries
mcp-scan scan ./my-mcp-server --codeql --codeql-queries-dir ./my-queries

# With minimum severity filter
mcp-scan scan ./my-mcp-server --codeql --codeql-min-severity 5.0

CLI Flags¶

Flag	Description	Default
`--codeql`	Enable CodeQL detection	false
`--no-codeql`	Disable CodeQL detection	-
`--codeql-path`	Path to codeql binary	auto-detect
`--codeql-timeout`	Analysis timeout	30m
`--codeql-queries-dir`	Custom queries directory	-
`--codeql-min-severity`	Minimum CVSS score	0.0

YAML Configuration¶

codeql:
  enabled: true
  path: ""  # Auto-detect in PATH
  timeout: "30m"
  queries_dir: "./codeql-queries/mcp"
  languages: ["python", "javascript"]
  min_severity: 5.0

MCP-Specific Queries¶

mcp-scan includes custom CodeQL queries for MCP vulnerabilities:

PromptInjection.ql - Detects prompt injection patterns in tool descriptions
UnsafeExec.ql - Detects command injection from tool inputs
PathTraversal.ql - Detects path traversal vulnerabilities

LSP Detection¶

The LSP detector uses Language Server Protocol to perform type-aware analysis.

Prerequisites¶

Install language servers:

# Python (Pyright)
npm install -g pyright

# TypeScript/JavaScript
npm install -g typescript typescript-language-server

# Go
go install golang.org/x/tools/gopls@latest

# Verify
./scripts/check-lsp.sh

Usage¶

mcp-scan scan ./my-mcp-server --lsp

CLI Flags¶

Flag	Description	Default
`--lsp`	Enable LSP detection	false
`--no-lsp`	Disable LSP detection	-

YAML Configuration¶

lsp:
  enabled: true
  languages: ["python", "typescript", "go"]

ML Detection¶

The ML detector uses machine learning to classify text as potential prompt injection. mcp-scan provides two ML approaches:

Model	Type	Accuracy	Speed	Use Case
Logistic Regression	Feature-based	~91%	Fast (~1ms)	CI/CD, real-time
DeBERTa	Transformer	~95%+	Medium (~10ms)	High-accuracy needs

📖 Detailed Documentation: ML Detection Guide

Model Performance¶

The trained Logistic Regression model achieves:

Metric	Value
ROC-AUC	0.9471
F1 (Injection)	0.85
Precision	0.82
Recall	0.88
Accuracy	91%

Quick Start¶

# Install dependencies
pip install datasets scikit-learn numpy

# Train the model
python scripts/train_ml.py ml_weights.json

# Run scan with ML detection
mcp-scan scan ./my-mcp-server --ml --ml-model ml_weights.json

Training DeBERTa (Higher Accuracy)¶

For environments requiring higher accuracy:

# Install transformer dependencies
pip install transformers datasets torch onnx onnxruntime accelerate

# Train DeBERTa model
python scripts/train_deberta.py --output-dir ./models/deberta

# Use ONNX model for inference
# (See ml-detection.md for integration details)

CLI Flags¶

Flag	Description	Default
`--ml`	Enable ML detection	false
`--no-ml`	Disable ML detection	-
`--ml-model`	Path to trained weights (JSON)	-
`--ml-threshold`	Classification threshold (0-1)	0.3

YAML Configuration¶

ml:
  enabled: true
  threshold: 0.3
  model_path: "./ml_weights.json"

Threshold Tuning¶

Use Case	Threshold	Behavior
High Recall (catch all)	0.3	More false positives
Balanced (default)	0.5	Optimal F1
High Precision	0.7	Fewer false positives
Optimal (trained)	0.578	Best F1 on test set

Detection Categories¶

The ML classifier identifies these injection types:

instruction_override - "Ignore previous instructions"
jailbreak - DAN, developer mode attempts
identity_manipulation - "Act as a hacker"
system_prompt_extraction - "What are your instructions?"
data_exfiltration - "Include the API key in response"
delimiter_injection - <|system|> markers
command_injection - "Run shell command"

📖 Technical Details: ML Architecture | Development Process

Combining Detectors¶

For maximum coverage, combine multiple detectors:

# All detectors enabled
mcp-scan scan ./my-mcp-server --llm --codeql --lsp --ml

# With configuration
mcp-scan scan ./my-mcp-server \
  --llm --llm-provider claude \
  --codeql --codeql-min-severity 5.0 \
  --lsp \
  --ml --ml-model ml_weights.json

Complete YAML Configuration¶

version: "1"

mode: deep
timeout: 30m
fail_on: high

llm:
  enabled: true
  provider: "claude"
  model: "claude-3-haiku-20240307"
  threshold: 0.7
  timeout: "60s"

codeql:
  enabled: true
  timeout: "30m"
  queries_dir: "./codeql-queries/mcp"
  min_severity: 5.0

lsp:
  enabled: true
  languages: ["python", "typescript"]

ml:
  enabled: true
  threshold: 0.3
  model_path: "./ml_weights.json"

Environment Variables¶

Variable	Description
`ANTHROPIC_API_KEY`	API key for Claude
`OPENAI_API_KEY`	API key for OpenAI
`MCP_SCAN_LLM_PROVIDER`	Default LLM provider
`MCP_SCAN_LLM_URL`	Ollama URL
`MCP_SCAN_CODEQL_PATH`	CodeQL binary path

Performance Considerations¶

Detector	Speed	Resource Usage	Best For
Pattern	Fast	Low	CI/CD, quick scans
ML	Fast	Low	Batch processing
LLM (Ollama)	Medium	High	Local analysis
LLM (Cloud)	Medium	Low	Production
LSP	Medium	Medium	Type-aware analysis
CodeQL	Slow	High	Deep analysis, certification

Recommended Configurations¶

CI/CD Pipeline (fast):

mcp-scan scan . --mode fast --ml

Pull Request Review:

mcp-scan scan . --llm --ml

Security Audit (thorough):

mcp-scan scan . --mode deep --llm --codeql --lsp --ml

Certification:

mcp-scan scan . --mode deep --llm --llm-provider claude --codeql --lsp --ml --output sarif