We Scanned 102 Open-Source AI Agent Repos. Here Is What Their AWS Credentials Could Actually Do.

AI coding agents and MCP servers increasingly run with real AWS credentials. We wanted a concrete answer to one question: if one of those agents misbehaves, gets prompt-injected, or has its config leaked, what could its credentials actually do? Not "is the tool malicious" — assume it isn't — but what is the blast radius when something goes wrong.

So we pointed shieldly agent-scan at 102 open-source AWS agent and MCP projects and reviewed every result by hand. Here is the method, the real numbers, and an honest detour where we found a bug in our own scanner partway through.

How We Picked the Repos

We built the candidate list from GitHub code search for IAM policy artifacts referencing agent-relevant actions (bedrock:InvokeModel, bedrock:InvokeAgent, bedrock-agentcore:InvokeAgentRuntime) in .json and .yaml files, filtered to repos whose names indicate AI agent, MCP, or GenAI projects, plus a hand-curated list of popular agent and MCP repos. 102 repos total.

For each repo we shallow-cloned it, ran the scanner against every IAM policy, trust policy, CloudFormation template, and agent config we found, then manually reviewed every hit. We excluded six repos whose only "policies" were test fixtures, generated test outputs, knowledge-base document metadata, or IAM reference datasets — files nobody deploys — and one vendored copy of another sampled repo. We classified every remaining policy as a runtime policy (the agent's execution role), a deployment policy (a CI or deploy user), or a role trust policy, because those carry different threat models.

The Numbers

Of the 102 repos, 36 ship at least one genuine IAM policy artifact — 109 files in total (75 runtime policies, 15 deployment policies, 19 trust policies). The other roughly two-thirds ship no reviewable policy at all: the permissions they need live in code or README prose, where no scanner and no reviewer ever sees them. That absence is its own finding.

Across the 36 repos that do ship a policy:

• 22% (8 of 36) grade CRITICAL on at least one shipped policy.
• Looking only at runtime policies — the agent's actual execution credentials — 32 repos ship one, and 47% of those (15 of 32) grade CRITICAL or HIGH.
• 28% (10 of 36) grant a high-severity privilege-escalation path: iam:PassRole, iam:Attach*, sts:AssumeRole on "*", or iam:CreateAccessKey.
• 33% (12 of 36) grant destructive actions — deletes on S3, DynamoDB, CloudFormation, EC2, or Route53 — with broad scope.
• 22% (8 of 36) ship a service trust policy with no aws:SourceAccount or aws:SourceArn scoping condition — the classic confused-deputy setup.
• The good news: we found no hardcoded long-lived access keys in any sampled config, and zero trust policies graded CRITICAL once our scanner handled them correctly. (More on that next.)

A Bug We Found in Our Own Scanner

Our first pass reported 17 CRITICAL repos. Reviewing the hits by hand, about half traced back to a bug in our own engine. A role trust policy has a Principal and no Resource element, because its sts:AssumeRole grants assumption of the role, not by it. Our engine was treating that missing Resource as a wildcard and applying identity-policy rules, so a perfectly well-scoped trust policy — service principal plus aws:SourceAccount and aws:SourceArn conditions — graded CRITICAL.

We fixed the engine, added real trust-policy checks (an open Principal with no condition is CRITICAL; an unscoped service principal is a MEDIUM confused-deputy finding), and rescanned before writing this. The 9-CRITICAL figure above is from the fixed engine. If there is a meta-lesson here, it is the one we already believed: automated scanner output is evidence to review, not a verdict — ours included.

What the Worst Cases Look Like

These are all official samples or tutorials, and we are not calling anyone out — the point is that even careful, well-intentioned projects reach for broad grants, because that is the fastest path to "it works." Each is linked so you can verify:

• An AWS AgentCore workshop lab hands a code-interpreter agent iam:CreateAccessKey, iam:PassRole, and iam:Attach*, plus DynamoDB and EC2 delete and SSM parameter read — a full privilege-escalation path attached to a code-execution agent.
• A multi-agent code-generation-and-execution sample ships a root iam_policy.json combining iam:PassRole, policy attach, Lambda and S3 delete, and full OpenSearch Serverless admin.
• A widely-followed Bedrock RAG tutorial ships an iam-policy.json with s3:Delete* and iam:PassRole — the kind of policy beginners copy verbatim into production.

Why This Happens

Almost none of this is malicious or even careless. An agent needs some AWS access to be useful, and the fastest path to a working demo is a broad policy. The problem is that "it works" and "it is scoped to what the agent actually needs" are different bars, and today almost nothing checks the second one. The most striking result was not any single policy — it was that two-thirds of the repos we looked at ship no reviewable policy artifact at all.

Check Your Own Agents

The scan runs fully offline. Point it at a role, a local profile, a policy file, a CloudFormation template, or let it auto-discover the agent configs in your project:

# Auto-discover agent configs (.mcp.json, Claude Code, Cursor, ...)
npx @shieldly/cli agent-scan

# Or scan a specific role
npx @shieldly/cli agent-scan --role-arn arn:aws:iam::123456789012:role/my-agent-role

Or paste a policy into the free browser analyzer at shieldly.io/tools/agent-blast-radius — nothing you paste is uploaded or logged.

Amazon Web Services (AWS) is a trademark of Amazon.com, Inc. Shieldly is not affiliated with, endorsed by, or sponsored by Amazon Web Services.