Exposed Secrets and Broken Clouds: The Hidden Attack Surface of 2025

It wasn't a sophisticated zero-day exploit. It wasn't a nation-state attack using cutting-edge malware. In December 2024, Chinese state-sponsored hackers breached the U.S. Treasury Department through something far simpler: a single compromised API key from BeyondTrust, their authentication vendor. With that one exposed credential, attackers bypassed millions of dollars in security investments, remotely accessed Treasury workstations, and exfiltrated unclassified documents from one of America's most sensitive government agencies.

This wasn't an anomaly. It was a symptom of a crisis that GitGuardian's 2025 State of Secrets Sprawl report calls "one of the most significant yet underestimated threats in cybersecurity".

                               The alarming statistics of secrets sprawl in 2024-2025

Table of Contents

1. The Numbers That Should Keep Every Security Leader Awake
2. The AI Coding Assistant Paradox
3. Beyond GitHub: Secrets Are Everywhere
4. Real Breaches, Real Consequences
5. The Enterprise Defense Playbook
6. The Developer Experience Imperative
7. The Path Forward
8. From Theory to Practice: Building Real-World Skills

The Numbers That Should Keep Every Security Leader Awake

The scale of secrets exposure has reached staggering proportions. GitGuardian's analysis of 1.4 billion commits across 69.6 million public repositories in 2024 uncovered a landscape of persistent vulnerability that's only getting worse:

                        Secrets leaked on public GitHub repositories by year (in millions)

23.8 million secrets were leaked on public GitHub in 2024 alone—a 25% jump from the previous year. But here's what makes this truly terrifying: 70% of secrets leaked in 2022 remain active today, creating an ever-expanding attack surface that compounds over time. These aren't theoretical risks; they're live credentials that attackers can exploit right now.

The problem isn't confined to public repositories. GitGuardian's analysis of private repositories revealed that 35% contain exposed secrets. Worse, sensitive credentials appear five times more frequently in private repositories than public ones—shattering the dangerous assumption that "private equals secure". But the most alarming trend? The tools designed to accelerate development are becoming secret-leakage accelerators.

The AI Coding Assistant Paradox

Here's an uncomfortable truth that many organizations aren't discussing: the tools designed to boost developer productivity may be amplifying security risks. Repositories using GitHub Copilot had a 6.4% secret leakage rate—40% higher than the 4.61% average across all public repositories.

GitHub Copilot usage surged 27% between 2023 and 2024. But researchers at the Chinese University of Hong Kong discovered something troubling: they successfully extracted over 2,700 hardcoded credentials from Copilot, with at least 200 (7.4%) being real, valid secrets that could be traced back to GitHub repositories.

The mechanism is straightforward but alarming. AI coding assistants learn from public code repositories—including code that contains leaked secrets. When developers use prompts that resemble patterns associated with credential storage, these models can inadvertently suggest real credentials from their training data. Security researchers demonstrated this by crafting prompts designed to "force" the model to emit memorized secrets, and it worked.

This doesn't mean abandoning AI coding tools. But it does mean organizations need stronger secret detection controls precisely because AI assistance is becoming ubiquitous.

Beyond GitHub: Secrets Are Everywhere

                    Distribution of secrets exposure across the software development lifecycle

The problem extends far beyond source code repositories. Secrets now sprawl across the entire software development lifecycle, often hiding in places where security teams have limited visibility:

Collaboration Tools Under Siege

38% of incidents in collaboration tools (Slack, Jira, Confluence) are classified as highly critical or urgent—higher than the 31% in source control management systems. Specifically:

• 2.4% of corporate Slack channels contained leaked secrets
• 6.1% of Jira tickets exposed credentials, making it the most vulnerable collaboration too
• These platforms lack built-in security safeguards, and employees using them are often less security-conscious than developers

The Docker Hub Time Bomb

GitGuardian's largest-scale analysis of 15 million public Docker images uncovered 100,000 valid secrets, including AWS keys and GitHub tokens from Fortune 500 companies. Alarmingly, 98% of detected secrets were found in image layers, with ENV instructions alone accounting for 65% of all secret leaks. And because Docker Hub lacks a partner notification system for secret exposure, these credentials often remain valid indefinitely.

The Private Repository Illusion

Perhaps most concerning is the false sense of security around private repositories. 

AWS IAM keys appear in 8% of private repositories—5x more frequent than public repositories

• Hardcoded passwords appear 3x more often in private repositories than public ones^[6]
• MongoDB credentials are the most commonly leaked secret in public repositories at 18.8%

Real Breaches, Real Consequences

The Treasury breach wasn't isolated. 2024 saw a cascade of high-profile incidents where leaked credentials served as the initial attack vector:

The Sisense Supply Chain Attack (April 2024)

Attackers gained unauthorized access to Sisense's self-managed GitLab repository, where they found a hardcoded token that granted access to Sisense's Amazon S3 buckets. From there, they exfiltrated terabytes of customer data, including millions of access tokens, email passwords, and SSL certificates. The breach prompted CISA to issue an urgent advisory to all Sisense customers—a reminder that supply chain attacks through exposed credentials can have exponential impact.

The New York Times GitHub Breach (January 2024)

An exposed GitHub token gave attackers access to 5,000+ repositories containing 273GB of source code, including the wildly popular Wordle game. GitGuardian's analysis of the leaked data uncovered over 4,000 unique secrets and at least 200 critical secrets lurking in the code. The breach demonstrated how a single exposed credential can unlock an organization's entire intellectual property.

The CircleCI Catastrophe (January 2023)

Malware on a CircleCI engineer's laptop compromised a valid, MFA-backed SSO session. Because the engineer had privileges to generate production access tokens, attackers exfiltrated customer environment variables, tokens, and keys from the popular CI/CD platform. CircleCI advised all customers to immediately rotate "any and all secrets"—affecting over a million software engineers worldwide.

The Uber Breach (September 2022)

After social engineering an employee to accept an MFA push notification, attackers found a PowerShell script containing hardcoded credentials for Uber's Privileged Access Management (PAM) solution—Thycotic. With admin access to the PAM system, they compromised AWS, GCP, Google Drive, Slack, SentinelOne, HackerOne, and internal dashboards. The entire breach pivoted on a single hardcoded credential in a script.

The Enterprise Defense Playbook

                      Secrets management architecture across the software development lifecycle

‍

Understanding the threat is step one. Here's how leading organizations are systematically eliminating hardcoded secrets from their distributed systems.

Strategy 1: Migrate to Managed Secret Stores

The Challenge

Most enterprises still have credentials scattered across configuration files, environment variables, CI/CD pipelines, and code repositories. The first step toward centralized control is comprehensive migration to managed secret stores.

AWS Secrets Manager: The Cloud-Native Choice

AWS recently announced managed external secrets—a feature offering default-enabled automatic rotation for third-party SaaS secrets without requiring Lambda function creation. 

                                           Handling Secrets with AWS Secrets Manager

HashiCorp Vault: Multi-Cloud Flexibility

Vault's database secrets engine generates credentials dynamically based on configured roles, eliminating the need for applications to hardcode credentials. Services "ask" Vault for database credentials, and administrators specify time-to-live (TTL) so credentials are automatically revoked when no longer used.

Dynamic secrets are suited for time-bound workloads like batch jobs or CI/CD executions, while auto-rotated secrets work better for longer-lasting workloads needing stable connections.

Migration Best Practice: AWS recommends a phased approach—integrate with Secrets Manager before cloud migration to simplify the transition, then enable automatic rotation post-migration to immediately address legacy exposure risks.

Strategy 2: Implement Zero-Downtime Automated Rotation

              Automated secret rotation workflow for zero-downtime credential management

The 70% Problem

The most damning statistic isn't the number of leaked secrets—it's that 70% of secrets leaked in 2022 remain active today. Remediation is slow due to enforcement gaps and complex workflows.

AWS Secrets Manager Rotation Architecture

AWS provides multiple rotation approaches:

Managed rotation: For supported secrets, AWS configures and manages rotation without Lambda functions

1. Managed external secrets rotation: Updates secrets held by AWS partners automatically
2. Lambda-based rotation: For custom secrets, uses a four-stage rotation function:
   
   • createSecret: Creates new secret version with staging label AWSPENDING
   • setSecret: Updates credential in the database/service
   • testSecret: Validates new credential works
   • finishSecret: Moves AWSCURRENT label to new version

‍

HashiCorp Vault Dynamic Secrets

Dynamic credentials offer an even more elegant solution. Instead of rotating static credentials, Vault generates unique, short-lived credentials for each request:

• Application requests credentials from Vault
• Vault creates temporary database user with specified TTL
• Credentials automatically expire after TTL
• Each application instance gets unique credentials—no sharing, easier auditing
• For longer-lived workloads, Vault supports static credential rotation where it manages the root credential and rotates it automatically on schedule.

Strategy 3: Eliminate Secrets from CI/CD Pipelines

The Anti-Pattern Gallery

• API keys embedded in workflow YAML files
• Database passwords in Jenkins credentials stored as plain text
• Cloud provider access keys in .gitlab-ci.yml configuration
• Docker images with secrets baked into layers

GitHub Actions Best Practices

Use environment-specific secrets with mandatory review requirements rather than repository-level secrets

• Implement OIDC (OpenID Connect) authentication instead of long-lived tokens for cloud provider access
• Never use structured data (JSON, XML, YAML) as secret values—GitHub only masks explicitly recognized secrets
• Rotate secrets every 30-90 days using least-privileged access principles

GitOps-Native Solutions

For Kubernetes environments, three approaches have emerged as industry standards:

1. Sealed Secrets: Encrypt secrets client-side, commit encrypted manifests to Git, controller decrypts in-cluster
2. External Secrets Operator (ESO) + ArgoCD: Store ExternalSecret manifests in Git that reference external providers; ESO controller syncs actual Kubernetes secrets
3. SOPS: Encrypt secrets with cloud KMS before committing to Git, decrypt at deployment time

            External Secrets Operator unifying multi-cloud secrets management in Kubernetes

Strategy 4: Deploy Multi-Layer Detection

GitHub Push Protection: Necessary but Insufficient

GitHub now enables push protection by default for all public repositories, automatically blocking commits when it detects secrets matching 200+ token types from 180+ service providers. The company discovered one million exposed secrets in public repositories in just the first two months of 2024.

However, there's a critical limitation: GitHub Push Protection only detects secrets with standardized patterns. It struggles with generic secrets—hardcoded passwords, database connection strings, custom authentication tokens—which accounted for 58% of all leaked credentials in 2024.
‍

GitHub Secret Protection · GitHub

Building a Comprehensive Detection Stack

Pre-commit hooks: Implement tools like GitGuardian's ggshield, TruffleHog, or Gitleaks to scan before code leaves developer machines

1. CI/CD scanning: Make secret scanning a mandatory, blocking step for all code merges
2. Historical scanning: Scan entire Git history to surface legacy credentials
3. Collaboration tool monitoring: Extend scanning to Slack, Jira, Confluence where many critical secrets leak
4. Container registry scanning: Scan Docker images before pushing to registries^]

The OWASP Recommendation

The OWASP Secrets Management Cheat Sheet emphasizes a lifecycle approach:

• Secrets should exist only as long as necessary (rotate often)
• Have a method for automatic rotation
• Be visible only to those who need them (least privilege)
• Be revokable with logging of attempts to use revoked secrets
• Never be logged in plaintext

Strategy 5: Secure Kubernetes with External Secrets Operator

The Native Kubernetes Secrets Problem

Native Kubernetes secrets are base64-encoded, not encrypted—trivially reversible by anyone with cluster access. They're difficult to manage at scale, have no built-in rotation, and secrets in manifests committed to Git create exposure risks.

External Secrets Operator Architecture

ESO provides three core components:

• SecretStore: Defines connections to external secret management systems (AWS, Vault, Azure Key Vault, GCP)
• ExternalSecret: Describes what data to retrieve and how to create Kubernetes Secret objects
• ESO Controller: Watches ExternalSecret resources, queries external stores, creates/updates Kubernetes Secrets

Security Best Practices

• Implement proper RBAC controls restricting access to ExternalSecret and SecretStore resources
• Define NetworkPolicies limiting outbound traffic to only kube-apiserver and secret providers
• Use a policy engine like Kyverno to explicitly deny unused providers and restrict access patterns
• Enable refreshPolicy to automatically sync secrets when changed in external stores
• Set deletionPolicy to automatically remove Kubernetes secrets when upstream credentials are deleted

The Developer Experience Imperative

Security tools that disrupt developer workflows face resistance. Modern solutions must balance security with productivity.

1Password Developer Tools exemplify this approach:

• SSH Agent: Generate, store, and autofill SSH keys for GitHub/GitLab with Touch ID authorization
• CLI Secret References: Store secrets in vaults, reference them in code as op://vault/item/field—never plaintext
• Secrets Automation: Automatically supply infrastructure secrets to CI/CD pipelines and terminal sessions

The key insight: centralized, encrypted storage across teams with automatic synchronization eliminates the "outdated .env file" problem while providing audit trails for compliance.

The Path Forward

The secret sprawl crisis won't solve itself. Every passing day, the 23.8 million secrets leaked in 2024 age into the 70% that remain active years later. Every unrotated credential, every hardcoded password in a private repository, every API key baked into a Docker image represents a potential breach vector waiting to be exploited.

But organizations that implement defense-in-depth for secrets management—centralized stores, automated rotation, multi-layer detection, and developer-friendly tooling—are systematically closing these attack vectors.

The Treasury breach, the Sisense supply chain attack, the New York Times code leak—these weren't sophisticated attacks. They were simple exploitation of exposed credentials. And they're entirely preventable.

The question isn't whether your organization has leaked secrets. The question is whether you'll find them before attackers do.

From Theory to Practice: Building Real-World Skills

Understanding the threat is step one. But implementing these strategies requires hands-on experience with the tools and techniques that make secrets management truly effective. This is where many security teams stumble—they know what needs to be done, but lack the practical skills to execute.

AppSecEngineer helps teams bridge this gap with practical training in AWS Secrets Manager, HashiCorp Vault, Kubernetes secrets hardening, and GitOps security for CI/CD pipelines. Our hands-on approach lets engineers experience the consequences of misconfiguration and master the tools that prevent breaches—because when it comes to secrets management, theoretical knowledge without practical experience is as dangerous as having no knowledge at all.