I was in the middle of a client sprint when the CI pipeline started failing with a "Potential secret detected" error. The scanner was a simple regex-based tool we had added a year ago to catch AWS keys, Slack tokens, and similar strings. It caught a few real leaks, but it also started flagging every line that contained a long base64 string, even harmless image data embedded in a CSS file. After an hour of digging, I realized the regex was the problem, not the code.

The temptation of regex for secret scanning

When you need a quick guard against accidental credential commits, regex looks attractive. It’s easy to drop a few patterns into a pre-commit hook or a CI job, and you get immediate feedback. A typical rule set might look like this:

(?:AKIA[0-9A-Z]{16})|(?:sk_live_[0-9a-zA-Z]{24})|(?:ghp_[0-9a-zA-Z]{36})
  • AKIA[0-9A-Z]{16}, catches AWS access keys.
  • sk_live_[0-9a-zA-Z]{24}, catches Stripe live secret keys.
  • ghp_[0-9a-zA-Z]{36}, catches GitHub personal access tokens.

It seems solid, but the moment you add a new service or a custom token format you have to extend the pattern, and each addition increases the chance of false alarms.

Why regex generates false positives

1. Context blind

Regex looks at a string in isolation. It cannot tell whether the matched text is inside a comment, a test fixture, or a deliberately stored encryption key. A base64-encoded image that happens to start with AKIA will trigger the same rule.

2. Entropy is a poor proxy

Many regex-based scanners use a simple "high-entropy" check, any string longer than 20 characters with a mix of letters, numbers, and symbols is suspect. That rule flags JSON blobs, JWTs, and even hashed passwords that are meant to be stored.

3. Maintenance nightmare

Every time a service changes its token format you must edit the regex. A typo can break the whole rule set, and you end up with a brittle solution that nobody trusts.

A more reliable workflow: combine pattern matching with entropy and context

I switched to a three-step approach that still uses regex for the obvious patterns but adds two safety nets.

  1. Initial pattern match, run the same simple regexes you already have.
  2. Entropy filter with length guard, calculate Shannon entropy for the matched string; only flag if entropy > 4.0 and length > 20.
  3. Context check, look at the surrounding file type and line. If the match appears in a .png binary, a .svg data URI, or a test fixture directory, suppress the warning.

Here’s a minimal Node.js script that demonstrates the idea:

const fs = require('fs');
const crypto = require('crypto');

const patterns = [
  /AKIA[0-9A-Z]{16}/g,
  /sk_live_[0-9a-zA-Z]{24}/g,
  /ghp_[0-9a-zA-Z]{36}/g,
];

function shannonEntropy(str) {
  const freq = {};
  for (const ch of str) freq[ch] = (freq[ch] || 0) + 1;
  let ent = 0;
  const len = str.length;
  for (const count of Object.values(freq)) {
    const p = count / len;
    ent -= p * Math.log2(p);
  }
  return ent;
}

function scanFile(path) {
  const content = fs.readFileSync(path, 'utf8');
  const ext = path.split('.').pop();
  const ignoreExt = ['png', 'jpg', 'svg'];
  if (ignoreExt.includes(ext)) return [];

  const findings = [];
  for (const re of patterns) {
    let match;
    while ((match = re.exec(content)) !== null) {
      const token = match[0];
      if (token.length > 20 && shannonEntropy(token) > 4.0) {
        findings.push({path, line: content.substr(0, match.index).split('\n').length, token});
      }
    }
  }
  return findings;
}

module.exports = {scanFile};
  • patterns, the same regexes you already trust.
  • shannonEntropy, calculates the randomness of the match.
  • ignoreExt, skips binary or image files where false positives are common.
  • The script reports line numbers only when the token passes both the length and entropy thresholds.

Integrating into CI without breaking the build

I added the script as a custom step in GitHub Actions. The key is to fail the job only when the scanner finds unignored findings. Here’s a snippet of the workflow file:

name: Secret Scan
on: [push, pull_request]
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Install Node
        uses: actions/setup-node@v3
        with:
          node-version: '20'
      - run: npm ci
      - name: Run secret scanner
        run: node scripts/scan.js || echo "No secrets found"
        continue-on-error: false
  • The continue-on-error: false ensures the job stops if the script exits with a non-zero code.
  • By keeping the scanner in the repo, you avoid external service latency and retain full control over the patterns.

When regex is still the right tool

There are cases where a pure regex solution is sufficient:

  • Small teams with a single language and a narrow set of services.
  • Pre-commit hooks where speed matters more than completeness.
  • One-off audits on a legacy repository that won’t be maintained.

In those scenarios, pair the regex with a manual review step to catch the inevitable false alarms.

Common pitfalls and how to avoid them

  • Skipping entropy, without it, any long string will be flagged, overwhelming developers.
  • Hard-coding ignore paths, maintain a .secretignore file similar to .gitignore so the list can evolve.
  • Running the scanner on generated assets, always exclude dist/, build/, and any folder containing compiled code.

Tools worth checking out

If you prefer not to roll your own, a few open-source projects already implement the pattern-plus-entropy model:

  • GitLeaks, supports custom regexes and entropy thresholds.
  • TruffleHog, searches git history and can be tuned with regex and entropy.
  • detect-secrets, offers a plug-in architecture for language-specific checks.

All of them let you add a .gitignore-style file for false-positive suppression.

FAQ

How do I decide which entropy threshold to use?

A threshold of 4.0 works well for most base64-encoded tokens. If you notice many false positives, raise it to 4.5; if you miss real secrets, lower it to 3.5. Test the setting on a known-good commit to find the sweet spot.

Can this approach catch secrets that are split across lines?

Not directly. Regex works line-by-line, so multi-line tokens need a pre-processing step that joins lines or a specialized scanner like TruffleHog that walks the AST.

What about secrets stored in environment files?

Treat .env files as source files; the same pattern-plus-entropy check applies. However, you should also add a git hook that prevents committing any file matching *.env*.

Wrapping up

If you’re still relying on a blunt regex rule set, expect noise and wasted time. Adding an entropy filter and a simple context check removes most of the chatter while keeping the workflow fast enough for CI. Start with the tiny script I shared, tune the thresholds, and you’ll have a reliable API secret scanner that actually helps developers instead of nagging them.