How to: Search for hard-coded credentials

Intro

Time to drop my 5 cents on the topic of searching for hard-coded credentials.

You might already be familiar with big names, like:

• trufflehog
• git-secrets
• gitleaks

In my experience they did an overall good job on git repositories and marginally okay one on the plain codebase. Yet I started to notice a familiar pattern — the moment I used a more primitive method, like grep patterns, I could often find a couple more secrets in the code that were missed by the above tools for some reason.

You see, these tools try to remove any false positives from the results either by post-processing or by having overly strict secret patterns. This might reduce the effort required to go through the tool output, but, if you ask me, it’s way better to scour through a lot of false positives for a couple hours instead of missing a couple of false positives.

So let me tell you what I do instead!

gf

There is a not-so-hidden gem lying around on Github - a tool called gf:

https://github.com/tomnomnom/gf

Essentially, it’s a glorified grep wrapper with the ability to predefine patterns and flags. The patterns are stored in the .gf folder in your home directory.

$ go install github.com/tomnomnom/gf@latest

$ mkdir ~/.gf
$ gf -save password-pattern -HanriE 'password'
$ gf -list
password-pattern

$ echo 'password' > test_file

$ gf password-pattern test_file
test:1:password

What’s now? You can go and write your own patterns, of course!

.

.

.

.

.

Joke’s on you :P this post is in fact a huge tribute to the work of Dwi Siswanto (dwisiswant0) and Mazin Ahmed (mazen160) from Github.

They have 3 absolutely stunning repositories that will come in handy for us.

I actually abandoned my own patterns in favor of theirs.

Community work

First of all, dwisiswant0 has a curated repository with secret patterns written for gf manually:

https://github.com/dwisiswant0/gf-secrets

$ git clone clone https://github.com/dwisiswant0/gf-secrets
$ mv gf-secrets/.gf/* ~/.gf/
$ gf -list | wc -l
17

This will give us 17 very useful patterns to search for credentials.

But it does not end there! If you want more coverage, mazen160 has a collection of over 1500 secret patterns used by popular scanners (yes, including trufflehog, gitleaks and others)

https://github.com/mazen160/secrets-patterns-db

Using the work of dwisiswant0 we can then convert all of these patterns to gf!

https://github.com/dwisiswant0/secpat2gf

$ pip install secpat2gf
$ secpat2gf -r https://raw.githubusercontent.com/mazen160/secrets-patterns-db/refs/heads/master/db/rules-stable.yml --save --flags="-HanriE"

$ gf -list | wc -l
1622

This will give us more than 1600 secret patterns that are very likely to pick up some lesser known API tokens/formats. A word of caution though, scan time can quickly balloon if you use so many patterns.

How to use it

I like to define a shell function for scanning with gf. An example function in Fish (interpreter that I use) looks like this:

function gf-scan-secrets
    for pattern in (gf -list)
        echo "------------------------ $pattern ------------------------" | tee -a ../gf_secrets_results
        gf $pattern . | tee -a ../gf_secrets_results
    end
end

Running the scan is then as simple as:

$ gf-scan-secrets
...
------------------------ asymmetric-keys_secrets ------------------------
./acrtp/session4/lab5/id_pwn:1:-----BEGIN OPENSSH PRIVATE KEY-----
./acrtp/session3/lab3/id_rsa:1:-----BEGIN OPENSSH PRIVATE KEY-----
./acrtp/session2/lab4/ec2.pem:1:-----BEGIN OPENSSH PRIVATE KEY-----
./acrtp/.venv/lib/python3.13/site-packages/googleapiclient/discovery_cache/documents/appengine.v1.json:2021:-----BEGIN RSA PRIVATE KEY-----
./acrtp/.venv/lib/python3.13/site-packages/googleapiclient/discovery_cache/documents/appengine.v1alpha.json:964:-----BEGIN RSA PRIVATE KEY-----
./acrtp/.venv/lib/python3.13/site-packages/googleapiclient/discovery_cache/documents/appengine.v1beta.json:2247:-----BEGIN RSA PRIVATE KEY-----
./acrtp/.venv/lib/python3.13/site-packages/ecdsa/test_pyecdsa.py:245:-----BEGIN EC PRIVATE KEY-----
./acrtp/.venv/lib/python3.13/site-packages/ecdsa/test_pyecdsa.py:258:-----BEGIN EC PRIVATE KEY-----
./acrtp/.venv/lib/python3.13/site-packages/ecdsa/test_pyecdsa.py:272:-----BEGIN EC PRIVATE KEY-----
./acrtp/.venv/lib/python3.13/site-packages/ecdsa/test_ecdh.py:305:-----BEGIN EC PRIVATE KEY-----
./acrtp/.venv/lib/python3.13/site-packages/truffleHogRegexes/regexes.json:3:-----BEGIN RSA PRIVATE KEY-----
./acrtp/.venv/lib/python3.13/site-packages/truffleHogRegexes/regexes.json:4:-----BEGIN OPENSSH PRIVATE KEY-----
./acrtp/.venv/lib/python3.13/site-packages/truffleHogRegexes/regexes.json:5:-----BEGIN DSA PRIVATE KEY-----
./acrtp/.venv/lib/python3.13/site-packages/truffleHogRegexes/regexes.json:6:-----BEGIN EC PRIVATE KEY-----
./acrtp/.venv/lib/python3.13/site-packages/truffleHogRegexes/regexes.json:7:-----BEGIN PGP PRIVATE KEY BLOCK-----
...

You might get a noticeable improvement in secret scanning coverage using this approach, given you will take time to sieve off the false positives ;) Good luck hunting secrets!