Regex for secret detection
If you are building some kind of system that is supposed to detect secrets you can approach the problem in different ways.
- Entropy based : a Secret is most likely randomly generated like API keys. However, you can also detect false positives if you feed username to the system. With a username like
0x_IaM_tH3_B3A5t_x0your entropy is likely to be really high. - AI based : AI can be really good at detecting patterns. That's what it is after all, pattern detection and pattern recreation on a large scale. However AI are not robust by design, they are complex probabilitic system where an attacker has control over the input. An adversarial example could easily defeat a secret detection algorithm (adversarial examples are inputs to machine learning models that an attacker has intentionally designed to cause the model to make a mistake). It is still an unsolved problem : https://openai.com/index/attacking-machine-learning-with-adversarial-examples/
- Regex : It is the most robust technique considering the false positive rate. Once your Regex are tuned you're garanteed to detect all secrets of the same kind. The hard part though is to list all Regex for all different kinds of secrets.
List of Regex for secret detection :
| SECRET DETECTOR | REGEX | CATEGORY | SEVERITY |
| SSH public key DSA | (^.*(-|_)(dsa|dss|ed25519))|(ecdsa-sha2-nistp256) | PublicKey | HIGH |
| SSH public key RSA | ^.*(-|_)rsa | PublicKey | HIGH |
| Typeform API Token | (?i)(?P<key>typeform[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}(?P<secret>tfp_[a-z0-9\-_\.=]{59}) | Typeform | LOW |
| Twitch API Token | (?i)(?P<key>twitch[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-z0-9]{30})['\"] | Twitch | LOW |
| LinkedIn Client Id | (?i)(?P<key>linkedin[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-z0-9]{14})['\"] | LOW | |
| LinkedIn Client Secret | (?i)(?P<key>linkedin[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-z]{16})['\"] | LOW | |
| Shippo API Token | shippo_(live|test)_[a-f0-9]{40} | Shippo | LOW |
| Sendinblue API Token | xkeysib-[a-f0-9]{64}\-(?i)[a-z0-9]{16} | Sendinblue | LOW |
| SendGrid API Token | SG\.(?i)[a-z0-9_\-\.]{66} | SendGrid | MEDIUM |
| RubyGems API Token | rubygems_[a-f0-9]{48} | RubyGems | MEDIUM |
| Pulumi API Token | pul-[a-f0-9]{40} | Pulumi | HIGH |
| Postman API Token | PMAK-(?i)[a-f0-9]{24}\-[a-f0-9]{34} | Postman | MEDIUM |
| Planetscale API Token | pscale_tkn_(?i)[a-z0-9\-_\.]{43} | Planetscale | MEDIUM |
| Planetscale Password | pscale_pw_(?i)[a-z0-9\-_\.]{43} | Planetscale | MEDIUM |
| Npm Access Token | ['\"](npm_(?i)[a-z0-9]{36})['\"] | Npm | CRITICAL |
| New Relic Ingest Browser API Token | ['\"](NRJS-[a-f0-9]{19})['\"] | NewRelic | MEDIUM |
| NewRelic User API Id | (?i)(?P<key>newrelic[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[A-Z0-9]{64})['\"] | NewRelic | MEDIUM |
| NewRelic User API Key | ['\"](NRAK-[A-Z0-9]{27})['\"] | NewRelic | MEDIUM |
| MessageBird API Client ID | (?i)(?P<key>messagebird[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-h0-9]{8}-[a-h0-9]{4}-[a-h0-9]{4}-[a-h0-9]{4}-[a-h0-9]{12})['\"] | MessageBird | MEDIUM |
| MessageBird API Token | (?i)(?P<key>messagebird[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-z0-9]{25})['\"] | MessageBird | MEDIUM |
| Mapbox API Token | (?i)(pk\.[a-z0-9]{60}\.[a-z0-9]{22}) | Mapbox | MEDIUM |
| Mailgun Webhook Signing Key | (?i)(?P<key>mailgun[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-h0-9]{32}-[a-h0-9]{8}-[a-h0-9]{8})['\"] | Mailgun | MEDIUM |
| Mailgun Private API Token | (?i)(?P<key>mailgun[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>(pub)?key-[a-f0-9]{32})['\"] | Mailgun | MEDIUM |
| Mailchimp API Key | (?i)(?P<key>mailchimp[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-f0-9]{32}-us20)['\"] | Mailchimp | MEDIUM |
| Lob Publishable API Key | (?i)(?P<key>lob[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>(test|live)_pub_[a-f0-9]{31})['\"] | Lob | LOW |
| Lob API Key | (?i)(?P<key>lob[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>(live|test)_[a-f0-9]{35})['\"] | Lob | LOW |
| Linear client Secret/ID | (?i)(?P<key>linear[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-f0-9]{32})['\"] | Linear | MEDIUM |
| Linear API Token | lin_api_(?i)[a-z0-9]{40} | Linear | MEDIUM |
| Ionic API Token | (?i)(ionic[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](ion_[a-z0-9]{42})['\"] | Ionic | MEDIUM |
| Intercom client Secret/ID | (?i)(?P<key>intercom[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-h0-9]{8}-[a-h0-9]{4}-[a-h0-9]{4}-[a-h0-9]{4}-[a-h0-9]{12})['\"] | Intercom | LOW |
| Intercom API Token | (?i)(?P<key>intercom[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-z0-9=_]{60})['\"] | Intercom | LOW |
| HubSpot API Token | (?i)(?P<key>hubspot[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-h0-9]{8}-[a-h0-9]{4}-[a-h0-9]{4}-[a-h0-9]{4}-[a-h0-9]{12})['\"] | HubSpot | LOW |
| HashiCorp Terraform user/org API Token | ['\"](?i)[a-z0-9]{14}\.atlasv1\.[a-z0-9\-_=]{60,70}['\"] | HashiCorp | MEDIUM |
| Grafana API Token | ['\"]eyJrIjoi(?i)[a-z0-9\-_=]{72,92}['\"] | Grafana | MEDIUM |
| GoCardless API Token | ['\"]live_(?i)[a-z0-9\-_=]{40}['\"] | GoCardless | MEDIUM |
| Frame.io API Token | fio-u-(?i)[a-z0-9\-_=]{64} | Frameio | LOW |
| Flutterwave Encrypted Key | FLWSECK_TEST[a-h0-9]{12} | Flutterwave | MEDIUM |
| Flutterwave Public/Secret Key | FLW(PUB|SEC)K_TEST-(?i)[a-h0-9]{32}-X | Flutterwave | MEDIUM |
| Finicity API Token | (?i)(?P<key>finicity[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-f0-9]{32})['\"] | Finicity | MEDIUM |
| Finicity Client Secret | (?i)(?P<key>finicity[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-z0-9]{20})['\"] | Finicity | MEDIUM |
| Fastly API Token | (?i)(?P<key>fastly[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-z0-9\-=_]{32})['\"] | Fastly | MEDIUM |
| Easypost API Token | ['\"]EZ[AT]K(?i)[a-z0-9]{54}['\"] | Easypost | LOW |
| Dynatrace API Token | ['\"]dt0c01\.(?i)[a-z0-9]{24}\.[a-z0-9]{64}['\"] | Dynatrace | MEDIUM |
| Duffle API Token | ['\"]duffel_(test|live)_(?i)[a-z0-9_-]{43}['\"] | Duffel | LOW |
| Dropbox Long Lived API Token | (?i)(dropbox[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"][a-z0-9]{11}(AAAAAAAAAA)[a-z0-9\-_=]{43}['\"] | Dropbox | HIGH |
| Dropbox Short Lived API Token | (?i)(dropbox[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](sl\.[a-z0-9\-=_]{135})['\"] | Dropbox | HIGH |
| Dropbox API secret/key | (?i)(dropbox[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"]([a-z0-9]{15})['\"] | Dropbox | HIGH |
| Doppler API Token | ['\"](dp\.pt\.)(?i)[a-z0-9]{43}['\"] | Doppler | MEDIUM |
| Discord Client Secret | (?i)(?P<key>discord[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-z0-9=_\-]{32})['\"] | Discord | MEDIUM |
| Discord Client Id | (?i)(?P<key>discord[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[0-9]{18})['\"] | Discord | MEDIUM |
| Discord API Token | (?i)(?P<key>discord[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-h0-9]{64})['\"] | Discord | MEDIUM |
| Databricks API Token | dapi[a-h0-9]{32} | Databricks | MEDIUM |
| Contentful delivery API token | (?i)(?P<key>contentful[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-z0-9\-=_]{43})['\"] | ContentfulDelivery | LOW |
| Clojars API Token | (CLOJARS_)(?i)[a-z0-9]{60} | Clojars | MEDIUM |
| Beamer API Token | (?i)(?P<key>beamer[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>b_[a-z0-9=_\-]{44})['\"] | Beamer | LOW |
| Bitbucket Client Secret | (?i)(?P<key>bitbucket[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-z0-9_\-]{64})['\"] | Bitbucket | HIGH |
| Bitbucket Client Id | (?i)(?P<key>bitbucket[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-z0-9]{32})['\"] | Bitbucket | HIGH |
| Atlassian API Token | (?i)(?P<key>atlassian[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-z0-9]{24})['\"] | Atlassian | HIGH |
| Asana Client Secret | (?i)(?P<key>asana[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-z0-9]{32})['\"] | Asana | MEDIUM |
| Asana Client Id | (?i)(?P<key>asana[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[0-9]{16})['\"] | Asana | MEDIUM |
| Alibaba Secret Key | (?i)(?P<key>alibaba[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-z0-9]{30})['\"] | Alibaba | HIGH |
| Alibaba AccessKey Id | ([^0-9a-z]|^)(?P<secret>(LTAI)(?i)[a-z0-9]{20})([^0-9a-z]|$) | Alibaba | HIGH |
| Adobe Client Secret | (p8e-)(?i)[a-z0-9]{32} | Adobe | LOW |
| Adobe Client ID (Oauth Web) | (?i)(?P<key>adobe[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-f0-9]{32})['\"] | Adobe | LOW |
| Twitter Token | (?i)(?P<key>twitter[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-f0-9]{35,44})['\"] | LOW | |
| Facebook Token | (?i)(?P<key>facebook[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[a-f0-9]{32})['\"] | LOW | |
| Age Secret Key | AGE-SECRET-KEY-1[QPZRY9X8GF2TVDW0S3JN54KHCE6MUA7L]{58} | Age | MEDIUM |
| Twilio API Key | SK[0-9a-fA-F]{32} | Twilio | MEDIUM |
| Slack Webhook | https:\/\/hooks.slack.com\/services\/[A-Za-z0-9+\/]{44,48} | Slack | MEDIUM |
| Heroku API Key | (?i)(?P<key>heroku[a-z0-9_ .\-,]{0,25})(=|>|:=|\|\|:|<=|=>|:).{0,5}['\"](?P<secret>[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12})['\"] | Heroku | HIGH |
| Google (GCP) Service-account | \"type\": \"service_account\" | CRITICAL | |
| Pypi Upload Token | pypi-AgEIcHlwaS5vcmc[A-Za-z0-9\-_]{50,1000} | PyPi | HIGH |
| Stripe Secret Key | (?i)sk_(test|live)_[0-9a-z]{10,32} | Stripe | CRITICAL |
| Stripe Publishable Key | (?i)pk_(test|live)_[0-9a-z]{10,32} | Stripe | LOW |
| Slack Token | xox[baprs]-([0-9a-zA-Z]{10,48}) | Slack | HIGH |
| Shopify Token | shp(ss|at|ca|pa)_[a-fA-F0-9]{32} | Shopify | HIGH |
| Private Key | (?i)-----\s*?BEGIN[ A-Z0-9_-]*?PRIVATE KEY( BLOCK)?\s*?-----[\s]*?(?P<secret>[\sA-Za-z0-9=+/\\\r\n]+)[\s]*?-----\s*?END[ A-Z0-9_-]*? PRIVATE KEY( BLOCK)?\s*?----- | AsymmetricPrivateKey | HIGH |
| GitLab Personal Access Token | glpat-[0-9a-zA-Z\-\_]{20} | Github | CRITICAL |
| Github Refresh Token | ghr_[0-9a-zA-Z]{76} | Github | CRITICAL |
| Github App Token | (ghu|ghs)_[0-9a-zA-Z]{36} | Github | CRITICAL |
| GitHub OAuth Access Token | gho_[0-9a-zA-Z]{36} | Github | CRITICAL |
| GitHub Personal Access Token | ghp_[0-9a-zA-Z]{36} | Github | CRITICAL |
| AWS Account Id | (?i)(^|\s+)["']?(aws)?_?account_?(id)?["']?\s*(:|=>|=)\s*["']?(?P<secret>[0-9]{4}\-?[0-9]{4}\-?[0-9]{4})["']?(\s+|$) | AWS | HIGH |
| AWS Secret Access Key | (?i)(^|\s+)["']?(aws)?_?(secret)?_?(access)?_?key["']?\s*(:|=>|=)\s*["']?(?P<secret>[A-Za-z0-9\/\+=]{40})["']?(\s+|$) | AWS | CRITICAL |
| AWS Access Key Id | ["']?(?P<secret>(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16})["']?(\s+|$) | AWS | CRITICAL |
