GitHub described credential detection combining regex and the ModernBERT model

GitHub described a deployed credential detection scanner that combines regex rules with a fine-tuned ModernBERT model. According to the company, the median scanning time after changes were pushed fell by 86 % compared with the previous LLM-based approach. The company says this speedup made it possible to extend generic credential detection to push protection, which checks changes before they are pushed.

Regex rules first find strings that resemble credentials. The ModernBERT model then assesses their value, file path, credential type, variable names, and surrounding code or text. Based on the resulting score, the scanner generates alerts or filters out likely harmless values, such as example credentials, checksums, and values in tests.

During training, the company emphasized examples of both real credentials and similar harmless strings. Before splitting the data, it grouped related examples to prevent nearly identical records from appearing in both training and evaluation. It selected decision thresholds during offline evaluation and then verified the behavior on traffic resembling a production environment.