Certificate Transparency-driven real-time website categorization with browser rendering and semantic analysis

This article develops and validates a method for streaming website categorization built on Certificate Transparency (CT) monitoring, browser-based rendering, and hybrid semantic analysis of domains for which TLS certificates have been issued. The work is motivated by the need to categorize newly registered web resources in near real time, thereby shortening the interval between certificate issuance and website categorization, including the detection of look-alike domains that reproduce well-known web pages. The aim is a pipeline that combines CT logs, language filtering, content extraction through Playwright/Chromium, keyword voting, URL patterns, and LDA topics as a supporting semantic signal.

The novelty lies in joining CT monitoring with full browser rendering and interpretable NLP classification without a pre-collected labeled corpus. Over 23.83 hours, the system processed 8.51 million X.509 records and produced 32,392 categorized websites across 36 categories, achieving 84.7% accuracy after keyword pruning. The pipeline identified 283 domains replicating the pages of Russian banks, marketplaces, and government services, concentrated across 120 registrable domains.

Rule R3, built on a brand marker and a label-count threshold, achieved Precision 92.1% and Recall 62.2% without rendering and serves as an early-stage filter over the CT stream.