Total domains20690unique domains committed to the crawler database
New CT domains (session)684brand-new unique domains inserted by CT since crawler start
New CT domains (last batch)409actual new domain rows, not raw certificate entries
Blocked8279current BLOCK verdict
Under review52current verdict · independent from queue state
Observe12292current verdict
Allow0current verdict
Unknown67not yet assigned a final verdict
Total processing backlog0admission + realtime + review + deep waiting
Admission / pre-screen waiting0discovered / passive / pre-scored
Pending realtime screen0waiting for realtime verification
72h retry scheduled2445long maintenance lane · zero hot retry
DNS retry (72h)first DNS failures parked outside realtime
DNS retry due nowprocessed together by the DNS maintenance sweep
Offline blacklisted5530failed the 72-hour DNS confirmation
Recovered online1641offline domains seen online again this session
Retry due now0claimed only by the dedicated retry lane
Retry crawlers active012 ready · 0 active means no retry is due
Review verification waiting0verdict=review + pending_enrich
Deep enrichment waiting0pending_enrich excluding review
Realtime screening now0active realtime workers
Review verification now0active review workers
Deep analysis now0active deep workers
Unique source domains20690unique domains represented by evidence sources
Source memberships20927non-additive · a domain may belong to multiple sources
Active sources7distinct evidence sources
Crawler-confirmed source hits9independent malicious page confirmation attributed back to discovery sources
Phishing-source domains252OpenPhish / PhishTank / custom
Campaigns83clustered groups
Internal reputation keys211482IP / page / form / redirect intelligence
Learning statestartingcontinuous learner remains active even when requeue is backpressured
Confident learned signals0fresh, repeatedly confirmed internal reputation features
Learning matches0historical rows matched in the latest self-learning step
Learning requeue budget0speculative SCREEN admissions allowed by current pipeline headroom
Agent policybootstrapbootstrap / calibrating / active / degraded
Agent feature memory0generalized semantic, behavioral and domain-shape features
Agent trusted positives0independently confirmed malicious training examples
Agent trusted negatives0reachable clean pages used as negative labels
Agent precision0.0precision on new trusted labels; zero until evaluation examples exist
Agent recall0.0recall on new trusted labels; tracked without self-training leakage
Agent evaluation events18026new trusted outcomes used to evaluate current learned memory
Malicious clusters187signals shared by 2+ blocked domains
Crawler stagecontinuous:activelive runtime state
Nightly full rescanwaiting-for-capacitybounded low-priority verification during the local night window
Nightly heartbeat2026-10-13T01:36:58+00:00proves the scheduler is alive even while waiting for its local window
Night unresolved UNKNOWN67historical UNKNOWN debt gets precedence over generic rescans
Night unresolved REVIEW52REVIEW debt gets precedence over generic rescans
UNKNOWN drainidlebounded 24/7 admission of legacy UNKNOWN rows
REVIEW debt drainidledue durable reviews are fed to dedicated verification workers
Nightly progress99.43%snapshot rows traversed in the current nightly run
Nightly rows queued12738historical domains submitted for fresh realtime verification
Nightly last block98eligible domains queued from the latest bounded nightly admission
Nightly learning sweep0historical rows consolidated after the latest rescan block
Nightly agent training0trusted refreshed outcomes that updated learned semantic features
CT cycle statepaused:cohort-rescan10-minute bounded CT bursts with adaptive drain
DB drain statereadyCT pauses at high-water 600 and resumes at/below low-water 180 after stable checks
DB drain workload0queued + active realtime / review / deep work
Drain queued0durable work still waiting in PostgreSQL
Drain active0realtime + review + deep jobs currently executing
CT pause reasoncohort-rescandownstream-workload means CT is intentionally giving classifiers priority
CT cycle workers0active only while a LIVE cycle is running
CT live logs ready16persistent cursors compare only unseen entries
RFC6962 CT logs9current legacy get-sth / get-entries shards
Static CT logs9current tiled_logs / Static CT API shards
Static CT entries0entries consumed from tile/data in the active burst
Healthy CT logs15logs currently eligible for requests
CT logs in cooldown3only failed/rate-limited logs pause; healthy logs continue
CT rate limited0HTTP 429 logs under per-log backoff
CT last successful request2026-09-26T20:24:26+00:00proves at least one CT endpoint is still advancing
CT supervisorrunningrestarts the CT lane without stopping classifiers or feeds
CT restarts0automatic CT-lane restarts since crawler service start
CT heartbeat2026-09-26T20:24:24+00:00fresh heartbeat while a CT engine is alive
CT inflight chunks0must return to zero after a bounded CT burst
CT stopping since—hard cleanup timeout prevents a permanent stopping state
CT forced cleanup11 means the last CT engine required forced bounded cleanup
Last CT cycle entries576new CT entries processed in the last comparison
Memory guardnormaladaptive crawler pressure level
Crawler memory243.2MiB currently charged to the crawler cgroup