Google Search Console

Module id: gsc · Mounted at: /api/modules/gsc (+ legacy alias /api/modules/seo-analytics) · No own settings row — its keys (gscSiteUrl, gscServiceAccountJson, gscVerified, cronGscSync) namespace into the '1a-blog-publisher' row.

Purpose

Pulls Google Search Console performance into the app: a per-workspace SEO analytics dashboard (impressions, clicks, CTR, position, keyword rankings, top pages) and an authoritative index-status signal — "is this page actually in Google's index?".

Features

• Ghost pages detector (agent-signals.js → classifyGhostCandidates + HTTP confirm, 2026-08-02 — the Antwerp case: a deleted landing page kept earning 670+ impressions/28d from GSC's trailing window and an AI analyst built a critical storyline instructing edits to a page that redirects to the homepage): URLs still EARNING impressions (≥25/28d, bare origin excluded) that the live sitemap no longer lists, each confirmed by following its redirects — gone means 404/410 or a redirect settling on the homepage/a listing page; a 301 onto another sitemap-listed content page is a healthy migration and is deliberately skipped. ONE aggregate atom (gsc:ghost-pages), ≤8 confirms/sweep paced ~1/s, unverified overflow named in the card. Sitemap unreachable → silent (no verdicts without ground truth).
• Site-level daily series (daily-totals.js → gsc_daily_totals, 2026-08-08): one row per workspace/date, backfilled 16 months on the first run and refreshed nightly. The date dimension was already fetched on every dashboard load and thrown away; nothing stored a site-level daily number, so the trend was capped at 90 live days and a new workspace saw nothing for a month. Google keeps 16 months and then deletes — past that this table is the only copy. One ['date'] request serves both backfill and refresh (~490 rows over 16 months), reusing the property getWorkspaceGscData already proved, and re-requesting a 10-day trailing window because Google revises recent days. Rows are upserted on (workspace_id, date, dimension, value), so a backfill and a nightly run may overlap freely. ctr is derived at read time, never stored (in GSC it is exactly clicks ÷ impressions, and a stored copy is a second definition); position is stored (an impression-weighted average nothing downstream can recompute) and never sums across days. The dimension/value key columns are constant ('total','') today and exist so the P3 breakdowns need no key migration. Not pruned: it grows one row per workspace per day, not per URL per day. Live check on the two workspaces that share one property by subdirectory — Forest Forward 822 clicks / 13,123 impressions against Give It Forward 57 / 3,806 over the same 357 days, which is also the proof that the anchored path filter scopes correctly.
• Per-query daily series (daily-queries.js → gsc_query_daily + the gsc_query_rollup RPC, 2026-08-09): one row per workspace/date/query, backfilled 16 months on the first run, refreshed nightly, never pruned. This retires the top-500 cap, which was ours and not Google's: the dashboard asked for 500 queries while Luniq has 1,885 in 28 days and 11,616 over 16 months, so "the site's keyword rankings" was 4% of its demand. It also ends a real drift — the mechanical detectors aggregated each PAGE's stored top-20 queries while the agent tool read the dashboard's top 500, so one query could carry two different best_position values depending on who asked. Sized before building: 195 rows/28d (Forest Forward) to 7,883 (Luniq); a full 16-month backfill of the largest is 72k rows in 28s, 3 pages, ~7 MB, and the nightly increment is 2.5s. position is impression-weighted in SQL (gsc_query_rollup) so there is one definition of a period's position; ctr is derived; days_seen rides along because 400 impressions on one day and across 28 are different facts. ⚠️ The rollup is read through selectAllPaginated — an RPC returning a table is subject to the same silent 1000-row PostgREST cap as a select, and the first live run returned exactly 1000 of 1,883 queries with nothing saying so.
• Branded vs non-branded (brand.js): mechanical, from the workspace's own name and domain — plus, since signal plan step 4 (2026-09-01), the brand/product NAMES the person wrote into their own profile (derivedBrandTerms, extracted once per profile change by the offer judge; the model only lifts names the person authored — sister brands, product names, the Byve "vebe" class). Matched with spacing and punctuation squashed out (so luniq, l'uniq and luniq stays all classify together). No model ever classifies an individual query — a classifier that sometimes disagrees with itself makes the two totals unreconcilable. Deliberately generous on the branded side, because misfiling branded traffic inflates "demand we won", the number that flatters. Terms under 4 characters and generic domain labels are refused; branded is null when no usable terms exist. Since step 4 the split also rides every DETECTOR row (aggregateSiteQueries sets branded) and the opportunity lanes (classifyQueries, classifyCtrLeaks) skip branded === true — someone typing the brand already found the business (audit defect 2).
• Dimension dailies + the breakdown (daily-dimensions.js + services/breakdown.js → gsc_dimension_rollup RPC, 2026-08-09): surface (web/discover/image/video/news — one ['date'] call per searchType), country (['date','country'], alpha-3) and device are STORED daily in gsc_daily_totals under the dimension/value key (16-month backfill, zero-rows skipped so absence = no traffic; Luniq: 20,960 rows in 8.5s). searchAppearance is served LIVE per period — Google refuses to group it with date (verified live: "Cannot group by search appearance dimension together with another dimension"), and Google holds 16 months, so any askable period is answerable without an archive. branded is DERIVED at read time from gsc_query_daily + brand.js — never stored, so it cannot drift from the per-query flags — with Google's withheld queries surfaced as an explicit anonymized row (measured 55% of Luniq's clicks) rather than letting branded + non_branded masquerade as the whole. content_type is derived from the cached fetch + the page catalog's own page_type, not URL fragments. All six come through ONE read (getSearchBreakdown) shared by the tool and, from P6, the Performance page. country/device cover web search only — a device split across surfaces would double-count against the surface dimension.
• Daily sync (runDailyGscSync) — registered on the scheduler's measure lane (ctx.cron.measure, 2026-08-16: kill-switch + active workspace only, so it keeps pulling for a half-onboarded or paused workspace — zero paid calls, Google's APIs, and the history is time-bound; the scheduler body returns "nothing to measure" when no gscSiteUrl is set yet): one forced GSC fetch feeds two halves, partitioned by OWNERSHIP since the corpus-canonical-split (2026-08-16, plans/corpus-canonical-split.md) — URLs matching an Orbit-written article (source='generated'; exact URL → normalized URL → collection-scoped slug+locale, the match lives in gsc-fetch.js) → blog_seo_snapshots; every other URL, INCLUDING pre-existing posts under a collection base → site_page_snapshots (which carries the full 7d/28d/90d window set since the site-page-snapshots-90d migration). Both halves are strictly read-only against their tables: a GSC URL with no matching row is skipped, never inserted. GSC article discovery is REMOVED — the old path partition minted source='discovered' blog_articles rows for any URL under a collection base (liveness-gated, 150/run), which duplicated site_pages rows for up to a week and starved attributed pages of metrics entirely (10k+ attributed pages fleet-wide held zero snapshots); a URL Orbit didn't write now belongs to site_pages, full stop. The articles half still adopts GSC-confirmed URLs on the cron path (adoptUrls: true — heals slug-tier-matched URL drift; adopting a URL is CLAIMING it, so the adoption also releases any site_pages twin at that address via the spine's releaseSitePageTwins — measured on Matubu, whose collection base URL is not where Shopify serves the blog, so publish-time URLs sat stale beside the real ones). The matcher itself (buildArticleMatcher, exported + pinned in scripts/test-corpus-split.mjs) scopes the slug tier to the collection that owns the URL's path, or to the SOLE collection on a root-hosted workspace (marcom.wisdom shape — no base paths to scope by), and never guesses across two root collections. Time-boxed (360s), 1095-day (3-year) snapshot retention prune (raised from 180 on 2026-08-15 — see below), and re-ranks the key-page tier (selectKeyPages) after fresh impressions land.

Retention vs what the agent may ask for — two different numbers

SNAPSHOT_RETENTION_DAYS (how long rows LIVE, now 1095) and URL_MAX_DAYS in agent-tools.js (how far back a tool may ASK, still 180) are deliberately decoupled. At 180/180 the per-article search series could never exceed six months, so "how long does a page take to be indexed, impressed and then clicked after publish" was permanently uncomputable past half a year — the question these snapshots exist to answer. Raising retention alone changes no product behaviour: the agent offers exactly the window it offered before, we simply stop destroying the history behind it. Measured cost of keeping it: ~30 rows per site per day, ~1.1M rows and ~200 MB a year at a hundred sites. The prune stays as a safety valve; if the table ever gets heavy the answer is partitioning by month, which keeps the history.

• URL Index Inspection (runIndexInspection): for URLs with zero impressions, calls Google's URL Inspection API to distinguish "indexed but no traffic" from "not indexed". Four lanes under one 500/run cap, in priority order: contradictions ("not indexed" + earning — the data disagreeing with itself; the Antares class. Re-verified NIGHTLY since 2026-08-23 (CONTRADICTION_RECHECK_HOURS = 20 — deliberately under the 24h cadence, because a 24h threshold against a fixed-clock cron is a millisecond coin-flip that skips the row a whole extra night half the time; capped at 150/run and logged when it drops any) — the shared 7-day cycle was the wrong cadence for the one lane whose premise is "this row is probably already wrong": Luniq's /en homepage was re-indexed by Google, kept a false verdict from Aug 16, and no lane would re-ask until Aug 23, so the app showed "Not indexed" for six days after the fix on a page serving 27 impressions. Fleet-wide the lane holds 27 rows averaging 3-7 days of staleness; nightly makes that ≤1 day), primary (never-inspected + zero-impression stale, 7-day recheck), rotation (earning pages incl. never-inspected earning ones, 30-day floor — canonical/rich-result drift watch), listed sweep (never-inspected zero-impression listed tail, URL-ordered so each night continues where the last stopped — 44,320 of 45,481 tracked pages fleet-wide had NO verdict before this lane existed). Concurrency 3; writes gsc_indexed, gsc_coverage_state, gsc_rich_results + the P4 diagnostics. deriveIndexed() / isIndexed() combine the impressions proxy + the inspection verdict by the who-is-newer rule (2026-08-09, two live cases the same day pulling opposite directions): a false verdict from inside the 28-day window WINS over trailing impressions (noindexed legal pages showed "Indexed" because last month's impressions hadn't rolled off) — UNLESS the page earned 7-day impressions against a >7-day-old verdict, in which case Google served it after the verdict and the verdict LOST (the Antares case: false from Jul 23, indexed and earning by Aug 9). Same rule in SQL for the Pages list (list_site_pages v3). Pinned in test-gsc-tools.mjs as paired cases so neither fix can undo the other. P4 (2026-08-09): the diagnostic half of the same response is now kept — gsc_google_canonical / gsc_user_canonical (the canonical conflict pair, both tables), gsc_inspection_link (deep link into the customer's own Search Console), and on site_pages the crawl forensics (gsc_crawled_as, gsc_page_fetch_state, gsc_robots_txt_state, gsc_referring_urls, gsc_sitemaps). And a rotation lane fixes the policy blind spot the canonical fields exposed: zero-impression-only inspection meant an EARNING page was never re-inspected, so the conflicts that cost something were exactly the ones that could never be seen. Up to 100 earning key pages + articles per run, oldest inspection first, 30-day floor, filling only the headroom the zero-impression set leaves under the 500 cap.
• Stale-verdict reconciliation (reconcileStaleCoverage / reconcileEarningRows, 2026-08-05): an Inspection verdict is a point-in-time fact; impressions are a trailing-window fact. When a URL earns impressions in a window that began AFTER its last inspection, the stored verdict predates the evidence and is stale. Both metrics halves call reconcileEarningRows after the snapshot upsert: gsc_indexed escalates to true on evidence alone (no string parsing, so it works on any property language), and gsc_coverage_state is cleared ONLY for the "not indexed yet" families (isNotYetIndexedState: unknown-to-Google / discovered / crawled-not-indexed) — structural verdicts (404, redirect, blocked, duplicate) are kept because they describe the URL itself. A verdict written INSIDE the impressions window still wins, so a fresh deindex is never overwritten. Why: pickCandidates deliberately skips URLs with impressions, and the old escalation was gated .is('gsc_indexed', null), so once Inspection wrote false a row could never recover and was never revisited — 289 fleet articles were flagged not-indexed while actively ranking, and Health raised not-indexed issues for page-one pages.
• Dashboard (getDashboard): 7 live GSC queries → KPIs with deltas, keyword rankings with prev-period position/click/CTR deltas + isNew, per-keyword page breakdown resolved to articles/site pages for deep links, and content-type buckets (Blogs/Location/Industry/Service/Other). Cached per (workspace, range) for 5 min with in-flight dedupe. Ranges: 7d / 28d / 90d.
• Property fallback walk (runGscWithFallback) across candidate identifiers (configured, origin, www-flipped, sc-domain:); persists the working one + gscVerified.
• Connection test (services/connection.js → testGscConnection / accessDeniedMessage, 2026-08-08): the one place "can we read this property?" is decided, lifted out of api/routes/onboarding.js. It authenticates as the WORKSPACE's gscServiceAccountJson — the route used to hardcode getGscClients(null), so a workspace with its own credentials was verified under an identity its syncs never use and the test could pass while every sync 403s. Refusal is classified by isGscAccessError (not by grepping the message for 403) and the account is named from gscServiceAccountEmail (not a string literal).
• What can this connection actually read (listAccessibleProperties → Sites API sites.list): on refusal, Orbit asks Google which properties the service account CAN read and offers them. The cause is almost always that a different property was granted than the one configured (domain vs URL-prefix, www vs apex), so a bare "Google refused" is unactionable. Feeds the connection-test 403 payload (availableProperties) and gsc:access-denied evidence (readable_properties). Unverified entries are dropped — Search Console lists them but they return no data.
• One property resolver, and unfinished connections do NOTHING (policy, 2026-08-09). resolveGscSite(settings) is gscSiteUrl only when gscVerified === true — a URL whose Test-connection never passed resolves to null, so every consumer (syncs, backfills, inspection, detectors, dashboard, reports, breakdown) treats the workspace as not connected and spends zero work on it. Measured before the rule: Event Online and MICE each burned 500 failed Inspection calls in one run on never-granted properties, Ledoux 85, and the nightly sync would have repeated it. The connection test reads the URL from its own request, so becoming verified stays reachable; the ops:gsc-not-connected card (no API calls) carries the setup nudge. Historically the resolver was gscSiteUrl and nothing else. It used to fall back to wordpressApiUrl / drupalSiteUrl / shopifyStore (a CMS API endpoint is not a GSC property), and services/dashboard.js bypassed it entirely for settings.gscSiteUrl || process.env.SEARCH_CONSOLE_SITE_URL — so the dashboard could target a different property than the sync, and one env var could point the whole fleet at one property. The env var is gone; all 16 live workspaces already carried an explicit gscSiteUrl.
• Path scoping is a segment match, not a substring (buildPathFilter → includingRegex). Two live workspaces share one property by subdirectory (forestforward.be/forestforward, .../give-it-forward), and contains '/shop' also matches /blog/tags/shop and ?ref=/shop. The optional locale group in the pattern is load-bearing: real URLs are /{locale}/{workspace-path}/…, so a naive ^https?://[^/]+/shop anchor matches zero of them and would silently empty the sync. Verified against the live catalogue: 42/42 and 24/24 own-prefix matches, no cross-matches, no hits among the other 45,452 pages. Pinned in scripts/test-gsc-tools.mjs.
• No Indexing API submit (removed 2026-08-08, with the auth/indexing scope). submitForIndexing / requestGoogleIndexing posted every published article to urlNotifications.publish, which Google honours ONLY for JobPosting / BroadcastEvent markup — so for an article it accepted the request, discarded it, and we logged a green checkmark. Its 200-publishes/day quota is per PROJECT, so one shared service account made the whole fleet compete for calls that could never work. Discovery is the sitemap plus internal links, both already checked by Health (sitemap-*, orphan-pages). The GSC client is now read-only.

Routes

Method	Path	Purpose
GET	/:workspaceId/dashboard?range=28d	Live KPI aggregate (6 GSC queries since P6 — the ['page'] call is gone). Feeds get_workspace_performance + the pre-backfill fallback
GET	/:workspaceId/series?days=&compare=	Stored site series + period totals for the tiles (seriesTotals: clicks/impressions sum, position impression-weighted, ctr derived)
GET	/:workspaceId/breakdown?dimension=&range=	One non-query dimension via getSearchBreakdown — same computation as the agent tool
GET	/:workspaceId/queries?range=	Every query for the window via listSiteQueries, rows carrying their attributed pages — identical to get_search_queries rows
GET	/:workspaceId/queries/series?query=&days=	One query's daily trajectory
POST	/:workspaceId/recheck {table,id}	On-demand index recheck (services/recheck.js, 2026-08-23) — one URL, one Inspection call, verdict written through applyInspection. table is site_pages or blog_articles. 60s per-row cooldown (read off the row's own gsc_last_inspected_at, so it survives a restart and costs no memory); inside it the STORED verdict comes back with status:'cooldown' rather than an error. 400 not-connected/no-URL, 404 unknown row, 502 access refused or ERROR verdict

The Performance page reads /series, /breakdown and /queries; the tiles, chart, dimension cards and query table are therefore the agent tools' own numbers rendered.

Agent tools

Tool	What it does
get_search_queries	Site-wide query report, now sourced from gsc_query_daily via rollupQueries — every query Google reported, no top-N cap, with total_queries_in_period as the real count. Per query: clicks/impressions/ctr_pct/impression-weighted position, the previous-period comparison (position_prev/position_delta/clicks_delta/impressions_delta/ctr_delta, is_new), days_seen + first_seen + last_seen (a spike and a trend can carry identical totals; a faded query shows it), branded, and near_ranking. Filters: query_contains, position_min/position_max, min_impressions, `branded: 'only'
get_performance_history	The DAILY series, two scopes. With url: ONE article/site page (getSnapshotHistory — resolves against blog_articles then site_pages, trailing-slash tolerant, up to 180 days) — per-day 7d + 28d + 90d impressions/clicks/CTR/position (SERIES_COLS carries all three windows for both tables since 2026-08-23 — bug B6 of the tool audit: page rows snapshotted before the 90d migration read 0 there, which the tool description says); include_queries adds each day's stored top_queries_7d/28d. With url OMITTED: the site-wide series from gsc_daily_totals (source:'stored', up to 480 days, zero GSC quota) — the total, or ONE stored dimension member via dimension + value (surface/country/device, getSiteSeries({ dimension, value }), 2026-08-23; an unknown member returns an empty series with a note naming the spelling rule, never the total in disguise), plus compare_previous for the equally-long preceding period as previous_series — shape against shape rather than one delta number, with previous_period_partial naming any shortfall so a history limit is never read as a decline. Falls back to the live dashboard window (source:'live') only while a workspace's store is still empty, labelled so the shorter series is not mistaken for all the history there is. Every response carries history_from + data_through (per-URL: its own bounds + retention_days), because "the series starts in March" otherwise reads as "the site launched in March". The description states that clicks/impressions sum across days and position does not. 2026-08-23: default days 28 (was 90); kept as days because the result is a series whose length is the ask, not a report window. getSnapshotHistory looks the URL up via core/utils/url-forms.js#urlForms (trailing slash / https tolerant).
get_search_breakdown	The non-query dimensions through getSearchBreakdown: surface / country / device (stored, per-dimension history_from/data_through), appearance (live, overlap-noted), branded (derived, with the anonymized remainder as its own row), content_type (derived, deltas explicitly NOT COMPARED). Rows carry clicks/impressions/ctr/position/share_pct + previous-period deltas; notes[] carries each dimension's caveats verbatim so the model repeats rather than smooths them. offset/limit/truncated for the country tail. 2026-08-23: explicit from + to beside range (same resolveWindows(); content_type rides the cached fixed windows and serves the nearest preset with a note). Tool file: agent-tools/search-breakdown.js.

Signal detectors (mechanical, no LLM)

agent-signals.js → classifyQueries() runs on the nightly agent sweep over aggregateSiteQueries. Since 2026-08-09 that helper reads the stored per-query series (rollupQueries, complete, impression-weighted position in SQL) with page attribution from the cached fetch — the same rollup the agent tool reads, so a card and the agent's answer about it can no longer carry two different positions for one query. Property-level counting also ended the page-sum inflation (a query two of the site's pages shared on one SERP counted its impressions twice), so the volume floors now measure true demand. A workspace whose 16-month backfill hasn't run yet falls back to the legacy per-page aggregation (aggregateSiteQueriesLive — delete when the fleet is backfilled). Probe-verified on Luniq before/after: same signal count, no junk flood, and one real catch the old source was structurally blind to — a query slid 19.8 → 35 and out of every page's top-20, which removed it from the exact aggregation meant to report the slide; the store kept the whole fall. Two outputs, deliberately different kinds:

Detector output	Category	Carries how?	Why
Uncovered demand — impressions ≥ 25, 0 clicks, best position > 20, NOT already answered, collapsed to one card per target page	topic	Yes (write_article)	Google surfaces the site for the query but no page answers it. Evidence class gsc_query — demand measured on this site, the strongest class in the doctrine. Born FOR a collection: the stamp (how.params.collection + evidence.collection_id) is the collection whose base URL the demand lands under (spine collectionForUrl, longest-prefix), else the position-first PRIMARY — the write destination is decided at bank time, never guessed at click time.
Page opportunity — classifyPageOpportunities(): ONE card per page (gsc:page-opportunity:<path>), its levers listed inside it: click_through (position ≤ 5.5, impressions ≥ 50, 0 clicks — or, where the site has ≥ 500 impressions of its own evidence at that rounded position, CTR under half the site's own curve, siteCtrCurve()), near_page_one (impressions ≥ 25, band 8–20, ranked by click potential ≈ 10% × impressions − clicks, floor 3), owned_but_buried (the query is essentially the page's own slug, alreadyAnsweredBy() ≥ 0.7 token overlap; the homepage answers its head terms), sliding (an established ≥ 100-impression query slid 2+ positions, 7d vs 28d; absence from the 7d window is never a collapse), competes (cannibalPairs() over aggregateQueryPages: total ≥ 30 impressions, ≥ 2 pages each ≥ 20% share min 10 at position ≤ 30, a top-3 winner ends it — each page's card names the other). Four pair guards, three of them written against the first live probe (Luniq, 2026-09-01, where 3 of 4 pairs were false): same-slug locale alternates and migration aliases never pair (original); a BRANDED query never pairs (the pair source aggregateQueryPages carries no branded verdict of its own, so the set rides in from the site aggregation — "luniq" paired / with /en); a pair spanning two locale PREFIXES is one piece in two languages whatever the slugs say (a translated slug defeats the path-equality guard: /nl/…/websitesnelheid-meten-… vs /en/…/pagespeed-vs-gtmetrix-…); and a locale ROOT (/, /nl) is never a contender — the front door surfaces for everything the site is known by, and there is no pick-a-winner play against your own homepage	content	No — deliberately	Signal plan step 5: the audit's worst noise pattern was one page collecting up to 11 cards from per-window per-lens families (near-ranking, CTR leak, owned-but-buried, position decay, cannibalization — all retired into this). The strongest lever titles the card (clicks-at-stake), severity warning when sliding/competing, effort M when multi-lever, measure page_search clicks_28d, every query on evidence.queries for the linked-keywords seed. A page already exists; the play is a refresh, which is a person's job — a how here would let the autopilot write a second page competing with the one that ranks. Catalog-aware: a lever on a pruned page (not in getCatalogUrlSet) is the page's removal, not a finding.

classifyMovement() is RETIRED (signal plan step 5): position decay is the per-page card's sliding lever above, and the weekly gainers/losers left in step 3 for the board's one movement subject (agent/detectors/trajectory.js), which reads this module's getSiteSeries/rollupQueries/getBrandTerms and judges every move against the same weeks last year. The two sweeps (query_opportunities + query_movement) are ONE detector now — query_opportunities reads the 28d and 7d rollups once, runs classifyQueries (the topic bank) and classifyPageOpportunities (everything about pages), with aggregateQueryPages and the catalog fetched best-effort: a failed pair read costs the competes lever, never the card.

A broken integration is itself a finding. If the GSC read throws, query_opportunities returns one signal carrying Google's own message instead of failing the detector, because every search-based signal for that workspace is then silently missing and nothing else in the sweep can say so. One breakage is one signal — and since step 5 the integration states share the site:measurement subject with the pixel's silence and the profile-gap card, so a workspace with several unreadable instruments works them from one card.

Which signal depends on isGscAccessError() (search-console.js — the ONE classifier, shared with the candidate-property fallback):

Google says	Signal	Severity	Because
403 / 404 — the property was never granted to our service account	gsc:access-denied	warning	Setup was never finished. Nothing broke, and nobody but the workspace can fix it — so the detail names the exact property and the service-account address to add under Search Console → Settings → Users and permissions (gscServiceAccountEmail(settings), env fallback included).
anything else	gsc:integration-broken	critical	A connection that used to work stopped. That is ours to look at.

The nightly cron stays quiet about the first row. runDailyGscSync tags each phase error setup: true when it is access-class, and scheduleGscSync throws only on the rest — an unfinished connection used to email the admin an error every night for a permission only the workspace can grant (live: sc-domain:ledoux.be, sc-domain:eventonline.be). It logs a warning and lets the Signals board do the asking.

The logging follows the same classifier, in one place. pushError() files the phase error and logs it — console.warn when setup, console.error otherwise. And inspectUrl() does not log access refusals at all: access is a property of the connection, not of the URL, so it fails identically for every page in the run. Left per-URL, one ungranted property wrote ~20 error rows a night (Ledoux Media) into the dashboard the setup flag had already taught the alerting to ignore.

One gap, one card (dedupeByTargetPage(), fleet audit 2026-08-08). 22 of 55 banked ideas across the fleet were the same gap phrased differently: one workspace banked six cards all pointing at /wat-doet-een-social-media-manager/, another banked one AV-cost topic three times, a third banked the same horeca query twice with the words reordered. Every one is a real query with real impressions, so no threshold catches them — what makes them ONE opportunity is that they land on the same page, which is the thing the site would write or fix. Uncovered demand now collapses to one candidate per best_page, the highest-impression query representing it; the siblings ride evidence.same_page_queries + same_page_impressions and the angle tells the writer to cover them in one piece. Rows with no best_page are never collapsed (nothing to land on = no evidence they are the same gap). Redundancy measured LARGER than off-territory banking and is the cheaper of the two to fix: pure, no model, no settings.

Presence is registered before the collapse, not after. A sibling that stops getting its own card is still true, and reconcileAbsentSignals auto-resolves any fingerprint absent for three sweeps — so collapsing before the present sink would silently close real demand as "fixed". Asserted directly in the safety test.

Capped per sweep (highest impressions first) with per-query fingerprints, so a recurring finding bumps times_seen instead of duplicating. Every classifier is pure and asserted directly in backend/scripts/test-autopilot-safety.mjs — including the page card's no-how invariant, the per-page presence sink, the own-curve evidence floor, and the locale-alternate exclusion. scripts/probe-detectors.mjs dry-runs the whole sweep against a live workspace (--curve dumps the CTR-by-position calibration data) — the judgment harness detector work goes through before shipping.

Sitemap health from Google's side (sitemap_health → classifySitemapIssues + listSitemaps via the Sitemaps API, 2026-08-09): Orbit's audit reads sitemaps over HTTP (structure, entries, robots linkage — the sitemap-* checks); only Google can say whether GOOGLE has fetched them, when, and what it rejected. ONE aggregated signal (gsc:sitemap-issues) when any sitemap carries errors (severity warning), warnings, or hasn't been downloaded in 14+ days (a stalled discovery pipe — new pages keep publishing and stop being found). Title names the WORST kind present (the first live run titled a warnings-only finding as a stalled pipe — the sitemap had been fetched the day before). Evidence carries per-sitemap verdicts + Google's own submitted-vs-indexed URL counts. Freshly-submitted pending sitemaps are normal, not findings; a MISSING sitemap stays the audit's sitemap-missing. Read failures stay quiet — one broken integration is one signal (query_opportunities owns it). Live: Luniq's sitemap carries 10 warnings Google reports and nothing HTTP-side could see. The card EXPLAINS itself (2026-08-19, Leon: "the title makes no sense and the 10 warnings are not explained"): Google's Sitemaps API returns COUNTS per sitemap, never the entries — so the title reads Search Console reports 10 warnings on sitemap.xml: entries Google read but will not use (errors: Google could not read part of it), the detail says what a warning/error IS (read fine but some entries unusable — redirects, errors, noindex, other property — vs. unreadable), facts name each sitemap's numbers, Google's last fetch and the submitted count (indexed EXACTLY 0 is the API artifact and is never printed; N of M indexed only when nonzero) plus the one place the entries are listed — the Search Console Sitemaps report (evidence.report_url, https://search.google.com/search-console/sitemaps?resource_id=<property>) — and plan walks: open the report, fix or drop the flagged entries, Google re-reads on its next fetch. Orbit's own crawl checks the same sitemap (sitemap-broken-entries, sitemap-noindex-entries) and those land on Site health.

Top-earner decay — RETIRED (signal plan step 3, 2026-09-01): the top_earner_decay detector and classifyTopEarnerDecay are gone. The board's one movement card names "the pages that carry the loss" from the same getPageClickWindows read (worst deltas, hydrated via getPageIdentities), with the year-ago guard the per-window watch never had; its open rows closed superseded via RETIRED_FINGERPRINTS (gsc:earner-decay:*).

getPageWindowMetrics (step 6, 2026-09-01): one page-dimension Search Analytics query at an arbitrary offset from the latest data date (offsetDays: 365, days: 28 = every page's same-window-last-year read) — exported through the barrel for the broker's refresh_prune detector, and deliberately NOT part of the cached getWorkspaceGscData fetch every consumer pays for.

Why it exists: the drafting lane consumes the topic bank, and the bank's only other producer is the research model's topic_idea event, which had never fired once across the entire fleet. This is the floor that cannot miss.

Correctness gates (check:data, added 2026-08-09)

Ported from the live end-to-end probe that shipped the stores — every one of these failed (or nearly failed) at least once during that test, which is the house bar for becoming permanent:

• gsc.store-physics (fail) — clicks ≤ impressions, position 1–500 or NULL, no future dates, across both daily stores. Its first-ever run caught 195 backfilled rows carrying position 0.00: GSC returns position: 0 on zero-impression days and ?? null kept the 0 (not nullish) — the same impossible-position-0 class the snapshot writers were burned by on 2026-08-03, now fixed at the mapping (r.position > 0 ? r.position : null) in all three syncs and repaired in the store.
• gsc.query-sums-within-totals (fail) — per-day query sums ≤ site totals on finalized days; the anonymized share can never be negative. Catches a path filter applied to one store but not the other.
• gsc.stale-index-contradictions (fail) — no article carries a STALE (>28d) not-indexed verdict while earning impressions; the reconciler's rule, held permanently.
• freshness.gsc-daily-stores (warn) — a backfilled workspace's store is ≤3 days behind.

Reconciliation methodology (the probe, retained as practice): finalized days (≤ end−2) must be EXACT against live — Google revises the trailing ~2 days continuously and the nightly 10-day trailing refresh re-syncs them, so fresh-day drift is expected and never a finding. Live-verified 2026-08-09 on three workspaces: finalized daily series and period sums exact, top-50 query rows exact, every finalized-window live query present (pathological >512-char pasted-prompt "queries" live under their truncated key — the btree guard, by design), per-day conservation held, and a stratified 12-URL live-Inspection sample agreed 12/12 with stored verdicts.

Crons

• Daily GSC sync — settings.cronGscSync (default 0 5 * * *, staggered per workspace) → runDailyGscSync. Skipped if the cron expression is empty.
• Operator trigger gscSync ("GSC sync (daily snapshot)").

Agent skills

None (the content inventory tools in Collections read the snapshots this module writes).

Data

Writes gsc_daily_totals (site-level daily series) and gsc_query_daily (per-query daily series) — both backfilled 16 months and never pruned, because past 16 months Google has deleted its own copy — plus blog_seo_snapshots, site_page_snapshots (both with 7d/28d/90d windows) + index columns onto site_pages/blog_articles; reads blog_articles (the ownership side of the fetch partition), site_pages, workspaces. Never inserts into blog_articles (discovery removed 2026-08-16; the only remaining write is cron-path URL adoption). RPCs: gsc_query_rollup and gsc_dimension_rollup (impression-weighted period totals — THE definition of a period's position, in SQL), latest_page_snapshots (recreated by the 90d migration to return the new columns), latest_page_snapshots_for, latest_blog_snapshots.

Key files

File	Role
index.js	Manifest (routeAlias/routeAliases, schedule, operatorCrons)
daily-sync.js	runDailyGscSync orchestrator (fetch → snapshots → re-rank → inspection → retention)
services/dashboard.js	The 7-query aggregation + resolution + content buckets
index-status.js	URL Inspection fallback (the four lanes) + deriveIndexed + deriveNoindex + the stale-verdict reconciler
inspection-write.js	buildInspectionUpdate / applyInspection — the ONE column shape a verdict lands in, shared by the nightly sweep and the on-demand recheck so the two paths cannot drift (site_pages also carries the crawl forensics; blog_articles has no column for them)
services/recheck.js	recheckIndexStatus — one URL, asked now. The door out of the quota-bounded nightly wait when a customer has just fixed a page
gsc-fetch.js	getWorkspaceGscData, in-flight dedupe (one shared fetch feeds both snapshot halves); returns the proven property so a second query this run skips the candidate walk
daily-totals.js	syncDailyTotals (16-month backfill + nightly refresh) and the reads getSiteSeries / getSeriesBounds
daily-queries.js	syncQueryDaily + rollupQueries / getQuerySeries / getQueryBounds — the uncapped query store
daily-dimensions.js	syncDimensionDailies (surface/country/device into gsc_daily_totals) + getDimensionRollup / getDimensionBounds
services/breakdown.js	getSearchBreakdown — all six dimensions, one shape, for the tool and the P6 page
brand.js	brandTermsFrom / isBrandedQuery — the mechanical branded split (pure, asserted)
query-flags.js	QUERY_FLAGS / queryHasFlag + the near-ranking band — the ONE backend definition of the query table's chip thresholds (duplicated client-side in QueryTable.tsx)
agent-tools/	shared.js (re-exports the platform window vocabulary from core/utils/window.js — WINDOW_PARAMS + windowOpts + RANGES — plus NOT_CONNECTED_MSG; empty series come back as { empty, reason, message } and the query list uses the shared { total, offset, returned, truncated } envelope, 2026-08-23), search-queries.js (get_search_queries), search-breakdown.js (get_search_breakdown); agent-tools.js keeps get_performance_history + the barrel
article-metrics.js / site-page-metrics.js	Snapshot refresh for the two halves. Both write position NULL (never 0) when Google returns no ranking — site-page snapshots used ?? 0 until 2026-08-05 and minted impossible position-0 rows
search-console.js	GSC client (read-only scope), property candidates + resolveGscSite, listAccessibleProperties, buildPathFilter, queryAnalytics, inspectUrl
services/connection.js	testGscConnection + accessDeniedMessage — the connection test, as the workspace's own credentials