# Competitor Scout 4.0.1

Chrome/Edge extension: Google result URLs → Moz DA → registration age → Ahrefs DR → organic traffic. DR runs only for sites passing DA and age. Verified values stay attached to the exact domain; unread values remain unknown.

## Install or update

1. Download and extract Competitor-Scout.zip into a permanent folder.
2. Open `chrome://extensions` (Edge: `edge://extensions`) and enable Developer mode.
3. Existing installation: export a memory backup, stop the current run, replace the files **in the same extension folder**, then click Reload. Keep the extension installed to preserve its identity and saved local data.
4. New installation: click Load unpacked and select the folder containing `manifest.json`.
5. Open Competitor Scout and check Settings. Use 1 Google search, 1 Bulk DA tab and Natural pacing.

The ZIP contains the complete extension. It does not include Keywords Everywhere, Buster, an Ahrefs account or provider credits.

## Companion extensions and browsers

This build supports desktop Chrome and Edge. Open Settings → Browser setup & companion extensions for install links in the same browser profile:

- [Keywords Everywhere – Keyword Tool](https://keywordseverywhere.com/): use the official publisher’s install link for Google DA widgets. Missing DA still uses Scout’s shared Bulk checker.
- Buster: Captcha Solver for Humans — [Chrome](https://chromewebstore.google.com/detail/buster-captcha-solver-for/mpbjkejclgfgadiemmefgebjfooflfhl) or [Edge](https://microsoftedge.microsoft.com/addons/detail/buster-captcha-solver-fo/admkpobhocmdideidcndkfaeffadipkc). Optional; use its own challenge controls manually when offered.
- Registration age needs no extra extension or key. Ahrefs DR uses your own API keys or its website fallback.

Companion extensions are installed separately from their publishers. Scout does not install or control them. Other browsers have not been verified for this build.

## Research workflow

- Enter one keyword per line and start with a small batch and one Google page.
- Google results are scanned together, then the page is traversed with overlapping viewports to catch lazy results in the middle. All distinct discovered organic URLs are retained, including multiple URLs for one domain. Google navigation, advertisements and provider purchase links are excluded. A normal page finishes as soon as its bottom is stable; a long page can scan for up to 60 seconds. A time-limited or disabled scroll scan reports incomplete coverage and offers page retry rather than claiming a complete page.
- Each domain shares its metrics across every matching URL and keyword. Previously saved sites with missing checks continue those checks. A newly visible Google DA updates saved DA; verified saved metrics can be reused.
- A readable domain-bound Google DA takes priority even when its result includes “purchase a package.” Missing DA from the same Google page is collected into the native Keywords Everywhere **Bulk** form. Different pages have separate batches. Select 10, 25, 50 or 100 domains per batch; even a single remaining domain uses Bulk.
- Scout reuses one owned DA checker tab. It submits the native form once per job, reads All result rows, waits for fresh provider results and matches DA by domain. Older results in the table are ignored. A table is indexed once per read cycle for the whole batch. Fresh individual scores can save while the provider still processes the remainder; unchanged historical scores wait for completed-job confirmation. Reader wakes/reinjection retain the submission claim instead of spending another lookup.
- A slow Chrome tab operation keeps its one pending operation. Rejected tab operations retry with backoff, then show a DA connection warning. Domains remain pending; **Retry DA connection** resumes the shared checker without turning browser errors into per-site review failures.
- DA quota and rate-limit failures keep unread domains pending until the provider cooldown. Retry does not override that cooldown. For provider verification, complete it in the retained checker tab, then retry the DA connection. Previously captured scores are retained.
- A verified DA below the threshold proceeds to age. A verified age below the limit proceeds to DR. Add your **Ahrefs DR API keys** in Settings: the official API is tried first. Temporary errors and timeouts retry through the API before website fallback. The API gets at least three recovery attempts (or your larger configured budget); the website retains its own result-attempt budget. Website fallback is used when no usable key remains or recovery keeps failing. The displayed source identifies API or website. Keys are saved privately in this browser profile and excluded from exports and memory backups.
- Keywords Everywhere has no API-key setup section in Scout. DA uses its Google widgets and public website Bulk checker. Sign in directly on the provider website if your account requires it.
- Unread domain results retry within the configured attempt budget. Persistent no-data or unreadable checks remain **Needs review**; they cannot qualify a site. Connection delays and provider-wide limits remain pending instead.
- Use **All saved sites → All sites CSV / All sites Excel** for every saved run and status, including Qualified, Needs review, rejected and pending sites. This export ignores dashboard filters and includes captured URLs, metrics, evidence and sources. Qualified-only export remains available. Finished snapshots stay independent; deliberate later corrections remain exportable after a new run starts. Export a memory backup when finished. Pause/Continue retains research; Stop ends the run without deleting saved data.

## Version 4 defaults and large batches

New research uses built-in public RDAP registration APIs for age, API-first DR, and deferred traffic. No age key is required. Only domains passing DA and age reach DR. DR-qualified sites can finish with traffic blank; choose Continue traffic when you want that later stage. Existing active runs retain their chosen age and traffic modes; future defaults can be changed in Settings.

Registry requests use the IANA-listed authority for each domain suffix. Where IANA lists alternate HTTPS endpoints, connection, timeout and server failures can try the next listed endpoint. The bundled data currently lists one endpoint per service, so a second API is not available for every domain. Registry quotas stay pending until their cooldown ends without consuming a failed domain attempt. Public services still have their own availability and limits.

Repeated runs apply their currently selected DR/traffic requirements to saved sites. Recent verified DR and traffic can be reused for 7 days, and registration lookups for 30 days; expired or undated saved values are refreshed before qualification. Original finished-run snapshots remain exportable.

Routine reader updates and dashboard polling avoid copying full archived histories. Captures and current assignments are saved immediately; archive copies use a 15-second checkpoint, with forced checkpoints on pause, stop, new runs, completed runs and export. Changed result rows are updated without rebuilding every dashboard row.

Ahrefs website fallback chooses the visible, ready domain field associated with the real check control, verifies the entered domain before submission, and waits for a fresh domain-bound result. It rejects nested foreign-domain numbers and previous-job values. Extended verification deadlines are honored, and explicit current provider blocks are recognized even while old numeric DOM remains.

## Multiple Ahrefs API keys

Open Settings → Ahrefs DR API. Add a label and key, or paste multiple lines (`label | key` or one key per line). The extension imposes no fixed number of saved keys; this does not provide unlimited provider credits or requests. Only use keys you are authorized to use.

- Green **Working** means a real API request succeeded. A new untested key is gray.
- Amber **Limited** means waiting for a provider allowance/cooldown; **Retrying** means a temporary error. Expired limits become ready for a new request, not falsely verified green.
- Red **Rejected** means authentication/permission failed. Test again after restoring access, or replace the key. It stays red until a real request succeeds.
- Disable, enable, remove or explicitly test each key. Test sends one request; ordinary research also updates status. During active research, failed keys receive bounded asynchronous recovery tests: at most one background probe per minute, with failed probes backing off for 2, 4, 8 and up to 15 minutes. Provider cooldowns take priority. Healthy and untested keys are not repeatedly probed. Disabled keys remain disabled. The interface reads stored status without spending provider lookups.

A failed/disabled credential can hand work to another available key. An explicitly credential-scoped quota waits on that key while another authorized credential can serve the queue. An unknown, account-wide or IP-wide limit pauses all API keys until the provider cooldown ends; adding/removing keys does not override that hold. Repeated rate limits keep sites pending instead of exhausting domain attempts. Ahrefs website usage limits also keep browser checks pending. DR still runs only after DA and age pass.

When a recovery test succeeds, future queued DR checks return to the API while an active website check continues. Retry unresolved starts API-first recovery for missing DR while preserving verified DA, age and traffic. Browser DR/traffic readers reject stale numbers and old errors from the previous domain, including late numeric/heading renders and valid zero values.

CSV retains long history fields. Excel has a 32,767-character cell limit; if a history cell exceeds that limit, use CSV or the memory backup instead of truncating it.

## Google verification and Buster

Natural pacing spaces new Google navigations by at least 10 seconds, or a longer configured delay. This reduces request frequency but cannot guarantee that Google will never show CAPTCHA. When verification appears, new Google navigation pauses across all workers while existing site checks can continue.

Open the retained Google verification tab and click “I'm not a robot” yourself. If Buster is installed, use its button inside the challenge when available. Buster does not expose a supported external solve command; Scout cannot trigger it with a synthetic click. Complete Google's requested step, then return to the matching search page and use the resume control if needed. An “inactive” Buster service worker is normal between events. Scout does not bypass provider CAPTCHA, quotas or subscription restrictions.

## Development

No npm dependencies or build step. Node.js 24, Python 3 and Chromium are used for validation:

```sh
cd /workspace/competitor-scout-extension
npm test
npm run validate
npm run smoke:browser
```

Tests cover Chrome worker orchestration, DOM readers, workflow, API routing, exports and package integrity. Browser smoke runs local HTTP fixtures, including native Bulk forms, Google markup and 12 successive jobs in each Ahrefs DR/traffic fixture, without external lookups. Set CHROMIUM_BIN if needed. Its --no-sandbox flag applies only to this isolated test process. See VALIDATION.txt for current counts and live-browser limitations; historical changelog entries describe earlier versions.
