From discovery to a report

A conceptual process, not a claim that every provider uses the same implementation. Open the cases to distinguish missing data, old evidence and a failed recheck.

Authored fictional examples on reserved domains. No provider was contacted and no current link was verified.

  1. Discovery

    Input: Candidate URLs and known references.

    Output: A queue of pages to attempt.

    What remains uncertain: An undiscovered page can contain a real link.

  2. Access

    Input: A queued URL and crawl permissions.

    Output: A response, redirect or failed request.

    What remains uncertain: A failed fetch does not prove removal.

  3. Extraction

    Input: The accessible representation of a page.

    Output: Reported source-target references and attributes.

    What remains uncertain: Rendering support and extraction rules differ.

  4. Normalization

    Input: Extracted URLs and repeated observations.

    Output: Provider-specific grouping or deduplication.

    What remains uncertain: Do not assume providers share a counting unit.

  5. Storage and rechecking

    Input: An observation with its timestamp.

    Output: Historical records and later observations.

    What remains uncertain: The latest successful observation can be old.

  6. Reporting and export

    Input: Stored records plus scope and filters.

    Output: A selected table, sample or downloadable file.

    What remains uncertain: The export is not a complete current map of the web.

Three evidence cases

No observation

The hypothetical index has no observation for this source. The real page has not been checked here.

Source URL
https://publisher.example/unobserved
Target URL
https://target.example/resource
Last successful observation
Unknown / not checked
Recheck attempt
Unknown / not checked
HTTP result
Unknown / not checked
Current presence
Unknown / not checked

Next check: Investigate discovery and coverage before saying the link is absent.

Old observation

The example was observed on 10 January and exported on 14 January. No new successful check is recorded.

Source URL
https://publisher.example/stale
Target URL
https://target.example/resource
Last successful observation
2026-01-10T09:00:00Z
Recheck attempt
Unknown / not checked
HTTP result
Unknown / not checked
Current presence
Unknown / not checked

Next check: Keep the observation date; a fresh export does not prove current presence.

Failed recheck

After an earlier observation, the example recheck returns HTTP 503. The response does not establish whether the reference was removed.

Source URL
https://publisher.example/failed
Target URL
https://target.example/resource
Last successful observation
2026-01-10T09:00:00Z
Recheck attempt
2026-01-14T09:00:00Z
HTTP result
503
Current presence
Unknown / not checked

Next check: Retry through the normal authorized workflow and keep the prior evidence separately.