zijra.Documentation
Dashboard
Integrations

Verified AI crawlers

Coming soon: planned verified server crawler observations, separate from human visits.

Verified crawlers · Coming soon

Coming soon. Stripe is the live revenue integration. Existing saved records remain readable; this catalog does not establish a provider connection. New connections, authorization and sync are unavailable for this provider in this release.

Planned setup

The following describes the planned connection, once released.

  1. Step 1

    Configure a trusted server or proxy to send signed crawler request observations.

  2. Step 2

    Verify supported official IP ranges or reverse-and-forward DNS. A bot-like user agent alone is insufficient.

  3. Step 3

    Inspect verified provider, page and time reports. Unknown agents are not relabelled as verified crawlers.

Official Verified crawlers documentation

Set up Stripe

Install server observation

Browser tracking cannot see crawlers that never run JavaScript. Download the Node.js 20+ helper into your server code, alongside its TypeScript declaration. It captures only supported crawler hints on GET and HEAD requests; Zijra independently verifies every request against official IP ranges or forward-confirmed reverse DNS.

Download server helper TypeScript declaration

  1. Create the signed feed

    Open your website’s Integrations and create the verified crawler feed. Copy its exact public HTTPS endpoint and signing secret into server-only environment variables named ZIJRA_CRAWLER_ENDPOINT and ZIJRA_CRAWLER_SECRET. Use a deployment that accepts external requests; a protected preview cannot receive crawler reports. Feed ready means configuration is saved; the report remains empty until a verified request arrives.

    Open crawler settings

  2. Keep the original source address

    On a real Vercel deployment, the helper reads the platform’s overwritten x-vercel-forwarded-for header. It refuses local, private and malformed addresses. On another host, pass the client address from the trusted server socket or documented platform metadata. Arbitrary client headers cannot establish a crawler’s identity. An extra CDN may hide the original crawler address; unknown requests remain unverified.

  3. Connect your durable queue

    Implement the helper’s CrawlerOutbox contract using your existing server database and scheduler. Append the sealed job once, lease due jobs atomically, and schedule retry work in the same transaction. Each claim increments its persisted attempt count. Completion and deletion must match the lease token. Enforce a bounded queue, erase expired jobs and never store plaintext source IPs or user agents.

    The helper seals observations with AES-256-GCM before the queue sees them. Five delivery attempts and a 23-hour expiry bound retention. It sends the same event ID on replay, respects Retry-After, blocks redirects and never waits inside the incoming request. Flush outstanding jobs before rotating the signing secret, because a changed secret cannot decrypt older queued observations.

  4. Attach the Next.js proxy

    For Next.js 16, place the code in src/proxy.ts or the project-root proxy.ts, beside your app directory. Merge it with existing routing and authentication. The example uses UmmahSignal; replace its exact allowed hosts for another website. Register background work with event.waitUntil; without that support the helper refuses to start.

    Code exampleTypeScript
    // Next.js 16, src/proxy.ts. Merge with your existing proxy.
import { NextResponse } from 'next/server';
import type { NextFetchEvent, NextRequest } from 'next/server';
import { createCrawlerSender, vercelClientIp } from './lib/zijra-crawler-server.mjs';
import { crawlerOutbox } from './lib/crawler-outbox'; // Your durable server adapter

const sender = process.env.ZIJRA_CRAWLER_ENDPOINT && process.env.ZIJRA_CRAWLER_SECRET
  ? createCrawlerSender({
      endpoint: process.env.ZIJRA_CRAWLER_ENDPOINT,
      secret: process.env.ZIJRA_CRAWLER_SECRET,
      allowedHosts: ['ummahsignal.com', 'www.ummahsignal.com'],
      outbox: crawlerOutbox,
    })
  : null;

export function proxy(request: NextRequest, event: NextFetchEvent) {
  sender?.observe(request, {
    ip: vercelClientIp(request),
    waitUntil: promise => event.waitUntil(promise),
  });
  return NextResponse.next(); // Preserve existing auth/redirect behavior here.
}

// Keep discovery documents such as robots.txt, sitemap.xml and llms.txt included.
export const config = {
  matcher: '/((?!api|_next|favicon.ico).*)',
};
    Preparing code…

    Keep your existing runtime and matcher compatible with your installed Next.js version. Older applications may use middleware.ts. The Node helper needs a Node runtime; use a server action or backend integration when the existing middleware uses Edge.

  5. Run the worker and verify an actual receipt

    Code exampleJavaScript
    // Invoke from your durable scheduler; never from a browser or public route.
// The retry adapter must schedule the next invocation atomically.
const result = await sender.flush({ limit: 1 });
// result contains verified, ignored, retry and dropped counts, without IPs or UA.
    Preparing code…

    The durable scheduler must invoke the worker after enqueue and at each saved retry time, including when no new website requests arrive. Inspect delivery failures in your server’s safe diagnostics. A user-agent-only test is ignored; it cannot prove a crawler visit. In Zijra, verify provider, purpose, requested page and actual time after a real crawler is received. Requests and response status are separate: this proxy observes the requested path and does not claim that the page returned a particular status.

Supported identities

OpenAI, Perplexity and Anthropic use their published IP lists. Googlebot and Bingbot require matching reverse DNS and a forward lookup back to the same address. Unsupported or unverified agents stay out of the verified report. No visitor identity, consent cookie, query string or account data enters the feed.

Sources: Next.js background proxy work, Vercel source-address header, Anthropic crawler identities, OpenAI crawlers, Perplexity crawlers.

X mentions and attributionDodo Payments