AadharshBot

A small, transparent crawler operated by aadhar.sh. If you see it in your access logs, this page tells you who it is, what it does, and how to stop it from visiting if you don't want it to.

Identity

User-Agent

AadharshBot/1.0 (+https://aadhar.sh/bot)

Signature-Agent

https://aadhar.sh/

JWKS

/.well-known/http-message-signatures-directory

Algorithm

sig1: Ed25519 (EdDSA), per RFC 9421 + Web Bot Auth draft

Operator

coffee@aadhar.sh

What it does

This is Aadharsh Pannirselvam's bot for aadhar.sh, running on Cloudflare Workers. The /around dashboard checks a small list of public homepages daily. The music and reading sections fetch public playlist metadata and bookmarks. /lens fetches public pages and discovery documents when a visitor asks to inspect a URL, and can read published MCP tool catalogues and NLWeb answers.

Content is used for linked references, metadata, and on-demand inspection. It is not used to train or fine-tune AI models or build a search index. Requests use bounded fan-outs and cached results. The bot does not log in to third-party sites or read content behind a login.

Lens also compares responses to sample browser and crawler User-Agent strings. Those diagnostic requests do not claim a Web Bot Auth identity for the sampled bot, but they still obey AadharshBot's robots.txt policy.

How to verify it's really AadharshBot

The signed HTTP readers for /around, music, bookmarks and Lens include Signature-Agent, Signature-Input, and Signature headers per RFC 9421 with the Web Bot Auth profile (tag="web-bot-auth"). Fetch the JWKS at the URL above, find the key whose kid matches (each kid is its key's RFC 7638 thumbprint), and verify the Ed25519 signature over the canonical components listed in Signature-Input. Treat an absent, expired or invalid signature as unverified. A User-Agent string alone does not prove who sent a request.

Browser rendering, webmention delivery and verification, and ancillary service calls can also use this User-Agent but do not currently carry Web Bot Auth signatures. They are outside the signed identity described here.

The second signature

Requests that fetch a page also carry a second label, sig2, a post-quantum ML-DSA-44 signature over the same covered components. Verify sig1. A verifier that only knows Ed25519 reads sig1 and skips the label it does not recognise, which is what makes a second one safe to send.

Probes carry sig1 alone: robots.txt reads, and the 28 discovery checks a /lens scan makes for well-known files. Signing all of them took a scan from 22ms to 32ms of CPU in production, so the post-quantum label rides on the requests that are this crawler's identity and skips the ones that only look around.

Its alg token, ml-dsa-44, is this site's spelling, because the IANA HTTP Signature Algorithms registry has no post-quantum entry yet. Its public key sits in the same JWKS as an AKP key (RFC 9964), and its keyid is that key's thumbprint, the same rule sig1 follows. The directory response carries both signatures too, one per key.

This is its second run. The first, from 2026-07-27 to 2026-08-15, signed in pure JavaScript, because Cloudflare's runtime had no ML-DSA in WebCrypto. That cost roughly 8.5ms a signature against a 10ms per-invocation budget, so the playlist scrape and the /lens discovery pass failed, since each signs many times per request. Cloudflare has since added ML-DSA to WebCrypto on Workers, where it signs in about 0.12ms. If that ever stops working, the label drops and sig1 ships alone. /garage/pqc has the measurements and the full argument.

How to opt out

The RN music panel has one operator-configured exception: its public Spotify playlist, track, and artist embed requests skip robots.txt. They retain the signed bot identity, bounded batches, and caching. Other readers, including Lens requests to Spotify, follow the policy below.

Add this to your robots.txt:

User-agent: AadharshBot
Disallow: /

Outside that exception, the signed HTTP readers read the origin's robots.txt (cached for up to 12 hours). They skip paths disallowed for AadharshBot or *, including redirect destinations. If the policy is unreachable, rate-limited, or too large to read safely, the bot skips the fetch. A positive Crawl-delay also makes this bot skip the origin. These rules apply to scheduled crawls and visitor-requested Lens HTTP reads. If you have a question or a complaint, email coffee@aadhar.sh and I'll reply by hand.