How a Website Decides You're a Bot

You land on a page, you have not clicked anything, you have not moved the mouse, and a blank panel has already decided it is not sure you are human. It reads like a magic trick. The server seems to know something about you before you have done a single thing.

I went looking for how the commercial anti-bot systems actually work, from the defender's side, and I came in with one narrow question. Nearly all the good ones belong to a handful of companies operating at a scale no small team can match. So how much of that machine can you build yourself, and how much is scale you will simply never have?

The short answer is that there is no magic and no single trick. Every serious system is the same layered idea: passive network signals at the edge, an obfuscated agent that probes your browser, server-side scoring that ties it all together, and a token that has to be re-earned. No layer is trusted on its own. The power is in correlation, in catching a client whose layers disagree with each other. And correlation, it turns out, is the one part a small team can do well.

No single signal#

Pick any one signal and it is beatable on its own. A User-Agent is a string the client chooses. navigator.webdriver is one flag you can patch out in a line. A TLS fingerprint can be forged by a library built to impersonate a browser. If a system leaned on any single check, bypassing it would be a weekend project, and for a lot of older defences it was.

So nobody leans on one. The state of the art is defence in depth: several independent layers, each collecting different evidence, and a scoring step whose real job is to notice when the layers tell different stories.

The rest of this is a walk down that pipeline, layer by layer, and then the honest accounting of which layers you can build and which you have to rent.

Before your JavaScript runs#

The first evaluation happens before the page sends you a single byte of script. When your client opens the connection, the TLS ClientHello already carries a fingerprint: the exact list of cipher suites, extensions, curves, and their arrangement. JA3 hashed those fields directly, and its weakness was that browsers started randomising extension order to avoid ossification, which made the hash unstable. JA4, FoxIO's successor, sorts the lists before hashing and strips GREASE, so reordering no longer changes the result. A JA4 like t13d1516h2_8daaf6152771_b186095e22b6 packs the transport, TLS version, cipher and extension counts, ALPN, and hashes of the sorted lists into one readable string.

The reason TLS matters so much is integrity. JavaScript cannot touch the handshake. It happens in the native TLS stack, below anything a page script can reach. The same goes for the HTTP/2 fingerprint (pseudo-header order, SETTINGS values, frame ordering) and the raw TCP options. Stack that with IP and ASN reputation, where a datacenter address is a strong negative and a mobile carrier a strong positive, and the edge already has an opinion before running a line of code.

This layer is the highest integrity and the lowest cost, and it should be the first thing you build. Cloudflare added cross-request JA4 signals in 2024, and DataDome documents how TLS fingerprinting feeds its model. A well-tuned JA4 plus IP reputation catches the majority of naive automation before you have written any client code at all.

The agent they ship to your browser#

Once the page loads, it runs an agent whose job is to collect everything the network layer cannot see. This is the DataDome tag, the HUMAN sensor, Akamai's bmak, Imperva's reese84, Kasada's p.js. The striking thing about these scripts is how hard they work to be unreadable.

Kasada is the clearest public example. Reverse engineers have reconstructed its client as a custom bytecode virtual machine, roughly 449 KB, wrapped around a 437 KB encoded string table with a rotating-alphabet decoder, on top of control-flow flattening and dead-code injection. The point of the VM is easy to miss: it does not detect anything. It is an anti-analysis moat. Because the real challenge logic is data (bytecode for a bespoke instruction set) rather than code, ordinary debugging, breakpoints, and AST tools show you the interpreter, not the logic. To read the logic you first have to reverse the VM, and the vendor can rotate the bytecode server-side whenever they like. Write-ups like Kernel's and tools like ips-disassembler exist precisely because that is a lot of work.

What the VM executes is a fingerprinting sweep. According to that same Kernel write-up, Kasada uses "crash-and-capture" probes: it deliberately triggers TypeErrors and inspects not whether they fail but how they fail, the exact message text and stack shape that a particular browser engine produces. It runs iframe probes doing binary searches over screen dimensions and media features. Then it serializes the result, encrypts it with a repeating key that was sitting in the shipped code, and posts it home. That last detail is the whole lesson about client-side crypto, and I will come back to it.

Three kinds of signal, one contradiction#

Everything a client collects falls into three buckets. Passive network signals, the ones from before your JavaScript runs. Active browser fingerprints: canvas and WebGL rendering hashes, AudioContext output, fonts, codecs, the whole navigator object, screen and timezone, WebRTC. And behavioural biometrics: mouse curvature and micro-corrections, keystroke cadence, scroll physics, touch, scored across the whole session.

There is a technique that ties the environment fingerprints back to a hard truth, and it is my favourite piece of the whole field. DataDome's Picasso (originally a Google idea) renders a sequence of canvas and GPU operations and hashes the pixels. The output is stable per device class, that is browser times OS times GPU, because it falls out of real graphics hardware and drivers. A headless browser on a software renderer produces a hash that matches no real GPU. So Picasso is not really identifying you, it is checking whether your machine is telling the truth about what it is.

That is the linchpin of the entire architecture: cross-signal consistency. Any one layer can be forged. Keeping every layer's story consistent at the same time is much harder.

Automation frameworks leak here constantly: navigator.webdriver, Selenium's cdc_ properties, the HeadlessChrome marker, stealth-plugin signatures, and the tampering they do to hide, like a patched Function.prototype.toString that no longer stringifies as native code. Some of these signals decay fast. Castle documented in 2025 that a V8 change quietly killed a classic Chrome DevTools Protocol trick that had relied on a getter firing when an error object was logged. Any defence resting on one clever trick is fragile. The consistency check is not, because it does not care which trick catches the lie, only that some layer does.

The verdict is computed server-side#

The client only collects and signs. It never decides. The verdict, allow or challenge or block, is computed on the server, and it starts with cheap deterministic rules before it reaches anything expensive.

Cloudflare's engines are a well-documented example. A heuristics engine runs first, matches requests against known-bad fingerprints in microseconds, and hands out a score of 1 for high-confidence hits. By Cloudflare's own account it labels around 15% of global traffic on its own, and those labels then train the machine-learning engine, which produces most of the 1 to 99 bot score. DataDome trains a separate model per customer site. reCAPTCHA v3 returns a 0.0 to 1.0 score with no interaction at all, where 1.0 is very likely human and the default cut is 0.5.

The multiplier none of them can live without is cross-customer threat intelligence. Google's reCAPTCHA draws on telemetry from trillions of transactions across millions of sites, and DataDome processes trillions of signals a day. An attacker burned on one site is flagged everywhere at once. This is the single component a small team cannot reproduce, and it is worth being honest about that up front.

Tokens that expire in seconds#

If the verdict is allow, the client gets a token, and the token design is where the cryptography actually lives. Kasada is again the clearest public shape: a reusable session token bound to your IP and TLS fingerprint and valid for around thirty minutes, plus a single-use, timestamped answer derived per request that must be only seconds old and is marked consumed server-side, with an HMAC binding the two so they cannot be mixed. Akamai's _abck, Imperva's reese84, and DataDome's cookie all follow the same pattern: a signed payload, bound to IP plus fingerprint, invalidated the moment it arrives from somewhere else.

Here is the lesson I promised. The key Kasada used to encrypt its telemetry was sitting in the shipped code, and researchers recovered it with a guess at the plaintext. Client-side encryption is obfuscation, not confidentiality, because the key always ships with it. Real integrity comes from the server holding the signing key. The client can be forced to produce a valid token, but it can never forge a server signature. Anything you build should put its trust there and nowhere on the client.

Make it expensive, or make it invisible#

Step back and the vendors split into two camps. One camp makes automation expensive: Kasada's proof-of-work puzzle, solved in a few milliseconds by a real browser but escalated hard for a fresh datacenter session, and Cloudflare Turnstile, which runs background proof-of-work and memory-hard proof-of-space. The other camp makes detection invisible and scores it: reCAPTCHA and per-site ML that never show a puzzle at all. The best systems now do both.

Proof of work deserves a special mention, because it is the one asymmetric weapon that needs no cross-customer data. You issue a server-seeded puzzle, make it trivial for one real browser and punishing for ten thousand headless ones, and verify the answer in microseconds. It is the single best equaliser for a small team.

What you can actually build#

Here is the split I came for. A small team can realistically build the parts that depend on your own traffic, and should rent the parts that depend on everyone's.

Here is the whole menu a top-tier vendor runs. The green items are the ones a small team can realistically ship on its own, and the empty ones need a scale you have to rent.

• Included: TLS fingerprinting (JA3 / JA4)
• Included: HTTP/2 and TCP stack fingerprinting
• Included: IP and ASN reputation
• Included: Canvas, WebGL and AudioContext fingerprints
• Included: navigator, screen and Client Hints collection
• Included: Automation-framework detection (webdriver, cdc_, CDP)
• Included: Behavioural biometrics (mouse, keystroke, scroll)
• Included: A cross-signal consistency check
• Included: Signed, short-lived, fingerprint-bound tokens
• Included: Proof-of-work with difficulty escalation
• Included: A rules-first scoring engine, then a small ML model
• Included: A challenge ladder, from invisible to CAPTCHA
• Not included: Per-site ML trained on billions of requests
• Not included: A cross-customer threat-intelligence graph
• Not included: A stable global device-ID graph
• Not included: A bytecode VM and constant obfuscation rotation

Build the edge layer first: JA4, the HTTP/2 fingerprint, IP and ASN reputation, evaluated on every request before you serve content. Add signed tokens next, with a server-held HMAC key, IP and fingerprint binding, and a nonce plus timestamp plus single-use record (Redis is enough) for replay protection, then bolt proof-of-work onto that with difficulty that rises for cold sessions. Then add the collection agent, and if you only ship one clever thing, make it a single cross-signal consistency check: does the TLS JA4 agree with the User-Agent, the Client Hints, and the canvas-derived OS? That one check catches the traffic that passes every individual layer. Only after all of that is it worth training an ML model, on labels your own heuristics and confirmed abuse have generated, and hardening the client's obfuscation, which buys time and never buys secrecy.

What you cannot build is the part that made the big vendors big: the global threat-intelligence graph, per-site models trained on billions of requests, a stable cross-site device identity, and the endless obfuscation-rotation arms race. For those, put a commercial layer in front. Turnstile is free, and Fingerprint has a free tier for device identity. Renting the scale you cannot build is not a compromise, it is the correct call.

Where it breaks#

None of this is clean, and the failure modes are worth naming.

False positives are the real cost. Privacy tools, VPNs, uncommon browsers, and assistive technology all score badly on behaviour and fingerprint checks, and a system tuned too tight punishes exactly the users you most want to keep. This is why the industry keeps drifting toward invisible scoring and proof of work and away from interactive puzzles, which are miserable for anyone using a screen reader. The agents are not free either: Akamai's sensor is around 512 KB, Kasada's around 449 KB, and proof of work spends real CPU on the client.

The evasion side is a live arms race. Residential and mobile proxies defeat IP reputation. Tools built to impersonate a real TLS stack, like curl-impersonate and uTLS, defeat JA-style fingerprints. Real-browser automation with a minimal control surface defeats most environment checks. Session-warming defeats proof-of-work escalation. The defences that actually hold are the structural ones: cross-signal consistency, GPU pixel hashing that a software renderer cannot fake, behavioural realism that does not survive a hundred concurrent sessions, and protocol-level leaks below the JavaScript layer.

Most of what is public about these systems, the 449 KB VM, the recovered key, the exact token headers, comes from reverse-engineering write-ups and from vendors who sell bypasses, not from official documentation. Treat the specific numbers as directional, not audited. The claims I trust most here are the ones a vendor published about their own system: DataDome on Picasso, Cloudflare on its engines, Castle on the CDP change, Google on reCAPTCHA.

And the legal dimension is real. Deep fingerprinting runs straight into GDPR consent and accessibility obligations, and it is jurisdiction-specific. That is a lawyer's question before it is an engineer's.

The part you can actually own#

The thing I keep coming back to is that the magic-trick feeling is the opposite of the truth. These systems are not smart in the way they look smart. They are not reading your soul from a blank page, they are stacking half a dozen cheap, independent measurements and waiting for you to contradict yourself. The intelligence is in the correlation, not in any one probe.

That is oddly encouraging if you are building rather than buying. The expensive, unbeatable-looking parts, the bytecode VM and the interactive puzzle, are the parts you should not try to copy. The cheap structural part, one honest cross-signal consistency check backed by proof of work and a server-signed token, is most of the value, and it is entirely within reach. You will never build the global brain that flags an attacker across a million sites. But you can build the thing that notices when a request claims to be Chrome while its own TLS handshake says otherwise. For most sites, most of the time, that is the check that pays for itself.

References

1. Kernel: Bot detection is dead, long live bot detection, reverse-engineering Kasada in an afternoon
2. umasii/ips-disassembler: a disassembler for Kasada's IPS.js VM
3. DataDome: The Art of Bot Detection, Picasso for device-class fingerprinting
4. Bursztein et al.: Picasso, Lightweight Device Class Fingerprinting for Web Clients (paper)
5. DataDome: how TLS fingerprinting reinforces DataDome's protection
6. DataDome: what is canvas fingerprinting
7. Castle: why a classic CDP bot-detection signal suddenly stopped working
8. Castle: from Puppeteer stealth to Nodriver, how anti-detect frameworks evolved
9. Cloudflare docs: bot detection engines
10. Cloudflare docs: bot scores
11. Cloudflare: Bot Management, machine learning and more
12. Cloudflare: improved Bot Management with high-precision heuristics
13. Cloudflare: advancing threat intelligence with JA4 signals
14. Cloudflare: Turnstile is free for everyone
15. FoxIO: JA4+ network fingerprinting suite (repo)
16. FoxIO: JA4+ network fingerprinting (blog)
17. Salesforce: JA3 TLS fingerprinting
18. Google: reCAPTCHA v3 developer docs
19. Fingerprint: identification accuracy explained
20. curl-impersonate: a curl that mimics browser TLS/HTTP fingerprints
21. uTLS: a Go TLS library for fine-grained ClientHello control