"I did a search" is often a lie — a live GPT-5 run fails all five sites, then lands all five
Your Agent's Biggest Lie: "I Searched the Web" — Rafael Levi, Bright Data · Rafael Levi
16 min total·Actually worth watching closely: ~3 min·2 must-watch clips
- 0:15 – 2:40Listen
Failure with no error: the model would rather make it up than admit defeat
Opens with the core chain: when an agent is stopped on the way to a page it doesn't throw and doesn't warn — because it's trained to please the user, it falls back on old training data and passes it off as live.
Web access failure is an invisible failure route — what you see isn't an error, it's a confidently worded wrong answer.
He's standing and talking, the screen basically just a few lines of title text, so it's all in what he says — you lose nothing listening on the commute.▶ Jump to 0:15Speaker · Rafael Levi - 2:40 – 4:17Listen
Blocking is infrastructure now, and some traps serve fake data
Lifts the problem from anecdote to infrastructure: Cloudflare blocks AI crawling on about a fifth of the web by default, and the new AI Labyrinth doesn't even block — it feeds crawlers fake data outright. Then he sets up the head-to-head demo that follows.
Blocking has evolved from "refusing you" to "deceiving you" — though judging by petabyte-a-day collection, no degradation in actual data quality has shown up yet.
Still pure spoken argument; the numbers and conclusions all come out of his mouth, with no matching chart on screen.▶ Jump to 2:40Speaker · Rafael Levi - 4:17 – 5:45Watch
Head-to-head, first half: five sites, zero success
He opens a GPT-5 with no added capabilities at all and sends one prompt at Rightmove, LinkedIn, Instagram, Amazon and TikTok in turn; on screen all five come back empty.
On sites with serious anti-bot defenses, a bare model's success rate is zero — not occasionally, across the board.
This is a live run: the model's output, the wording of each failure, the march through five attempts all scroll on screen — audio alone misses the key part, how it acts as though nothing went wrong.▶ Jump to 4:17Speaker · Rafael Levi - 5:45 – 7:26Watch
Head-to-head, second half: everything lands once scraping is connected
He skims the list of ready-made endpoints for the major sites, reruns the identical prompt, and this time all five sites return real data — then has the model compare the two runs itself.
The most direct fix for this kind of hallucination isn't prompt tuning, it's making sure the agent isn't locked out at the door.
The endpoint list and the five successful returns are all screen content; the model's self-comparison in particular is worth reading line by line — on audio it's just "as you can see, they all worked".▶ Jump to 5:45Speaker · Rafael Levi - 7:34 – 10:43Listen
The public-data-only legal line, and IP quality
Draws the boundary: collecting behind a login is treated as not legal because you accepted the terms when you signed up, and that's the root of all the scraping lawsuits; a page an incognito window opens directly is public data that can legally be collected. He also covers how much IP quality decides success.
To judge whether data can be collected, ask one question: does an incognito window open it? A low-quality data center IP will likely be blocked, and a home IP is good for five to ten profiles.
For a few seconds at the start he actually opens an incognito window on a LinkedIn page as proof — worth a glance; the remaining three minutes are all spoken argument.▶ Jump to 7:34Speaker · Rafael Levi - 10:49 – 14:04Listen
Look more human, plus three token-saving practices
The anti-blocking approach isn't reverse-engineering detection but making access as human as possible — pre-recorded mouse tracks, realistic typing — so the challenge never appears. He also names something harder than blocking: some sites return three different prices for phone, computer and proxy.
Three practices you can use immediately: for bulk parsing, have the model write the parser and let a script run it, saving about 99% of tokens; mount tools on demand — if you use two, don't stuff all sixty-odd into context; pull back markdown with the tags stripped rather than making the model chew raw HTML.
All spoken experience, with no code samples or data charts on screen to follow — the densest stretch of the talk and the one most worth replaying with headphones.▶ Jump to 10:49Speaker · Rafael Levi - 14:12 – 15:33Listen
Closing Q&A: what real search results are worth
Audience questions, answered around the value of getting real search results versus the model's own retrieval, and where the free tier fits.
Not much by way of conclusions — mostly a bit of input for deciding whether to pay for real retrieval.
Question and answer, camera mostly on the person, nothing on screen — skippable if you're short on time.▶ Jump to 14:12Speaker · Rafael Levi