Bluesky web client
Login
2024
@subium.com
v1.0.0-α.14
·
Report bug
·
Contact
Post
Mike Elgan
@mikeelgan.bsky.social
·
1 mo
Claim of the moment: Perplexity AI ignores robot.txt files and crawls websites even when the site owner says no.
rknight.me/blog/perplex...
Perplexity AI Is Lying about Their User Agent
rknight.me
Perplexity AI claims it sends a user agent and respects robots.txt but it absolutely does not
1
2