https://blog.cloudflare.com/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives/

Perplexity is using stealth, undeclared crawlers to evade website no-crawl directivesPerplexity is repeatedly modifying their user agent and changing IPs and ASNs to hide their crawling activity, in direct conflict with explicit no-crawl preferences expressed by websites.blog.cloudflare.com




์ฑ—์ง€ํ”ผํ‹ฐ ๊ฒ€์ƒ‰ ๊ตฌ๋ฆฐ์ด์œ ๋„ ์žˆ๋Š”๋“ฏ