I had to set up fail2ban + bot detection just to block the suckers. Its mostly amazon and openai for me and my small sites. Google/bing look at robots.text and respect it. Openai in particular slams any site to the ground quick…