Drunk & Root@sh.itjust.works to Selfhosted@lemmy.worldEnglish · 2 days agoHow to combat large amounts of Ai scrapersmessage-squaremessage-square52fedilinkarrow-up1115arrow-down14file-text
arrow-up1111arrow-down1message-squareHow to combat large amounts of Ai scrapersDrunk & Root@sh.itjust.works to Selfhosted@lemmy.worldEnglish · 2 days agomessage-square52fedilinkfile-text
everytime i check nginx logs its more scrapers then i can count and i could not find any good open source solutions
minus-squarekrakenfury@lemmy.sdf.orglinkfedilinkEnglisharrow-up2arrow-down1·5 hours agoIs ignoring robots.txt considered “honest”?
minus-squareSheldan@lemmy.worldlinkfedilinkEnglisharrow-up1·3 hours agoThat’s not what I was talking about
Is ignoring robots.txt considered “honest”?
That’s not what I was talking about