molave@reddthat.com to Selfhosted@lemmy.worldEnglish · 20 hours agoRunning web sites makes me not want to - Lowkey Partylowkey.partyexternal-linkmessage-square27linkfedilinkarrow-up1104arrow-down14
arrow-up1100arrow-down1external-linkRunning web sites makes me not want to - Lowkey Partylowkey.partymolave@reddthat.com to Selfhosted@lemmy.worldEnglish · 20 hours agomessage-square27linkfedilink
minus-squareMonkderVierte@lemmy.ziplinkfedilinkEnglisharrow-up7·edit-26 hours agoYou can set a trap that isn’t in robots.txt or is forbidden there. Anyone who ignores your robots.txt is not friendly.
You can set a trap that isn’t in robots.txt or is forbidden there. Anyone who ignores your robots.txt is not friendly.