Files
LingvAI/public/robots.txt
2021-03-21 16:34:26 +01:00

7 lines
69 B
Plaintext

User-Agent: *
Disallow: /
Allow: /$
User-Agent: Twitterbot
Allow: /