spaCy/website/robots.txt.jade
2016-10-03 20:19:13 +02:00

8 lines
164 B
Plaintext

//- ----------------------------------
//- 💥 ROBOTS.TXT
//- ----------------------------------
if environment != "deploy"
| User-agent: *
| Disallow: /