1
1
mirror of https://github.com/explosion/spaCy.git synced 2025-03-14 23:22:19 +03:00
spaCy/website/robots.txt.jade
2017-06-05 20:07:52 +02:00

10 lines
142 B
Plaintext

//- 💫 ROBOTS.TXT
if environment != "deploy"
| User-agent: *
| Disallow: /
if ALPHA
| User-agent: Googlebot
| Disallow: /