spaCy/website/robots.txt.jade

mirror of https://github.com/explosion/spaCy.git synced 2024-11-11 04:08:09 +03:00
//- 💫 ROBOTS.TXT

if environment != "deploy"
    | User-agent: *
    | Disallow: /

if ALPHA
    | User-agent: Googlebot
    | Disallow: /
Update to new website 2016-10-31 21:04:15 +03:00			`//- 💫 ROBOTS.TXT`
Update website 2016-10-03 21:19:13 +03:00
Fix robots and meta 2017-06-05 21:07:52 +03:00			`if environment != "deploy"`
Update website 2016-10-03 21:19:13 +03:00			`\| User-agent: *`
			`\| Disallow: /`
Fix robots and meta 2017-06-05 21:07:52 +03:00
			`if ALPHA`
			`\| User-agent: Googlebot`
			`\| Disallow: /`