skorotkiewicz/rDrama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

dfs

Browse source
This commit is contained in:
Aevann1 2021-09-24 21:53:39 +02:00
parent 65c67aa7f4
commit 53e91de4db
1 changed files with 16 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

16
files/assets/robots.txt
View file

@ -1,2 +1,18 @@
User-Agent: *
Disallow: /admin/*
Disallow: /submit
Disallow: /settings
Disallow: /me
User-Agent: DotBot
User-Agent: TurnitinBot
User-Agent: Huaweisymantecspider
User-Agent: SiteSnagger
User-Agent: TeleportPro
User-Agent: Xaldon_WebSpider
User-Agent: MJ12bot
User-Agent: SEMrushBot
User-Agent: AhrefsBot
User-Agent: CCBot
User-agent: SBSearch
Disallow: /