#4566 (Add a default robots.txt to all sphinx sites)
by Luc Saffre · in Developers · assigned to Luc Saffre
Summary
This should doable by adding an [url www.sphinx-doc.org/en/master/usage/configuration.html#confval-html_extra_path html_extra_path] variable (which points to a directory in atelier containing the file itself) in the conf.py
Children
Comments
- 4 years, 3 months ago by Luc Saffre about #4566 (Add a default robots.txt to all sphinx sites) : Seems that we don't want this. When there is no robots.txt, crawlers will simply index all files. Which is what we want.