Description
Path disclosure in robots.txt occurs when sensitive paths or directories are inadvertently exposed in the robots.txt file. The robots.txt file is used to instruct web robots on which parts of a website to avoid crawling or indexing. While intended for cooperation with search engine crawlers, disclosing sensitive paths can provide malicious actors with valuable information that could be exploited for unauthorized access or to identify potential attack vectors.
Test for Robotxt.txt Vulnerabilities with our online tool!
Try Robots.txt ValidatorRecommendation
To mitigate the risk of path disclosure in robots.txt, carefully review and sanitize the contents of the file to avoid revealing sensitive paths or directories. Ensure that only necessary and safe paths are included in the robots.txt file, and regularly review and update it to remove any inadvertently disclosed information.
References
- Is your robots.txt file vulnerable? Here’s how to check and secure it
- Wikipedia: Robots exclusion standard
- CWE-200
- CAPEC-118
- OWASP 2021-A5
Related Issues
- Hidden Resource in Robots.txt - Vulnerability
- Robots.txt Found - Vulnerability
- Unix Path Disclosure - Vulnerability
- Windows Path Disclosure - Vulnerability
- Tags:
- Information Disclosure
- Robots Exclusion Standard