Popularity
6.8
Stable
Activity
0.0
Stable
12
4
6

Monthly Downloads: 41
Programming language: DIGITAL Command Language
License: BSD 3-clause "New" or "Revised" License
Tags: Web    

robots-txt alternatives and similar packages

Based on the "Web" category.
Alternatively, view robots-txt alternatives based on common mentions on social networks and blogs.

Do you think we are missing an alternative of robots-txt or a related project?

Add another 'Web' Package

README

robots-txt

[Alt logo](logo.png)

This is a simple package for parsing robots.txt files. Based on http://www.robotstxt.org/orig.html

Build Status