|
||||||||||
| PREV NEXT | FRAMES NO FRAMES | |||||||||
| Packages that use RobotsExclusionPolicy | |
|---|---|
| org.archive.crawler.datamodel | |
| Uses of RobotsExclusionPolicy in org.archive.crawler.datamodel |
|---|
| Fields in org.archive.crawler.datamodel declared as RobotsExclusionPolicy | |
|---|---|
static RobotsExclusionPolicy |
RobotsExclusionPolicy.ALLOWALL
|
static RobotsExclusionPolicy |
RobotsExclusionPolicy.DENYALL
|
| Methods in org.archive.crawler.datamodel that return RobotsExclusionPolicy | |
|---|---|
RobotsExclusionPolicy |
CrawlServer.getRobots()
Get the robots exclusion policy for this server. |
static RobotsExclusionPolicy |
RobotsExclusionPolicy.policyFor(CrawlerSettings settings,
java.io.BufferedReader reader,
RobotsHonoringPolicy honoringPolicy)
|
| Methods in org.archive.crawler.datamodel with parameters of type RobotsExclusionPolicy | |
|---|---|
void |
CrawlServer.setRobots(RobotsExclusionPolicy policy)
Set the robots exclusion policy for this server. |
|
||||||||||
| PREV NEXT | FRAMES NO FRAMES | |||||||||