|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Objectorg.archive.crawler.frontier.WorkQueue
org.archive.crawler.frontier.BdbWorkQueue
public class BdbWorkQueue
One independent queue of items with the same 'classKey' (eg host).
| Field Summary |
|---|
| Fields inherited from class org.archive.crawler.frontier.WorkQueue |
|---|
classKey, substats |
| Constructor Summary | |
|---|---|
BdbWorkQueue(java.lang.String classKey,
BdbFrontier frontier)
Create a virtual queue inside the given BdbMultipleWorkQueues |
|
| Method Summary | |
|---|---|
protected void |
deleteItem(WorkQueueFrontier frontier,
CrawlURI peekItem)
Removes the given item from the queue. |
protected long |
deleteMatchingFromQueue(WorkQueueFrontier frontier,
java.lang.String match)
Delete URIs matching the given pattern from this queue. |
protected static java.lang.String |
getPrefixClassKey(byte[] byteArray)
|
protected void |
insertItem(WorkQueueFrontier frontier,
CrawlURI curi,
boolean overwriteIfPresent)
Insert the given curi, whether it is already present or not. |
protected CrawlURI |
peekItem(WorkQueueFrontier frontier)
Returns first item from queue (does not delete) |
| Methods inherited from class org.archive.crawler.frontier.WorkQueue |
|---|
clearHeld, compareTo, deleteMatching, dequeue, enqueue, expend, getClassKey, getContextUURI, getCount, getPendingExpenditure, getReports, getSessionBalance, getSubstats, getTotalBudget, getTotalExpenditure, getWakeTime, incrementSessionBalance, isHeld, isOverBudget, isRetired, noteError, peek, refund, reportTo, reportTo, resume, setActive, setHeld, setRetired, setSessionBalance, setTotalBudget, setWakeTime, singleLineLegend, singleLineReport, singleLineReportTo, suspend, unpeek, update |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Methods inherited from interface java.lang.Comparable |
|---|
compareTo |
| Constructor Detail |
|---|
public BdbWorkQueue(java.lang.String classKey,
BdbFrontier frontier)
classKey - | Method Detail |
|---|
protected long deleteMatchingFromQueue(WorkQueueFrontier frontier,
java.lang.String match)
throws java.io.IOException
WorkQueue
deleteMatchingFromQueue in class WorkQueuefrontier - WorkQueues manager.match - the pattern to match
java.io.IOException - if there was a problem while deleting
protected void deleteItem(WorkQueueFrontier frontier,
CrawlURI peekItem)
throws java.io.IOException
WorkQueue
deleteItem in class WorkQueuefrontier - Work queues manager.
java.io.IOException - if there was a problem while deleting the item
protected CrawlURI peekItem(WorkQueueFrontier frontier)
throws java.io.IOException
WorkQueue
peekItem in class WorkQueuejava.io.IOException - if there was a problem while peeking
protected void insertItem(WorkQueueFrontier frontier,
CrawlURI curi,
boolean overwriteIfPresent)
throws java.io.IOException
WorkQueue
insertItem in class WorkQueuefrontier - WorkQueueFrontier.curi - CrawlURI to insert.
java.io.IOException - if there was a problem while inserting the itemprotected static java.lang.String getPrefixClassKey(byte[] byteArray)
byteArray - Byte array to get hex string of.
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||