us.codecraft.webmagic.scheduler.component

Interface DuplicateRemover

All Known Implementing Classes:

BloomFilterDuplicateRemover, HashSetDuplicateRemover, RedisPriorityScheduler, RedisScheduler

public interface DuplicateRemover

Remove duplicate requests.

Since:

0.5.1

Author:

code4crafer@gmail.com

Method Summary

Modifier and Type	Method and Description
int	getTotalRequestsCount(Task task) Get TotalRequestsCount for monitor.
boolean	isDuplicate(Request request, Task task) Check whether the request is duplicate.
void	resetDuplicateCheck(Task task) Reset duplicate check.

Method Detail

isDuplicate

boolean isDuplicate(Request request,
                    Task task)

Check whether the request is duplicate.

Parameters:

request - request

task - task

Returns:

true if is duplicate

resetDuplicateCheck

void resetDuplicateCheck(Task task)

Reset duplicate check.

Parameters:

task - task

getTotalRequestsCount

int getTotalRequestsCount(Task task)

Get TotalRequestsCount for monitor.

Parameters:

task - task

Returns:

number of total request