Package | Description |
---|---|
us.codecraft.webmagic.downloader |
Downloader is the part that downloads web pages and store in Page object.
|
us.codecraft.webmagic.proxy | |
us.codecraft.webmagic.utils |
Static utils of webmagic.
|
Modifier and Type | Method and Description |
---|---|
HttpClientRequestContext |
HttpUriRequestConverter.convert(Request request,
Site site,
Proxy proxy) |
Modifier and Type | Method and Description |
---|---|
Proxy |
SimpleProxyProvider.getProxy(Task task) |
Proxy |
ProxyProvider.getProxy(Task task)
Get a proxy for task by some strategy.
|
Modifier and Type | Method and Description |
---|---|
static SimpleProxyProvider |
SimpleProxyProvider.from(Proxy... proxies) |
void |
SimpleProxyProvider.returnProxy(Proxy proxy,
Page page,
Task task) |
void |
ProxyProvider.returnProxy(Proxy proxy,
Page page,
Task task)
Return proxy to Provider when complete a download.
|
Constructor and Description |
---|
SimpleProxyProvider(List<Proxy> proxies) |
Modifier and Type | Method and Description |
---|---|
static boolean |
ProxyUtils.validateProxy(Proxy p) |
Copyright © 2017. All rights reserved.