us.codecraft.webmagic.processor

Interface PageProcessor

All Known Implementing Classes:

AlexanderMcqueenGoodsProcessor, AmanzonPageProcessor, AngularJSProcessor, BaiduBaikePageProcessor, CompositePageProcessor, ConfigurablePageProcessor, DiandianBlogProcessor, DiaoyuwengProcessor, F58PageProcesser, GithubRepoPageMapper, GithubRepoPageProcessor, GithubRepoPageProcessor, HuxiuProcessor, InfoQMiniBookProcessor, IteyeBlogProcessor, KaichibaProcessor, MamacnPageProcessor, MeicanProcessor, NjuBBSProcessor, PhantomJSPageProcessor, QzoneBlogProcessor, ScriptProcessor, SimplePageProcessor, SinaBlogProcessor, TianyaPageProcesser, ZhihuPageProcessor, ZhihuPageProcessor, ZipCodePageProcessor

public interface PageProcessor

Interface to be implemented to customize a crawler.

In PageProcessor, you can customize:
start urls and other settings in Site
how the urls to fetch are detected
how the data are extracted and stored

Since:

0.1.0

Author:

code4crafter@gmail.com

See Also:

Site, Page

Method Summary

Modifier and Type	Method and Description
Site	getSite() get the site settings
void	process(Page page) process the page, extract urls to fetch, extract the data and store

Method Detail

process

void process(Page page)

process the page, extract urls to fetch, extract the data and store

Parameters:

page - page

getSite

Site getSite()

get the site settings

Returns:

site

See Also:

Site