us.codecraft.webmagic

Class Page

java.lang.Object

us.codecraft.webmagic.Page

public class Page
extends Object

Object storing extracted result and urls to fetch.
Not thread safe.
Main method：
getUrl() get url of current page
getHtml() get content of current page
putField(String, Object) save extracted result
getResultItems() get extract results to be used in Pipeline
addTargetRequests(java.util.List) addTargetRequest(String) add urls to fetch

Since:

0.1.0

Author:

code4crafter@gmail.com

See Also:

Downloader, PageProcessor

Constructor Summary

Constructors

Constructor and Description
Page()

Method Summary

Modifier and Type	Method and Description
void	addTargetRequest(Request request) add requests to fetch
void	addTargetRequest(String requestString) add url to fetch
void	addTargetRequests(List<String> requests) add urls to fetch
void	addTargetRequests(List<String> requests, long priority) add urls to fetch
static Page	fail()
byte[]	getBytes()
String	getCharset()
Map<String,List<String>>	getHeaders()
Html	getHtml() get html content of page
Json	getJson() get json content of page
String	getRawText()
Request	getRequest() get request of current page
ResultItems	getResultItems()
int	getStatusCode()
List<Request>	getTargetRequests()
Selectable	getUrl() get url of current page
boolean	isDownloadSuccess()
void	putField(String key, Object field) store extract results
void	setBytes(byte[] bytes)
void	setCharset(String charset)
void	setDownloadSuccess(boolean downloadSuccess)
void	setHeaders(Map<String,List<String>> headers)
void	setHtml(Html html) Deprecated. since 0.4.0 The html is parse just when first time of calling getHtml(), so use setRawText(String) instead.
Page	setRawText(String rawText)
void	setRequest(Request request)
Page	setSkip(boolean skip)
void	setStatusCode(int statusCode)
void	setUrl(Selectable url)
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Page

public Page()

Method Detail

fail

public static Page fail()

setSkip

public Page setSkip(boolean skip)

putField

public void putField(String key,
                     Object field)

store extract results

Parameters:

key - key

field - field

getHtml

public Html getHtml()

get html content of page

Returns:

html

getJson

public Json getJson()

get json content of page

Returns:

json

Since:

0.5.0

setHtml

public void setHtml(Html html)

Deprecated. since 0.4.0 The html is parse just when first time of calling getHtml(), so use setRawText(String) instead.

Parameters:

html - html

getTargetRequests

public List<Request> getTargetRequests()

addTargetRequests

public void addTargetRequests(List<String> requests)

add urls to fetch

Parameters:

requests - requests

addTargetRequests

public void addTargetRequests(List<String> requests,
                              long priority)

add urls to fetch

Parameters:

requests - requests

priority - priority

addTargetRequest

public void addTargetRequest(String requestString)

add url to fetch

Parameters:

requestString - requestString

addTargetRequest

public void addTargetRequest(Request request)

add requests to fetch

Parameters:

request - request

getUrl

public Selectable getUrl()

get url of current page

Returns:

url of current page

setUrl

public void setUrl(Selectable url)

getRequest

public Request getRequest()

get request of current page

Returns:

request

setRequest

public void setRequest(Request request)

getResultItems

public ResultItems getResultItems()

getStatusCode

public int getStatusCode()

setStatusCode

public void setStatusCode(int statusCode)

getRawText

public String getRawText()

setRawText

public Page setRawText(String rawText)

getHeaders

public Map<String,List<String>> getHeaders()

setHeaders

public void setHeaders(Map<String,List<String>> headers)

isDownloadSuccess

public boolean isDownloadSuccess()

setDownloadSuccess

public void setDownloadSuccess(boolean downloadSuccess)

getBytes

public byte[] getBytes()

setBytes

public void setBytes(byte[] bytes)

getCharset

public String getCharset()

setCharset

public void setCharset(String charset)

toString

public String toString()

Overrides:

toString in class Object