1. Overview of WebMagic

WebMagic is a simple crawler for Java developer.

WebMagic contains two parts: core and extension. Webmagic-core is a simple and well modulized implemetation of crawler, and webmagic-extension supply some convenient features for crawler depleloping.

The architect of WebMagic-core is refer to Scrapy. It supply simple but flexible API. You can write a crawler just if you are familiar to Java.

Webmagic-extension supply some convenient features, such as writing a crawler only with a POJO and some annotation. There are also some default implementation of the components.

Webmagic also contains some other extensions and an complete product "WebMagic-Avalon".