We have moved to gitlab.com. Click here to learn how to join: https://gitlab.com/elenx_net/Tabula/-/wikis/home
//check -> https://elenx.net/blank/tabula/wikis/2_Research/Crawler-WebMagic
Crawler4j is a set of classes that aims to allow creation of basic web-crawlers with multi-threading in mind (WebCrawler implements Runnable interface).
To create a simplest crawler, we have to create crawler class (extending WebCrawler) with overriden and implemented methods: shouldVisit and visit.