Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://github.com/Tjatse/node-readability below:

Tjatse/node-readability: Scrape/Crawl article from any site automatically. Make any web page readable, no matter Chinese or English.

1. Readability reference to Arc90's.
2. Scrape article from any page (automatically).
3. Make any web page readable, no matter Chinese or English.

快速抓取网页文章标题和内容，适合node.js爬虫使用，服务于ElasticSearch。

• Features
• Performance
• Installation
• Usage
• Debug
• Score Rule
• Extract Selectors
• Image Fallback
• Threshold
• Customize Settings
• Output
• Notes

In my case, the speed of spider is about 1500k documents per day, and the maximize crawling speed is 1.2k /minute, avg 1k /minute, the memory cost are about 200 MB on each spider kernel, and the accuracy is about 90%, the rest 10% can be fixed by customizing Score Rules or Selectors. it's better than any other readability modules.

(4) Server infos:

• 20M bandwidth of fibre-optical
• 8 Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz cpus
• 32G memory

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4