Hi,
When doing #1109 and checking scrapy/scrapy#803 I've run a couple of benchmarks:
The problem is that BytesIO copies the data, so using it just to make interface file-like for readonly data could be quite wasteful.
I believe this is a problem in curl_httpclient, simple_httpclient and wsgi tornado modules, and by using another wrapper (or at least by falling back to cStringIO in Python 2.x) they can be made faster and more memory efficient.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4