Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://github.com/timhudson/pagination-stream below:

timhudson/pagination-stream: Combine paginated requests in to a single stream

Combine paginated requests in to a single stream

npm install pagination-stream -g

var paginationStream = require('pagination-stream')
var JSONStream = require('JSONStream')

var options = {
  urlFormat: 'https://api.github.com/repos/joyent/node/events?page=%d',
  headers: {'user-agent': 'pug'},
  start: 0,
  end: 5,
  retries: 2
}

paginationStream(options)
  .pipe(JSONStream.parse('*'))
  .on('data', function(data) {
    console.log(data)
  })

pagination-stream passes opts on to request and retry. See their documentation for all possible options.

• opts.urlFormat - required url string with pagination placeholder
• opts.end - required end page
• opts.start - start page, defaults to 1

Usage: pagination-stream [options]

Options:
-u, --url-format <required>  URL string with pagination placeholder
-e, --end <required>         ending page number
-s, --start                  starting page number, default is 0
-r, --retries                amount of times to retry the operation, default is 10

Example:

$ pagination-stream -u "https://api.github.com/repos/joyent/node/events?page=%d" -s 1 -e 5 -H 'User-Agent: pug' | jsonfilter "*.actor.login"

MIT

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4