Home Reference Source Repository

node-warc

Parse WARC files or create WARC files using either Electron or chrome-remote-interface

Run npm install node-warc or yarn add node-warc to ge started

npm Package

API

Full API documentation available at n0tan3rd.github.io/node-warc

Example usage

Example 1: Both .warc and .warc.gz

const AutoWARCParser = require('node-warc')

const parser = new AutoWARCParser('<path-to-warcfile>')
parser.on('record', record => { console.log(record) })
parser.on('done', finalRecord => { console.log(finalRecord) })
parser.on('error', error => { console.error(error) })
parser.start()

Example 2: Only .warc.gz

const WARCGzParser = require('node-warc').WARCGzParser

const parser = new WARCGzParser('<path-to-warcfile>')
parser.on('record', record => { console.log(record) })
parser.on('done', finalRecord => { console.log(finalRecord) })
parser.on('error', error => { console.error(error) })
parser.start()

Example 3: Only .warc

const WARCParser = require('node-warc').WARCParser

const parser = new WARCParser('<path-to-warcfile>')
parser.on('record', record => { console.log(record) })
parser.on('done', finalRecord => { console.log(finalRecord) })
parser.on('error', error => { console.error(error) })
parser.start()

Benchmark

UN-GZIPPED

GZIPPED

JavaScript Style Guide