import NetIdleWatcher from 'squidwarc/lib/crawler/netIdleWatcher.js'

public class | source

NetIdleWatcher

Extends:

EventEmitter → NetIdleWatcher

Monitors the HTTP requests made by a page and emits the 'network-idle' event when it has been determined the network is idle Used by PuppeteerCrawler

Static Method Summary

Static Public Methods
public static	idlePromise(page: Page, options: NetIdleOptions): Promise<void> Start monitoring the network and receive a Promise that resolves once network idle occurred or the global wait time has been reached

Constructor Summary

Public Constructor
public	constructor(page: Page, options: NetIdleOptions)

Member Summary

Public Members
public	page: Page The page object of the current page the crawler is visting

Private Members
private	_doneTimers: boolean Flag indicating if we are in a network tracking state of not
private	_globalWaitTimer: number The id of the global crawler setTimeout timer
private	_idleInflight: number The number of in-flight requests there should be before starting the network-idle timer
private	_idleTime: number The amount of time no new HTTP requests should be made before emitting the network-idle event
private	_idleTimer: number The id of the setTimeout for the network-idle timer
private	_pageListenrs: {emitter: !EventEmitter, eventName: string, handler: function()}[] An array of listeners registered on the page object
private	_requestIds: Set<string> Set of the HTTP requests ids, used for tracking network-idle
private	_timeout: number Maximum amount of time a crawler going to visit a page

Method Summary

Public Methods
public	reqFinished(info: Response \| Request) Indicate that a request has finished
public	reqStarted(info: Request) Indicate that a request was made
public	start() Setup the necessary listeners

Private Methods
private	_clearTimers() Clear all timers
private	_emitNetIdle() Emit the network-idle event
private	_globalNetworkTimeout() Called when the global time limit was hit
private	_networkIdled() Called when the network idle has been determined

Static Public Methods

public static idlePromise(page: Page, options: NetIdleOptions): Promise<void> source

Start monitoring the network and receive a Promise that resolves once network idle occurred or the global wait time has been reached

Params:

Name	Type	Attribute	Description
page	Page		Puppeteer page object for the page being crawled
options	NetIdleOptions	optional default: {} nullable: true	Optional options to control fine tune network idle determination

Return:

Promise<void>

Public Constructors

public constructor(page: Page, options: NetIdleOptions) source

Params:

Name	Type	Attribute	Description
page	Page		Puppeteer page object for the page being crawled
options	NetIdleOptions	optional default: {} nullable: true	Optional options to control fine tune network idle determination

Public Members

public page: Page source

The page object of the current page the crawler is visting

Private Members

private _doneTimers: boolean source

Flag indicating if we are in a network tracking state of not

private _globalWaitTimer: number source

The id of the global crawler setTimeout timer

private _idleInflight: number source

The number of in-flight requests there should be before starting the network-idle timer

private _idleTime: number source

The amount of time no new HTTP requests should be made before emitting the network-idle event

private _idleTimer: number source

The id of the setTimeout for the network-idle timer

private _pageListenrs: {emitter: !EventEmitter, eventName: string, handler: function()}[] source

An array of listeners registered on the page object

private _requestIds: Set<string> source

Set of the HTTP requests ids, used for tracking network-idle

private _timeout: number source

Maximum amount of time a crawler going to visit a page

Public Methods

public reqFinished(info: Response | Request) source

Indicate that a request has finished

Params:

Name	Type	Attribute	Description
info	Response \| Request		Puppeteer Request or Response object

public reqStarted(info: Request) source

Indicate that a request was made

Params:

Name	Type	Attribute	Description
info	Request		Puppeteer Request object

public start() source

Setup the necessary listeners

Private Methods

private _clearTimers() source

Clear all timers

private _emitNetIdle() source

Emit the network-idle event

private _globalNetworkTimeout() source

Called when the global time limit was hit

private _networkIdled() source

Called when the network idle has been determined