Reference Source 
import Config from 'squidwarc/lib/config/index.js'
public class | source

Config

Crawl config loader

Static Method Summary

Static Public Methods
public static

Load the user supplied script (if it exists) and perform validation of it
public static

async ensureSeeds(seeds: string | string[], mode: string, depth: number): Promise<Seed|Seed[]>

Normalizes the supplied seeds.
public static

async loadConfig(configPath: string): Promise<CrawlConfig>

Loads the config file and performs the preliminary normalization

Static Public Methods

public static async ensureScript(scriptP: string): Promise<UserScript|null> source

Load the user supplied script (if it exists) and perform validation of it

Params:

NameTypeAttributeDescription
scriptP string
  • nullable: true

The path to the user script

Return:

Promise<UserScript|null>

public static async ensureSeeds(seeds: string | string[], mode: string, depth: number): Promise<Seed|Seed[]> source

Normalizes the supplied seeds. If it value of the seeds field is a string loads the seeds from the file

Params:

NameTypeAttributeDescription
seeds string | string[]

The seeds to be normalized
mode string

The crawl mode for the seeds
depth number

The depth of the crawl

Return:

Promise<Seed|Seed[]>

public static async loadConfig(configPath: string): Promise<CrawlConfig> source

Loads the config file and performs the preliminary normalization

Params:

NameTypeAttributeDescription
configPath string

Path to this crawls config file

Return:

Promise<CrawlConfig>