Skip to content

Set domcontentloaded as default strategy #3140

@monolithed

Description

@monolithed

Which package is the feature request for? If unsure which one to select, leave blank

@crawlee/core

Feature

There have already been several issues where the crawler either doesn’t start working or crashes due to the load strategy.
I believe the load event should be replaced with the safer and more reasonable domcontentloaded event, which definitely won’t backfire.

Motivation

We live in an era where internet providers in many countries, due to regulatory decisions, block or slow down network requests. Sites that use third-party ads or trackers also often load very slowly — sometimes even Google counters can take more than 30 seconds to load.

Ideal solution or implementation, and any additional constraints

domcontentloaded

Alternative solutions or implementations

No response

Other context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureIssues that represent new features or improvements to existing features.t-toolingIssues with this label are in the ownership of the tooling team.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions