Skip to content

fix: ignore whitespace-only Disallow paths in extractUrlsFromRobotsTxt#1973

Open
juliosuas wants to merge 1 commit intosmicallef:masterfrom
juliosuas:fix/robots-txt-whitespace-disallow
Open

fix: ignore whitespace-only Disallow paths in extractUrlsFromRobotsTxt#1973
juliosuas wants to merge 1 commit intosmicallef:masterfrom
juliosuas:fix/robots-txt-whitespace-disallow

Commits

Commits on Mar 30, 2026