Depth: Depth limits how many link levels are crawled from the entered page. No limit means crawling until the page limit is reached.
Changefreq: Changefreq adds the expected update frequency tag to sitemap.xml.
Exclude URL masks: Exclude masks remove URLs matching listed fragments or wildcards, one per line.
Include only URL masks: Include masks keep only URLs matching listed fragments or wildcards.
Priority: priority adds approximate page importance values to sitemap.xml.
Lastmod: lastmod adds the generation date for each found URL.
Robots.txt: robots.txt skips URLs disallowed for crawlers.
Noindex: noindex removes pages that ask search engines not to index them.
Nofollow: nofollow skips links or pages marked not to be followed.
Canonical: Canonical filtering keeps the preferred URL and removes duplicate alternatives.
Remove GET parameters: GET cleanup removes query parameters such as utm_source or sorting filters.
Only URLs with 200 OK: 200 OK filtering keeps only pages that return a successful HTTP response.
