If a crawl fails, open the failed run in Crawler > History, then check the error code in Crawl Failure Details. Find that code below for an explanation and recommended next steps.
Error Codes and Recommended Actions
| Error Code | What It Means | How to Fix It |
|---|---|---|
| SOLR_PREFLIGHT_FAILED | The crawl couldn’t start because the search index was unavailable or incorrectly configured. | Retry the crawl. If it fails again, contact Support with the crawl ID. |
| SOLR_SCHEMA_PREFLIGHT_FAILED | The crawl couldn’t start because one or more crawler fields don’t match the search schema. | Review the reported fields and compare crawler field definitions, mappings, and transformations with the live schema. Correct missing or invalid fields, then retry. |
| CRAWL_SUBMISSION_FAILED | The crawl couldn’t be submitted for processing. | Retry the crawl. If it fails again, contact Support with the crawl ID. |
| CRAWL_RUNTIME_STATE_LOST | The crawl stopped unexpectedly, but a specific failure reason was not available. | Retry the crawl. If it fails again, contact Support with the crawl ID. |
| CRAWLER_STARTUP_FAILED | The crawler stopped before it could request the Start URL. | Retry the crawl. If it fails again, contact Support with the crawl ID. |
| CRAWLER_FETCH_FAILED | The crawler couldn’t retrieve a required page or file. | Test the affected URL using the crawler's configured headers. Check authentication, firewall/WAF rules, rate limits, website availability, content type, and file size. |
| AUTH_401 | Authentication failed. The crawler couldn’t access the requested page because authentication failed. | Verify your authentication credentials or HTTP headers in Crawler Settings. |
| ACCESS_403 | Access to the website was denied. | Verify the crawler is allowed through your website's firewall, WAF, IP restrictions, or authentication settings. See Crawler IP Allowlisting for more details. |
| NOT_FOUND_404 | The requested page couldn’t be found. | Confirm the URL still exists and is correct. If this is your Start URL or sitemap, update the crawler configuration. If it's an internal page, verify the page hasn't been moved or deleted. |
| RATE_LIMITED_429 | The website is temporarily limiting crawler requests. | Wait a few minutes before retrying the crawl. If the issue continues, review your website's rate-limiting configuration or adjust crawler settings if appropriate. |
| SERVER_5XX | The website encountered a server error. | Verify your website is available and functioning normally. Retry the crawl once the server issue has been resolved. |
| TIMEOUT | The website didn’t respond before the request timed out. | Verify that the website is online and responding normally. If the page loads slowly, try again later or increase any server-side timeout settings if applicable. Check whether your website, CDN, firewall, or hosting provider is delaying or blocking requests from the crawler. Once the site is responding consistently, retry the crawl. |
| DNS_FAILED | The crawler couldn’t locate or connect to the website's domain. | Verify that the domain name in your Start URL or sitemap is correct and that the website's DNS records are properly configured. Confirm the domain resolves in a web browser or with your DNS provider. If the domain was recently updated, wait for DNS changes to propagate before rerunning the crawl. |
| DOWNLOAD_LIMIT_EXCEEDED | The URL was not processed because its download exceeded the configured limit. | Increase the download limit or exclude this URL if the document should not be indexed. |
| DOCUMENT_TYPE_MISSING | The crawler couldn’t determine the document type. | Verify that the server returns a supported Content-Type header. |
| CRAWLER_PROCESSING_ERROR | The crawler couldn’t process the downloaded response. | Verify that the URL returns supported, well-formed content before retrying the crawl. |
| ALL_URLS_FAILED | The crawler couldn’t retrieve the source content. | Verify that the source site is online and review its DNS, network and TLS configuration. |
| NO_INDEXABLE_CONTENT | The crawl completed, but no content was available to add to the index. | Check the sitemap, inclusion/exclusion rules, allowed domains, supported content types, and noindex directives. Confirm that at least one eligible page should be indexed. |
| PARSER_PROCESSING_FAILED | The crawler retrieved content, but it couldn’t be processed for indexing. | Review the affected file or page and configured field extraction or transformations. Depending on the issue, this may require removing password protection, using a supported file type, or correcting an invalid XPath or transformation. |
| INDEXER_SUBMISSION_FAILED | The crawl processed content, but the items couldn’t be added to the search index. | Retry the crawl. If it fails again, contact Support with the crawl ID. |
| SOLR_DOCUMENT_REJECTED | One or more items couldn’t be indexed because their field data didn’t match the search schema. | Review the reported field names, field types, required values, and single-/multi-valued settings. Use the sample URL to inspect the extracted item, correct the crawler field configuration, and retry. |
| SOLR_AUTH_FAILED | The crawler couldn’t connect to the search index. | Retry the crawl. If it fails again, contact Support with the crawl ID. |
| PLAN_LIMIT_EXCEEDED | The crawl exceeded the application's item limit. | Review the Subscription Items limit. If your subscription limit needs to be increased, contact Support before retrying the crawl. |
| FAILED_CANCEL_REQUEST | The crawl was stopped by a system request, but no specific reason was recorded. | Retry the crawl. If it fails again, contact Support with the crawl ID. |
| FINAL_COMMIT_AUTH_FAILED | The crawl finished processing, but the final index update couldn’t be completed because authentication failed. | Retry the crawl. If it fails again, contact Support with the crawl ID. |
| FINAL_COMMIT_FAILED | The crawl finished processing, but the final index update couldn’t be completed. | Retry the crawl. If it fails again, contact Support with the crawl ID. |