Ingest API Best Practices

Updated

This guide provides recommendations for reliable ingest workflows. It complements the Ingest API endpoint reference.

Article Overview

This guide covers these practices:

  • credential usage patterns
  • batching and payload sizing
  • retries and monitoring
  • cleanup and reprocessing workflows

For endpoint-level request and response contracts, see the Ingest API reference.

Request Parameters (Cross-Link Context)

Name Required Type Description
Authorization Yes header Use Authorization: Token <read-write-token> for ingest write operations.
commit No query string Optional update query parameter controlling commit behavior (true or false).

Recommended Operational Practices

  • Use read-write tokens only for write paths.
  • Keep payloads below hard request-size ceilings.
  • Use staged retries for transient failures.
  • Validate index state after large updates.

HTTP Status Codes and Error Handling (Operational View)

HTTP Code When It Happens Typical Response Body What to Do
200 Successful write operation Update success payload Continue pipeline
400 Payload reaches parser but contains invalid command or schema Parser error payload Correct payload structure
401 Wrong credential scope (for example read-only token on write) {"message":"Unauthorized"} Use read-write credentials
413 Request payload too large Content-length limit response (10485760) Reduce batch size
414 Request line or URL too large URI and request-too-large response Reduce request size and URL
429 Shared rate and plan limit condition Too-many-requests response Retry with backoff

Pagination

Not applicable. This section covers write operations, not paged reads.

Rate Limits and Payload Size Guidance

  • Shared app-level guidance: 20+ requests/second.
  • Effective payload-size boundary is around 10 MB.
  • Replace any remaining 2048 KB guidance with 10 MB guidance.

Shared API Foundations

Use these shared references for authentication, request and response structure, and pagination behavior used across Site Search APIs:

Articles in this section