This is a comparison of web search APIs (SERP APIs) for LLM-powered AI workflows and agents.
Why Use a Search API With an AI Workflow?
You may need a web search API if you’re building an LLM-powered system and want to retrieve relevant web content to include in the context you provide to the LLM.
For example, if you’re creating a fact checking tool, you might want to search the web and retrieve content relating to a fact that your system is checking.
How Do AI Agents Use Web Search APIs?
If you’re developing an AI agent, you’ll typically want it to make use of web search APIs through functionality baked into LLMs known as ‘tool-calling’, ‘tool-use’ or ‘function-calling’.
Frameworks such as LangChain have support for tool-/function-calling and have off-the-shelf integrations with some of the APIs (e.g. ones for Tavily, Exa and YOU.com.)
If an integration isn’t already available, though, it’s typically fairly simple to build one.
Web Search API Pricing
| Name | Free Tier | Lowest-Volume Paid Tier | Lowest Advertised Cost (at Scale) * | Search Index | Content Snippet Size | Highest Advertised Rate Limit * |
|---|---|---|---|---|---|---|
| Apify | 1428 calls/month | $39/month for 11,000 calls | $3.50 per 1000 calls | 160 chars | 30 calls per second | |
| Bing | 1000 calls/month | $0 – PAYG / $15 per 1000 calls | $15 per 1000 calls | Bing | Up to 250 chars | 250 calls per second |
| Brave | 2000 calls/month; 1 call/second | $0 – PAYG / $5 per 1000 calls | $5 per 1000 calls | Brave | 400 chars | 50 calls per second |
| Bright Data | n/a | $0 – PAYG / $1.50 per 1000 calls | $1 per 1000 calls | 160 chars | no limit | |
| DataForSEO | n/a | $50 one-off / $0.6 per 1000 calls | $0.6 per 1000 calls | 160 chars | 2000 calls per minute (avg. 33 calls/sec) | |
| Exa | $10 of free credits | $0 – PAYG / $2.5 per 1000 calls | $2.50 per 1000 calls | Exa | 150 chars | 5 calls per second |
| Linkup | 1000 calls/month | $0 – PAYG / $5 per 1000 calls | $5 per 1000 calls | Linkup | Up to 5000 chars | 20 calls per second |
| SearchAPI | 100 free calls | $40/month for 10,000 calls | $1 per 1000 calls | 160 chars | 20% of monthly call volume per hour | |
| Serper | 2500 calls | $50 (valid for 6 months) for 50,000 calls | $0.30 per 1000 calls | 150 chars | 300 calls per second | |
| SerpApi | 100 calls/month | $75/month for 5,000 searches | $5.50 per 1000 calls | 160 chars | 20% of monthly plan volume per hour | |
| Tavily | 1000 calls/month | $0 – PAYG / $8 per 1000 calls | $5 per 1000 calls | Tavily | 3000+ chars | 1000 calls per minute (avg. 17 calls/sec) |
| you.com | 1000 calls/month for 60 days | $100/month for 11,765 calls | $8 per 1000 calls | you.com | 600 chars | Not advertised |
What to Consider When Choosing a Web Search API
1. Underlying Search Index
Different services use different search indexes to retrieve their results.
Many services are wrappers around Google Search and should therefore return very similar results to each other. (These list ‘Google’ in the ‘Search Index’ column of the table above.)
Other services have their own indexes and may return very different sets of results.
Just as you may prefer Google over Bing search when you’re searching manually, you may prefer the results from one web search API over those from another.
2. Amount of Content for Each Search Result
Different services return different amounts of information for each search result. This can have a big impact on the effectiveness of your RAG workflow or agent.
In some cases you may prefer a service that returns relatively long snippets of content even if it costs more per request. In others you may find that smaller snippets of content are fine or even preferable as the LLM’s prompt ends up being more focussed.
3. Pricing
Prices vary considerably between services.
When you’re comparing prices, be aware that some services have optional parameters that, if you need them, can significantly impact pricing, e.g. doubling the cost of requests.
4. Latency
Services differ widely in how quickly they respond to requests.
Low cost services that work by scraping Google (these will list ‘Google’ in the ‘Search Index’ column) may be much slower than services that query their own indexes. These ‘Google wrapper’ services sometimes offer a choice of more expensive, faster options and cheaper, slower options.
5. Rate Limits
All services are limited, to some extent, in the rate of requests they can handle. Some have fixed rate limits that they make public.
6. Other Terms and Conditions
Depending on the nature of your business, factors such as a provider’s privacy policies and/or the country where they operate their servers may be very important.
Some of the links in this article are affiliate links. This means I may earn a commission if you make a purchase through them—at no extra cost to you.