This is a comparison of web search APIs (SERP APIs) for LLM-based AI workflows and agents.
Why Use a Search API With an AI Workflow?
You may need a web search API if you’re building an LLM-powered system and want to retrieve relevant web content to include in the context you provide to the LLM.
For example, if you’re creating a fact checking tool, you might want to search the web and retrieve content relating to a fact that your system is checking.
How Do AI Agents Use Web Search APIs?
If you’re developing an AI agent, you’ll typically want it to make use of web search APIs through functionality baked into LLMs known as ‘tool-calling’, ‘tool-use’ or ‘function-calling’.
Frameworks such as LangChain have support for tool-/function-calling and have off-the-shelf integrations with some of the APIs (e.g. ones for Tavily, Exa and YOU.com.)
If an integration isn’t already available, though, it’s typically fairly simple to build one.
Web Search API Pricing
| Name | Free Tier | Lowest-Volume Paid Tier | Lowest Advertised Cost* (at Scale) | Content Snippet Size | Highest Advertised Rate Limit* |
|---|---|---|---|---|---|
| Bing | 1000 calls/month | Pay-as-you-go ($0.015/call) | $0.015/call | Up to 250 chars | 250 calls per second |
| Brave | 2000 calls/month; 1 call/second | Pay-as-you-go ($0.005/call) | $0.005/call | 400 chars | 50 calls per second |
| Bright Data | n/a | Pay-as-you-go ($0.0015/call) | $0.001/call | 150 chars | no limit |
| DataForSEO | n/a | Pay-as-you-go ($0.0006/call) | $0.0006/call | 150 chars | 2000 calls per minute |
| Exa | $10 of free credits | Pay-as-you-go ($0.0025/call) | $0.0025/call | 150 chars | 5 calls per second |
| Linkup | 1000 calls/month | Pay-as-you-go ($0.005/call) | $0.005/call | 400 chars | Not advertised |
| Serper | 2500 calls | $50 (valid for 6 months) | $0.0003/call | 150 chars | 300 calls per second |
| SerpApi | 100 calls/month | $75/month (for 5,000 searches) | $0.0055/call | 150 chars | 20% of monthly plan volume per hour |
| Tavily | 1000 calls/month | Pay-as-you-go ($0.008/call) | $0.005/call | 3000+ chars | 1000 calls per minute |
| YOU | 1000 calls/month for 60 days | $100/month | $0.008/call | 600 chars | Not advertised |
What to Consider When Choosing a Web Search API
- Relevance of search results – different services use different search engines to retrieve their results. Just as you may prefer Google search over Bing search when you’re searching manually, you may prefer the results from one web search API over those from another.
- Content of each search result – different services return different amounts of information for each search result and this can have a big impact on the effectiveness of your agent. You may, for example, prefer a service that returns relatively long snippets of content even if it costs more per request.
- Pricing – unsurprisingly, different services have different prices. Note that some services have optional parameters that can significantly impact pricing, e.g. doubling the effective cost of requests.
- Latency – this can vary hugely from near-instant to potentially as much as 5 minutes or more.
- Rate limits – all services will, in practice, be limited in the rate of requests they can handle. Some have fixed rate limits that they make public.
- Other terms and conditions – depending on the nature of your business, factors such as the providers’ privacy policies and/or geographic location may be very important.
Some of the links in this article are affiliate links. This means I may earn a commission if you make a purchase through them—at no extra cost to you. I only recommend products I genuinely believe in.