Large Language Models (LLMs) lie.
Not maliciously, of course. But they hallucinate. They state falsehoods with unwavering confidence, and with a little nudging and creative prompting, you can get them to agree with just about anything.
In a world increasingly leaning on AI for answers (which it arguably shouldn’t be, but that’s another post entirely), this is a problem.
So, how can we ground an LLM in reality? Let’s find out.
We’re going to build a real-time fact checking AI agent (no LangChain here. We’ll write the core, auditable pipeline ourselves, using OpenAI’s SDKs and Bright Data’s SERP API) that doesn’t just guess based on training data — but searches, reads, analyzes live Google search data, and then decides. All off the back of one guiding principle: No black boxes.
This agent will:
- Take a claim as input
- Plan and execute focused web searches
- Extract evidence from live, real, trusted sources using Bright Data’s SERP API with Google
- Analyze the search results with OpenAI’s models.
- Return a clear verdict — with confidence scores and justifications
I gave it this claim:
“Listening to Mozart makes you smarter.”
And here’s what it returned:
Claim: Listening to Mozart makes you smarter.
Verdict: Misleading
Confidence: 90%
Sources: 21
Explanation: While an early 1993 study found a brief improvement in
spatial reasoning after 10 minutes of Mozart (PMC1281386),
multiple reviews and meta-analyses (ParentingScience, Britannica, BBC Future)
conclude there is no lasting increase in general intelligence.
The idea that listening to Mozart reliably makes you smarter
(popularly known as 'The Mozart Effect') is largely a media‐driven myth,
not a well-established scientific fact.
Now this is the kind of output we want from agents in the LLM age —instead of adding to the noise on the internet today, we’re building AI tooling that cuts through it and produces verifiable answers with auditable receipts.
From a B2B standpoint, that kind of agentic architecture has wide implications. You could repurpose this project for automated compliance monitoring, market intelligence, or real-time misinformation detection. Bright Data’s tooling lets our agent systematically collect and synthesize public information at scale, forming the backbone for enterprise-grade AI pipelines that depend on fresh, verifiable knowledge.
If you’d rather jump right into the code, it’s hosted here, under an MIT License.
The Tech Stack
Here’s what I used to build this:
@ai-sdk/openai&ai— Vercel’s AI SDK’s for OpenAI integration. Comes with agenerateObject()function for structured AI responses with schema validation, so we can always get consistent JSON outputs from GPT models.zod— TypeScript-first schema validation library, needed for the above. This makes sure the AI’s responses match the expected data structures we want, and prevents runtime errors from malformed AI outputs.node-fetch— Simple, lightweight HTTP client we’ll use for making API requests on NodeJS.https-proxy-agent— Lets us make HTTP requests through proxy servers. Essential for routing search queries through Bright Data’s proxy infrastructure without getting blocked by Google.dotenv— Pretty standard. Loads environment variables from .envfiles.
Use your package manager of choice to get these.
npm install @ai-sdk/openai ai zod node-fetch https-proxy-agent dotenv
I’m using OpenAI here because that’s what I have a subscription for. Just use what you have. Just FYI: for best results, you’ll probably want to have access to a reasoning model in addition to the usual low-latency one.
Why Bright Data? Google search is the primary source of factual evidence, but at scale, you’ll hit rate limits and blocks. Also, it’s fragile. Layouts change weekly (plus Google loves running A/B tests every week).
This gives us normalized JSON output for Google (and other engines like Bing, DuckDuckGo, Yandex, Baidu) searches, supports global geo-targeting, and rotates IPs automatically.
In short, it makes scraping search results reliable and production-grade. For B2B teams building AI products or monitoring pipelines, this kind of infrastructure is non negotiable. You could use it for brand monitoring, regulatory compliance, competitive intel, or LLM grounding at scale, and without it, you’re stuck duct-taping together proxies, selectors, and screen scrapers that break silently (and often).
Reliable access to consistent SERP structure is non-negotiable when my pipeline depends on evidence-gathering.
Setting Up Bright Data’s SERP API
If you don’t have a Bright Data account yet, you can sign up for free. Adding a payment method will grant you a $5 credit to get started — no charges upfront.
1. Sign in to Bright Data
Log in to your Bright Data account and you’ll be on the dashboard.
2. Creating a Proxy Zone
- From there, find the My Zones page, go to the SERP API section and click Get Started.
- If, somehow, you were already using an active Bright Data proxy, you don’t have to create a new zone, just click Add in the top-right corner of this page.
3. Assign a name to your SERP API zone
- Choose a meaningful name, as it cannot be changed once created.
4. Click “Add” and Verify Your Account
- If you haven’t verified your account yet, you’ll be prompted to add a payment method at this stage.
- First-time users receive a $5 bonus credit, so you can test the service without any upfront costs.
There’s additional ways to configure the SERP API you just created, but we won’t have to think about those just yet.
Once that’s done, this is what you’ll need to copy out in the “Overview” tab, for your .env file.
# Bright Data SERP API Configuration
BRIGHT_DATA_CUSTOMER_ID=hl_xxxxxxx
BRIGHT_DATA_ZONE=xxxxxx
BRIGHT_DATA_PASSWORD=xxxxxxxxxxxxx
# OpenAI
OPENAI_API_KEY="sk-proj-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
Throw in an OpenAI (or whatever LLM-as-a-service you’re using) key here too, while we have the file open.
Perceive, Reason, Act.
At a high level, this agent does two things — and does them well:
- Gathers external evidence at scale
- Reasons over that evidence using an LLM
Everything has to be structured and auditable. No relying on the model’s internal memory. I need to be able to give it a factual claim, and then have it go out into the wild, find relevant sources, weigh the evidence, and come back with a verdict — and the receipts.
This architecture is built around a classic AI pattern, PRA : Perception → Reasoning → Action.
Perception: Gathering Data from the Real World
This is how the agent “sees” the world. It starts by taking a natural-language claim like:
“The Great Wall of China is the only man-made object visible from space.”
Then:
- Decides whether to decompose it into sub-claims, if needed
- Searches the web using the SERP API (Google results via Bright Data proxy)
- Parses and filters the results: removing fluff, prioritizing credible sources, and extracting relevant evidence
Even seemingly straightforward claims like “Vaccines cause autism and are unsafe for children” might actually contain multiple distinct assertions that need separate verification.
So we’ll use AI to first analyze whether to decompose the claim, then run parallel searches for each sub-claim (single search if no decomposition needed), and finally synthesize the evidence —we’re pretty much mimicking how human fact-checkers work.
By the end of this stage, the agent has a clean, structured bundle of facts.
Reasoning: Thinking, Not Just Predicting
This is where the model earns its keep. We give GPT the claim and the structured, gathered evidence from the previous stage — nothing else. No assumptions, no hallucinated knowledge.
💡 This phase is where you’ll want to use a complex reasoning model like OpenAI’s o-series, Claude 4, Qwen’s QWQ, etc.
The prompt is designed for critical analysis:
- Weigh the evidence
- Identify consensus
- Spot contradictions
- Evaluate credibility of sources
LLMs are not reliable sources of factual truth on their own. They’re fundamentally pattern predictors, trained to continue text based on what they’ve seen during training, nothing else.
If you’re asking a model, “What year did the Berlin Wall fall?”, it’s probably going to say 1989 — but it’s doing that based on what that sentence usually looks like in its training data, and not because it has an actual database of verified history. It’s fundamentally different from querying Wikipedia.
That’s why we feed it real data, and use a prompt that limits it to finding patterns within given data and evaluating given facts — which GenAI was made for, and excels at.
Action: Reporting the Verdict
The final step is simple but crucial: tell the truth clearly. How do we quantify the truth? That’s harder to tell, but this is what I wanted the agent to output, for each given claim:
- If the claim was True, False, Partially True/Misleading
- A confidence score
- A reasoned explanation, grounded in the sources
All inputs, outputs, and prompts are logged. Every decision is traceable. You can audit every part of what the agent saw, thought, and concluded.
This is how you build AI systems you can trust — models are never perfect, but your process can strive to be.
Why I Built It Myself
The AI ecosystem is overflowing with agent frameworks. LangChain, AutoGPT, ReAct, CrewAI, AgentOps — pick your flavor. They’re all trying to do similar things: abstract away the hard parts of chaining tools and models into something that behaves like an “agent.”
So why did I roll my own?
Frankly, control. When you adopt some of these frameworks wholesale, you’re not building with LLMs — you’re building on top of someone else’s architecture for what LLMs should do.
I wanted to understand the moving parts. Most agent frameworks do way more than you think behind the scenes. They inject prompts you don’t see, chain tools in ways you don’t always control, and rely on global state or hidden assumptions about memory, retry logic, or task planning. That’s fine for demos. Not for a real-world system I need to debug, scale, and evolve.
Also, no shade thrown, but have you ever tried debugging LangChain? 😅 With my approach, each stage is a pure function. I could benchmark, swap parts out of, or even rewrite all of this in another language if needed.
The Code
Let’s start with main(), the entry point for this whole thing. This should be quite simple. We’re just gonna take in a claim via command line argument, and provide a basic one if nothing’s passed in.
async function main() {
try {
const claim = process.argv[2] || "Vaccines cause autism and are unsafe for children";
// execute the main agent cycle (perception -> reasoning -> action)
const result = await agentTick(claim);
if (result.error) {
process.exit(1); // uh oh. something went wrong with agent init/execution
} else {
// agent completed successfully with verdict and confidence score
}
} catch (error) {
// general purpose error
process.exit(1);
}
}
As you can tell, at the heart of this project is a single agentTick(claim) function — the orchestrator. This kicks off our Perception–Reasoning–Action (PRA) loop.
async function agentTick(claim) {
const perception = await perceive(claim);
const reasoning = await reason(claim, perception);
const action = await act(claim, perception, reasoning);
return action;
}
Perception
Under the hood, perceive() does three things: plan a search, execute said plan, and clean the results to return structured, enriched data for the Reasoning phase.
async function perceive(claim) {
try {
// STEP 1: PLANNING - do we need single or multiple focused searches?
const decomposition = await shouldDecompose(claim);
// STEP 2: EXECUTION - perform single search or parallel sub-searches based on said plan
const rawSearchData = await executePlan(claim, decomposition);
// STEP 2.5 - decide on a return format based on that
let actualSearchData;
let searchStrategy = 'single';
let subSearches = null;
if (rawSearchData.search_strategy === 'decomposed') {
// decomposed search: extract merged results and sub-query metadata
actualSearchData = rawSearchData.combined_results;
searchStrategy = 'decomposed';
subSearches = rawSearchData.sub_searches;;
} else {
// single search: use results directly
actualSearchData = rawSearchData;
}
// STEP 3: PERCEPTION BUILDING - Clean raw data for the reasoning phase
const perception = buildPerceptionFromSearchData(actualSearchData, claim, searchStrategy, subSearches);
return perception;
} catch (error) {
throw new Error(`Perception failure: ${error.message}`);
}
}
Step 1: Plan the search — shouldDecompose() uses AI to determine if the given claim should be a single, one-and-done search using the SERP API, or be decomposed into sub-claims (meaning multiple SERP API calls in parallel).
const result = await generateObject({
model: openai('gpt-4o-mini'),
schema: z.object({
needs_breakdown: z.boolean(),
sub_queries: z.array(z.string()).optional(),
reasoning: z.string()
}),
prompt: `Should this claim be broken into multiple focused searches for better fact-checking?
Claim: "${claim}"`
});
return result.object;
Use the cheapest, most low-latency model you have access to. This will never need anything more.
The generateObject method together with that Zod schema lets us define the shape of data we want from the LLM, every time.
Step 2: Execute said plan — We take that result, and run one or many queries using Google’s search engine via Bright Data.
async function executePlan(claim, decomposition) {
// simple claim - just a single one-and-done SERP API call will do
if (!decomposition.needs_breakdown) {
return await fetchWithBrightDataProxy(claim);
}
// claim needs breakdown into multiple sub-claims - multiple SERP API calls in parallel
try {
const subResults = await Promise.all(
decomposition.sub_queries.map(async (query, index) => {
return await fetchWithBrightDataProxy(query);
})
);
// merge results and return
return {
original_claim: claim,
search_strategy: 'decomposed',
sub_searches: decomposition.sub_queries,
decomposition_reasoning: decomposition.reasoning,
combined_results: mergeSearchResults(subResults),
individual_results: subResults
};
} catch (error) {
// fall back to single search if something went wrong
return await fetchWithBrightDataProxy(claim);
}
}
Fact-checking depends on high-quality, real-time evidence — and the open web is still the best place to find it.
But scraping Google at scale is a pain:
- IPs get rate-limited or blocked.
- Captchas slow things down.
- HTML scraping is brittle and changes constantly.
To solve this, our agent uses Bright Data’s proxy infrastructure to fetch clean, structured SERP data — automatically, reliably, and legally — in fetchWithBrightDataProxy().
async function fetchWithBrightDataProxy(claim) {
try {
// Bright Data proxy URL with credentials and zone config
const proxyUrl = `http://brd-customer-${CONFIG.customerId}-zone-${CONFIG.zone}:${CONFIG.password}@${CONFIG.proxyHost}:${CONFIG.proxyPort}`;
// create proxy agent for HTTPS req
const agent = new HttpsProxyAgent(proxyUrl, {
rejectUnauthorized: false // this'll allow proxy self-signed certs if present
});
// Google search URL with Bright Data JSON enrichment flag
const searchUrl = `https://www.google.com/search?q=${encodeURIComponent(claim)}&brd_json=1`;
// exec proxied fetch to Google Search
const response = await fetch(searchUrl, {
method: 'GET',
agent,
headers: {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
'Accept': 'application/json, text/html, */*',
'Accept-Encoding': 'gzip, deflate, br'
}
});
const responseText = await response.text();
if (!response.ok) {
throw new Error(`Bright Data proxy error: HTTP ${response.status} - ${response.statusText}`);
}
// parse JSON returned by SERP API
try {
const data = JSON.parse(responseText);
return data;
} catch (parseError) {
// uh oh. detect if we were served HTML instead of JSON (proxy config issue)
if (responseText.trim().startsWith('<!DOCTYPE') || responseText.trim().startsWith('<html')) {
throw new Error('Expected JSON but received HTML -- proxy may be misconfigured or blocked');
}
throw new Error('Response body is not valid JSON');
}
} catch (error) {
// propagate error up Perception chain w/ context
throw new Error(`Failed to fetch search results: ${error.message}`);
}
}
fetchWithBrightDataProxy() is the gateway between our agent and the real-time web. It abstracts away the messy world of scraping and returns just the facts — clean, structured, and ready for reasoning. We construct a Google Search query with a special brd_json=1 query param (because we want results as structured JSON, not raw HTML).
Bright Data’s proxy automatically:
- Routes the request through a clean residential IP
- Ensures we get a structured JSON response instead of raw HTML
- Bypasses rate limits, location restrictions, and captchas
const agent = new HttpsProxyAgent(proxyUrl, {
rejectUnauthorized: false
});
We create an HTTPS proxy agent using Bright Data credentials (zone, customer ID, password — all obtained from env vars). This routes the request through their proxy pool.
If all goes well, we receive structured JSON — not raw HTML — containing:
- Organic results
- Knowledge panels
- “People Also Ask” sections
- Other SERP metadata
Without Bright Data, we’d be stuck reverse-engineering brittle HTML from constantly shifting SERP layouts, with Google actively throttling or blocking our requests. There would be no clean or reliable path to scaling our system to handle hundreds or thousands of claims.
Bright Data eliminates those constraints entirely — our Perception phase becomes resilient, deterministic, and repeatable. We get structured, real-time search data that’s ready for downstream reasoning, and we can run the entire pipeline in production without worrying about manually handling CAPTCHAs, geo-restrictions, or IP bans.
Once all these searches are done — whether it was one or several — we need to stitch them together. That’s what mergeSearchResults(subResults) does. We’re not gonna go into detail here because it’s just JS, but you’ll see it in the full code.
Step 3: Clean it.
Finally, we use a buildPerceptionFromSearchData function as our data janitor — we essentially say “I only care about the facts, not the fluff” — it strips out everything except the core content (titles, descriptions, links from organic results, knowledge graph facts, and people-also-ask questions) and restructures it into a clean, predictable format.
Again, this section doesn’t need an explanation, it’s just JS reformatting one object into another structured perception object. Here’s what we get at the end of this stage:
{
"claim": "Vaccines cause autism",
"search_strategy": "decomposed",
"sub_searches": [
"vaccines autism scientific studies",
"vaccine safety children clinical trials"
],
"sources": {
"google_search": { /* cleaned results */ },
"raw_data": { /* unfiltered SERP payload */ }
},
"metadata": {
"organic_results_count": 42,
"people_also_ask_count": 33,
"has_knowledge_graph": true,
"used_plan_decomposition": true
}
}
Reasoning
With a clean snapshot of reality in hand (perception), the next question is: what do we conclude from it?
This is the job of the reason() function — the judgment phase of our agent. It uses a complex, reasoning-focused AI model to:
- Read through the cleaned search data
- Decide if the claim is true, false, misleading, etc.
- Justify its reasoning, and
- Return a confidence score between 0–100
More importantly, the model is sandboxed: it’s told to only use the evidence we gave it — no outside knowledge, no hallucinating.
// use AI to analyze the claim based on gathered Perception, and provide a verdict
async function reason(claim, perception) {
try {
if (!process.env.OPENAI_API_KEY) {
throw new Error('OPENAI_API_KEY environment variable is not set');
}
const searchData = perception.sources.google_search;
const formattedResults = JSON.stringify(searchData, null, 2);
const prompt = `You are a fact-checking assistant. Analyze the given claim based solely on the provided search results.
Claim: "${claim}"
Search Results:
${formattedResults}
Based only on the information in the search results:
1. Determine if the claim is: True, Likely True, Misleading, False, Likely False, or Unverifiable
2. Provide a concise explanation referencing specific search results
3. Include a confidence score (0-100) for your assessment
4. Do not use external knowledge - stick to the provided data
Output in JSON format with "verdict", "explanation", and "confidence" keys.`;
const result = await generateObject({
model: openai('o4-mini'),
schema: z.object({
verdict: z.enum(['True', 'Likely True', 'Misleading', 'False', 'Likely False', 'Unverifiable']),
explanation: z.string(),
confidence: z.number().min(0).max(100).describe('Confidence score from 0-100')
}),
prompt: prompt
});
const reasoning = {
timestamp: new Date().toISOString(),
claim: claim,
verdict: result.object.verdict,
explanation: result.object.explanation,
confidence: result.object.confidence,
sources_analyzed: perception.metadata.organic_results_count,
reasoning_model: 'o4-mini'
};
return reasoning;
} catch (error) {
return {
timestamp: new Date().toISOString(),
claim: claim,
verdict: 'Unverifiable',
explanation: `AI reasoning failed: ${error.message}`,
confidence: 0,
sources_analyzed: perception?.metadata?.organic_results_count || 0,
reasoning_model: 'error-fallback'
};
}
}
LLMs, even advanced ones, still rely on contextual clarity. Giving them well-formatted, indented JSON helps them “see” the data better and reduces misinterpretation.
Our prompt here is the crucial design choice. We tell the AI:
- You are a fact-checker — priming it with a role
- Stick to the data I gave you — sandboxing the response
- Output must follow JSON format — self explanatory.
We can fine-tune this prompt all day if we want, but the bottom line is that it effectively turns the model into a bounded evaluator, not a general chatbot.
Also, we can’t just hope the model replies with a clean response — so we enforce a schema using zod. If the AI response doesn’t match this shape exactly, the call fails.
Finally, we return a result wrapped in metadata for more traceability (also, fallback logic because you don’t want a single misfire to take down the whole analysis chain).
Action
The final phase of our fact-checking agent is act(), which transforms internal reasoning into external action. While earlier phases (perceive and reason) were about collecting and analyzing data, act() is where the system does something with that conclusion.
This Action module is intentionally simple for this tutorial — we only console.log the results — but it lays the groundwork for future extensibility (more on that below).
// take actions based on the Reasoning results (right now, just displaying the analysis)
async function act(claim, perception, reasoning) {
try {
console.log('\nAGENT ANALYSIS COMPLETE:');
console.log('═'.repeat(60));
console.log(`Claim: ${claim}`);
console.log(`Verdict: ${reasoning.verdict}`);
console.log(`Confidence: ${reasoning.confidence}%`);
console.log(`Sources: ${perception.metadata.organic_results_count}`);
console.log(`Explanation: ${reasoning.explanation}`);
console.log('═'.repeat(60));
return {
timestamp: new Date().toISOString(),
claim: claim,
actions_taken: ['display_results'],
status: 'success'
};
} catch (error) {
console.error('\nAGENT ANALYSIS FAILED:', error.message);
return {
timestamp: new Date().toISOString(),
claim: claim,
actions_taken: [],
status: 'failed',
error: error.message
};
}
}
Even though right now we’re just formatting and printing the final analysis to the console in a human-readable way, for the purposes of this tutorial, we return a structured result that describes what the agent did.
This may seem overkill — until you want to:
- Send results to a webhook
- Log them to a database
- Post them in Slack or Discord
- Trigger an alert or follow-up workflow
Our approach separates decision from execution, a principle that makes the system modular, testable, and evolvable. It’s designed to scale up into notifications, reports, or even API responses.
Because we return a typed response with actions_taken, status, and a timestamp, we can chain this phase into automated systems whenever we want to scale this up, without rewriting the function.
Bringing it all together
Again, this project is available at https://github.com/sixthextinction/fact-check-agent under an MIT License. Please feel free to check it out, fork, make improvements, whatever.
I’ll also include the full code here.
// Simple AI Fact-Checking Agent with Plan-and-Execute Engine
const path = require('path');
require('dotenv').config({ path: path.join(__dirname, '../../.env') });
process.env["NODE_TLS_REJECT_UNAUTHORIZED"] = 0;
const { HttpsProxyAgent } = require('https-proxy-agent');
const fetch = require('node-fetch');
const { openai } = require('@ai-sdk/openai');
const { generateObject } = require('ai');
const { z } = require('zod');
// ============================================================================
// CONFIGURATION
// ============================================================================
const CONFIG = {
customerId: process.env.BRIGHT_DATA_CUSTOMER_ID,
zone: process.env.BRIGHT_DATA_ZONE,
password: process.env.BRIGHT_DATA_PASSWORD,
proxyHost: 'brd.superproxy.io',
proxyPort: 33335
};
// ============================================================================
// PLAN-AND-EXECUTE ENGINE
// ============================================================================
// AI decides whether a claim should be broken down into multiple focused searches
async function shouldDecompose(claim) {
try {
if (!process.env.OPENAI_API_KEY) {
console.log('WARNING: OpenAI API key not found, using single search approach');
return { needs_breakdown: false };
}
console.log('PLAN DECOMPOSITION: Analyzing claim complexity...');
const result = await generateObject({
model: openai('gpt-4o-mini'),
schema: z.object({
needs_breakdown: z.boolean(),
sub_queries: z.array(z.string()).optional(),
reasoning: z.string()
}),
prompt: `Should this claim be broken into multiple focused searches for better fact-checking?
Claim: "${claim}"
Consider:
- Does this claim have multiple distinct parts that need separate verification?
- Would different search queries target different aspects more effectively?
- Is this complex enough to benefit from decomposition?
If yes, provide 2-3 specific search queries that target different aspects.
If no, return needs_breakdown: false.
Examples:
- "Vaccines cause autism" → Single search (one claim)
- "Vaccines cause autism and are unsafe for children" → Multiple searches (two distinct claims)
- "Climate policies reduced emissions while maintaining economic growth" → Multiple searches (emissions data, policy effects, economic impact)`
});
console.log(`Plan decision: ${result.object.needs_breakdown ? 'DECOMPOSE' : 'SINGLE SEARCH'}`);
if (result.object.needs_breakdown) {
console.log(`Sub-queries: ${result.object.sub_queries?.length || 0}`);
result.object.sub_queries?.forEach((query, i) => {
console.log(` ${i + 1}. ${query}`);
});
}
return result.object;
} catch (error) {
console.error('ERROR: Plan decomposition failed:', error.message);
return { needs_breakdown: false, reasoning: `Decomposition failed: ${error.message}` };
}
}
// Executes the search plan - either single search or parallel sub-searches based on decomposition
async function executePlan(claim, decomposition) {
console.log('PLAN EXECUTION: Starting search strategy...');
if (!decomposition.needs_breakdown) {
console.log('Executing single search strategy');
return await fetchWithBrightDataProxy(claim);
}
console.log(`Executing parallel search strategy (${decomposition.sub_queries?.length} queries)`);
try {
const subResults = await Promise.all(
decomposition.sub_queries.map(async (query, index) => {
console.log(` Sub-search ${index + 1}: "${query}"`);
return await fetchWithBrightDataProxy(query);
})
);
console.log('SUCCESS: All sub-searches completed, merging results...');
return {
original_claim: claim,
search_strategy: 'decomposed',
sub_searches: decomposition.sub_queries,
decomposition_reasoning: decomposition.reasoning,
combined_results: mergeSearchResults(subResults),
individual_results: subResults
};
} catch (error) {
console.error('ERROR: Parallel search execution failed:', error.message);
console.log('Falling back to single search...');
return await fetchWithBrightDataProxy(claim);
}
}
// Combines multiple search results into a single result set, removing duplicates
function mergeSearchResults(results) {
console.log('Merging search results...');
const merged = {
organic: [],
knowledge: null,
people_also_ask: []
};
results.forEach((result, index) => {
if (result.organic) {
const organicWithSource = result.organic.map(item => ({
...item,
source_query_index: index,
source_query: results.length > 1 ? `sub-search-${index + 1}` : 'main-search'
}));
merged.organic.push(...organicWithSource);
}
});
merged.knowledge = results.find(r => r.knowledge)?.knowledge || null;
results.forEach(result => {
if (result.people_also_ask) {
merged.people_also_ask.push(...result.people_also_ask);
}
});
// Remove duplicates based on links
const seenLinks = new Set();
merged.organic = merged.organic.filter(item => {
if (seenLinks.has(item.link)) return false;
seenLinks.add(item.link);
return true;
});
console.log(`Merged results: ${merged.organic.length} unique organic results`);
return merged;
}
// ============================================================================
// BRIGHT DATA PROXY SEARCH
// ============================================================================
// Fetches Google search results through Bright Data proxy with JSON response
async function fetchWithBrightDataProxy(claim) {
try {
const proxyUrl = `http://brd-customer-${CONFIG.customerId}-zone-${CONFIG.zone}:${CONFIG.password}@${CONFIG.proxyHost}:${CONFIG.proxyPort}`;
const agent = new HttpsProxyAgent(proxyUrl, {
rejectUnauthorized: false
});
const searchUrl = `https://www.google.com/search?q=${encodeURIComponent(claim)}&brd_json=1`;
console.log(`Fetching search results through Bright Data proxy...`);
console.log(`Query: ${claim}`);
const response = await fetch(searchUrl, {
method: 'GET',
agent: agent,
headers: {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
'Accept': 'application/json, text/html, */*',
'Accept-Encoding': 'gzip, deflate, br'
}
});
const responseText = await response.text();
if (!response.ok) {
throw new Error(`HTTP error! Status: ${response.status} - ${response.statusText}`);
}
let data;
try {
data = JSON.parse(responseText);
console.log('SUCCESS: Successfully received structured JSON data');
logSearchResultsSummary(data);
return data;
} catch (parseError) {
if (responseText.trim().startsWith('<!DOCTYPE') || responseText.trim().startsWith('<html')) {
throw new Error('Received HTML instead of JSON - proxy may not be working correctly');
} else {
throw new Error('Response is not valid JSON');
}
}
} catch (error) {
console.error('ERROR: Proxy request failed:', error.message);
throw error;
}
}
// Logs a summary of search results including counts and top results
function logSearchResultsSummary(data) {
const summaryItems = [];
if (data.organic?.length > 0) {
summaryItems.push(`${data.organic.length} organic results`);
}
if (data.ads?.length > 0) {
summaryItems.push(`${data.ads.length} ads`);
}
if (data.knowledge_graph) {
summaryItems.push('knowledge graph data');
}
if (summaryItems.length > 0) {
console.log(`Found: ${summaryItems.join(', ')}`);
}
if (data.organic?.length > 0) {
console.log('Top 3 results:');
data.organic.slice(0, 3).forEach((result, index) => {
console.log(` ${index + 1}. ${result.title || 'No title'}`);
console.log(` ${result.link || 'No link'}`);
});
}
}
// Builds complete perception object directly from raw search results (optimized single-pass)
function buildPerceptionFromSearchData(searchResults, claim, searchStrategy, subSearches) {
// Clean and structure the search data
const cleanedGoogleSearch = {
organic: searchResults.organic?.map(result => ({
title: result.title,
description: result.description,
link: result.link,
display_link: result.display_link
})) || [],
knowledge: searchResults.knowledge ? {
description: searchResults.knowledge.description,
facts: searchResults.knowledge.facts?.map(fact => ({
key: fact.key,
value: fact.value
})) || []
} : null,
people_also_ask: searchResults.people_also_ask?.map(paa => ({
question: paa.question,
answers: paa.answers?.map(answer => ({
text: answer.value?.text
})) || []
})) || []
};
// Return complete perception object in single pass
return {
timestamp: new Date().toISOString(),
claim: claim,
search_strategy: searchStrategy,
sub_searches: subSearches,
sources: {
google_search: cleanedGoogleSearch,
raw_data: searchResults
},
metadata: {
organic_results_count: cleanedGoogleSearch.organic?.length || 0,
has_knowledge_graph: !!cleanedGoogleSearch.knowledge,
people_also_ask_count: cleanedGoogleSearch.people_also_ask?.length || 0,
used_plan_decomposition: searchStrategy === 'decomposed'
}
};
}
// ============================================================================
// AGENT ARCHITECTURE: PERCEPTION-REASONING-ACTION
// ============================================================================
// Gathers and structures environmental data using the Plan-and-Execute engine
async function perceive(claim) {
console.log('PERCEPTION PHASE: Gathering environmental data...');
try {
// STEP 1: PLANNING - AI decides search strategy (single vs decomposed)
// do we need single or multiple focused searches?
const decomposition = await shouldDecompose(claim);
// STEP 2: EXECUTION - Execute the planned search strategy
// perform single search or parallel sub-searches based on decomposition plan
const rawSearchData = await executePlan(claim, decomposition);
// decide on a return format based on that
let actualSearchData;
let searchStrategy = 'single';
let subSearches = null;
if (rawSearchData.search_strategy === 'decomposed') {
// Decomposed search: extract merged results and sub-query metadata
actualSearchData = rawSearchData.combined_results;
searchStrategy = 'decomposed';
subSearches = rawSearchData.sub_searches;
console.log(`Used decomposed search strategy with ${subSearches.length} sub-queries`);
} else {
// Single search: use results directly
actualSearchData = rawSearchData;
console.log('Used single search strategy');
}
// STEP 3: PERCEPTION BUILDING - Transform raw data into structured agent perception
// clean, structure, and enrich data with metadata for the reasoning phase
const perception = buildPerceptionFromSearchData(actualSearchData, claim, searchStrategy, subSearches);
console.log(`SUCCESS: Perception complete: ${perception.metadata.organic_results_count} sources gathered via ${searchStrategy} search`);
return perception;
} catch (error) {
console.error('ERROR: Perception phase failed:', error.message);
throw new Error(`Perception failure: ${error.message}`);
}
}
// Uses AI to analyze the claim and provide a verdict based on gathered evidence
async function reason(claim, perception) {
console.log('REASONING PHASE: AI analysis and decision making...');
try {
if (!process.env.OPENAI_API_KEY) {
throw new Error('OPENAI_API_KEY environment variable is not set');
}
const searchData = perception.sources.google_search;
const formattedResults = JSON.stringify(searchData, null, 2);
const prompt = `You are a fact-checking assistant. Analyze the given claim based solely on the provided search results.
Claim: "${claim}"
Search Results:
${formattedResults}
Based only on the information in the search results:
1. Determine if the claim is: True, Likely True, Misleading, False, Likely False, or Unverifiable
2. Provide a concise explanation referencing specific search results
3. Include a confidence score (0-100) for your assessment
4. Do not use external knowledge - stick to the provided data
Output in JSON format with "verdict", "explanation", and "confidence" keys.`;
console.log('Invoking AI reasoning...');
const result = await generateObject({
model: openai('o4-mini'),
schema: z.object({
verdict: z.enum(['True', 'Likely True', 'Misleading', 'False', 'Likely False', 'Unverifiable']),
explanation: z.string(),
confidence: z.number().min(0).max(100).describe('Confidence score from 0-100')
}),
prompt: prompt
});
const reasoning = {
timestamp: new Date().toISOString(),
claim: claim,
verdict: result.object.verdict,
explanation: result.object.explanation,
confidence: result.object.confidence,
sources_analyzed: perception.metadata.organic_results_count,
reasoning_model: 'o4-mini'
};
console.log(`SUCCESS: Reasoning complete: ${reasoning.verdict} (${reasoning.confidence}% confidence)`);
return reasoning;
} catch (error) {
console.error('ERROR: Reasoning phase failed:', error.message);
return {
timestamp: new Date().toISOString(),
claim: claim,
verdict: 'Unverifiable',
explanation: `AI reasoning failed: ${error.message}`,
confidence: 0,
sources_analyzed: perception?.metadata?.organic_results_count || 0,
reasoning_model: 'error-fallback'
};
}
}
// Takes actions based on the reasoning results, primarily displaying the analysis
async function act(claim, perception, reasoning) {
console.log('ACTION PHASE: Taking actions based on analysis...');
try {
// Simple console display for now
console.log('\nAGENT ANALYSIS COMPLETE:');
console.log('═'.repeat(60));
console.log(`Claim: ${claim}`);
console.log(`Verdict: ${reasoning.verdict}`);
console.log(`Confidence: ${reasoning.confidence}%`);
console.log(`Sources: ${perception.metadata.organic_results_count}`);
console.log(`Search Strategy: ${perception.search_strategy}`);
if (perception.sub_searches) {
console.log(`Sub-queries: ${perception.sub_searches.join(', ')}`);
}
console.log(`Explanation: ${reasoning.explanation}`);
console.log('═'.repeat(60));
return {
timestamp: new Date().toISOString(),
claim: claim,
actions_taken: ['display_results'],
status: 'success'
};
} catch (error) {
console.error('ERROR: Action phase failed:', error.message);
return {
timestamp: new Date().toISOString(),
claim: claim,
actions_taken: [],
status: 'failed',
error: error.message
};
}
}
// Main agent execution cycle that orchestrates the Perception-Reasoning-Action flow
async function agentTick(claim) {
console.log('AGENT TICK: Starting PRA cycle...');
console.log(`Target claim: "${claim}"`);
console.log('─'.repeat(60));
const startTime = Date.now();
try {
const perception = await perceive(claim);
const reasoning = await reason(claim, perception);
const actions = await act(claim, perception, reasoning);
const endTime = Date.now();
const executionTime = endTime - startTime;
const agentResult = {
claim: claim,
execution_time_ms: executionTime,
timestamp: new Date().toISOString(),
phases: {
perception,
reasoning,
actions
},
summary: {
verdict: reasoning.verdict,
confidence: reasoning.confidence,
sources_analyzed: perception.metadata.organic_results_count,
search_strategy: perception.search_strategy
}
};
console.log(`\nAGENT TICK COMPLETE (${executionTime}ms)`);
return agentResult;
} catch (error) {
const endTime = Date.now();
const executionTime = endTime - startTime;
console.error('FATAL: AGENT TICK FAILED:', error.message);
return {
claim: claim,
execution_time_ms: executionTime,
timestamp: new Date().toISOString(),
error: error.message,
status: 'failed'
};
}
}
// ============================================================================
// MAIN EXECUTION
// ============================================================================
// Entry point that initializes and runs the fact-checking agent
async function main() {
try {
console.log('Starting Simple AI Fact-Checking Agent...\n');
console.log('Architecture: Perception-Reasoning-Action (PRA) Pattern');
console.log('Planning: Plan-and-Execute Engine');
console.log('Data: Bright Data Proxy + SERP API (Google)');
console.log('─'.repeat(60));
// claim to test
const testClaim = process.argv[2] || "Vaccines cause autism and are unsafe for children";
console.log(`Processing claim: "${testClaim}"`);
const result = await agentTick(testClaim);
if (result.error) {
console.error('\nFATAL: Agent execution failed:', result.error);
process.exit(1);
} else {
console.log('\nSUCCESS: AI Agent execution completed successfully!');
console.log(`Total execution time: ${result.execution_time_ms}ms`);
console.log(`Final verdict: ${result.summary.verdict} (${result.summary.confidence}% confidence)`);
}
} catch (error) {
console.error('FATAL ERROR:', error.message);
process.exit(1);
}
}
// Allow command line usage: node fact-check-simple.js "Your claim here"
if (require.main === module) {
main();
}
module.exports = { agentTick, perceive, reason, act };
Where to Go From Here?
Bright Data’s SERP API handles data gathering at scale with the scaling pains (proxies, IPs, throttling), so you can focus on the fun part: building smart, reliable AI-powered tools.
Here’s what you can do next:
- Add dynamic Goal/Task handling — To make this useful, you’ll probably want this as an API or Discord/Slack/Social Media bot, reading claims off a queue, with priorities assigned to each claim.
- Smarter claim splitting — Instead of asking the LLM to plan out searches, use
compromiseornlp.jsto detect conjunctions and logical operators, then recursively split using dependency trees. A little more complex, but saves you a good portion of OpenAI costs. - Better confidence scoring — Use a mix of source reliability weights and token-level certainty from
logprobs(in OpenAI Completions) to refine score. This is a pain though, you can weight based on metadata like domain authority or citations using JS — just no out-of-the-box library for that as far as I’m aware. - Add memory — If you want to take past claims into consideration, this is a no-brainer. SQLite for a quick local cache, or a vector DB like Weaviate/Pinecone if you want semantic search.
- Scale it up — Use BullMQ, Agenda, or Bree with Redis to queue and parallelize jobs. I haven’t tested this but I think with this code, I can run concurrent Perception + Reasoning jobs in workers if I wanted to.
- Put it in production — Node.js + Vercel, AWS Lambda, or just Next.js API routes. Deploy as a Slack bot, webhook, or web dashboard.
Whether you’re fact-checking claims, moderating content, or just exploring what’s possible with LLMs, this setup gives you a strong starting point, and the first few thousand queries are basically free — go build something cool.