In today’s rapidly evolving landscape of Artificial Intelligence (AI), the need for robust, reliable, and transparent AI agents is paramount, particularly in enterprise environments. Langfuse, an open-source LLM (Large Language Model) engineering platform, emerges as a crucial solution, offering observability, tracing, and monitoring capabilities essential for enterprise use cases such as compliance tracking. In sectors where adherence to regulations and data governance is non-negotiable, the ability to monitor and validate every decision made by an AI agent is critical.
One of the significant challenges in building a compliance-focused AI agent using frameworks like LangChain is ensuring unrestricted access to authoritative web data. This includes real-time regulatory updates, government guidelines, and other crucial information necessary to maintain compliance. The reliability of these AI agents heavily depends on their ability to gather and process data accurately and consistently from various online sources.
IPFLY’s premium proxy solutions provide a comprehensive solution to this challenge. With a vast network of over 90 million global IPs spanning across more than 190 countries, and offering static and dynamic residential, as well as data center proxies, IPFLY ensures seamless and reliable access to web data. The multi-layer IP filtering technology effectively bypasses anti-scraping measures, ensuring that AI agents can access even the most heavily guarded websites. The global coverage provided by IPFLY unlocks access to region-specific compliance data, allowing businesses to stay compliant with local regulations worldwide. Furthermore, the 99.9% uptime guarantee ensures consistent data ingestion, critical for real-time compliance monitoring and updates.
This guide will walk you through the process of building a LangChain compliance AI agent, integrating IPFLY for efficient and reliable web data collection, and leveraging Langfuse to trace every step of the process. From the initial prompt inputs to the final stages of proxy-powered web scraping, you’ll gain insight into how these technologies can be combined to create a robust and transparent AI solution.
Introduction to Langfuse, AI Agent Observability, and IPFLY’s Role
Enterprise AI agents, particularly those focused on compliance tracking, rely on two fundamental pillars: accurate web data and full observability. Accurate web data is crucial for staying up-to-date with ever-changing regulations like GDPR (General Data Protection Regulation) and CCPA (California Consumer Privacy Act). Full observability is essential for validating decision-making processes, tracking operational costs, and ensuring consistent compliance with internal and external regulations.
Langfuse plays a vital role by providing end-to-end tracing, metrics, and debugging capabilities for LLM applications. This allows development teams to monitor every step of an AI agent’s workflow, from the initial prompts and tool calls to the final responses. With Langfuse, teams can gain deep insights into the performance and behavior of their AI agents.
LangChain serves as the orchestrator of AI agent logic, connecting LLMs to external tools such as web scrapers for efficient data retrieval. It provides a structured framework for building complex AI workflows and ensures that different components of the system work seamlessly together.
IPFLY addresses the critical issue of web data access bottlenecks by offering a suite of proxy solutions specifically designed for AI applications. Dynamic residential proxies mimic real user behavior, making it difficult for websites to detect and block scraping attempts. Static residential proxies ensure consistent access to trusted regulatory sites, which is crucial for maintaining compliance. Data center proxies handle large-scale scraping tasks efficiently, providing the necessary bandwidth and speed. With global coverage, IPFLY ensures access to region-specific compliance data, enabling businesses to stay compliant with regulations worldwide.
Together, Langfuse, LangChain, and IPFLY create an enterprise-ready stack that addresses the key challenges of building and deploying AI agents for compliance. IPFLY provides the fuel by supplying high-quality web data, LangChain manages the workflow, and Langfuse ensures complete visibility into performance and reliability.
What Is Langfuse?
Langfuse is an open-source, cloud-native platform designed for LLM application development and monitoring. It empowers development teams with a comprehensive set of tools to build, deploy, and maintain high-performing AI applications.
Key features of Langfuse include:
- Trace workflows: Track every step of AI agent runs, including prompt inputs, tool calls, LLM outputs, latency, and associated costs. This detailed tracing provides valuable insights into the inner workings of AI agents.
- Manage prompts: Version-control prompts collaboratively without the need to edit code. This allows teams to experiment with different prompts and optimize the performance of their AI agents.
- Evaluate performance: Collect human feedback, run automated tests, and score the accuracy of AI agents. This data-driven approach ensures continuous improvement and helps teams identify areas for optimization.
- Collaborate: Annotate traces, add comments, and share insights across teams. Langfuse facilitates seamless collaboration, allowing team members to share knowledge and work together effectively.
- Deploy flexibly: Use the hosted cloud service (with a free tier available) or self-host for full data control. This provides flexibility to choose the deployment option that best suits the needs of the organization.
For compliance AI agents, Langfuse’s tracing capabilities are invaluable. It creates a comprehensive audit trail of how the agent sourced web data (via IPFLY) and arrived at regulatory insights, simplifying compliance with both internal governance policies and external regulations. This audit trail provides clear evidence of the agent’s decision-making process, which is essential for demonstrating compliance to regulators and stakeholders.
Why Integrate Langfuse Into Your AI Agent
AI agents designed for compliance often interact with sensitive documents, external web data, and complex regulatory rules. Blind spots in these interactions can lead to costly mistakes or non-compliance. Langfuse addresses this challenge by providing the following benefits:
- Providing end-to-end tracing: Monitor every tool call (e.g., IPFLY web scrapes) and data source to validate insights. This ensures that all data used by the AI agent is reliable and up-to-date.
- Tracking key metrics: Measure latency, LLM costs, and web scraping success rates, which is critical for optimizing IPFLY proxy usage. By monitoring these metrics, teams can identify potential bottlenecks and optimize the performance of their AI agents.
- Enabling fast debugging: Identify failed scrapes, outdated prompts, or LLM hallucinations with detailed logs. This enables quick resolution of issues and ensures that the AI agent is functioning correctly.
- Supporting compliance: Create immutable records of agent behavior for audits. These records provide a clear and auditable history of the AI agent’s actions, which is essential for demonstrating compliance.
When paired with IPFLY, Langfuse ensures not only that your agent functions correctly but also that you can reliably and lawfully prove its reliability and adherence to regulations. This is particularly important in highly regulated industries where compliance is paramount.
How to Use Langfuse to Trace a Compliance-Tracking AI Agent (LangChain + IPFLY)
In this section, we will guide you through the process of building an enterprise-grade compliance AI agent that:
- Loads internal PDF documents (e.g., data processing workflows).
- Analyzes the PDF to identify privacy and regulatory risks.
- Uses IPFLY proxies to search for updated regulations (SERP data) and scrape authoritative sources (government sites).
- Generates a comprehensive compliance report with citations from both internal documents and web data.
- Integrates Langfuse for full workflow tracing, providing complete visibility into the agent’s operations.
Prerequisites
Before starting, ensure you have the following:
- Python 3.10 or higher installed on your system.
- An OpenAI API key (or an API key from another LLM provider).
- An active IPFLY account with API key and access to dynamic residential proxies.
- A Langfuse account with configured public and secret API keys.
- Basic familiarity with LangChain and Python programming.
Step #1: Set Up Your LangChain AI Agent Project
First, create a project folder and a virtual environment:
mkdir compliance-ai-agent-ipfly-langfuse
cd compliance-ai-agent-ipfly-langfuse
python -m venv .venv
# Activate: macOS/Linux → source .venv/bin/activate; Windows → .venv\Scripts\activate
pip install langchain langchain-openai langgraph langchain-community pypdf python-dotenv langfuse requests
Next, create two files: agent.py (which will contain the core logic of the agent) and .env (for storing credentials securely):
compliance-ai-agent-ipfly-langfuse/
├── .venv/
├── agent.py
└── .env
Step #2: Configure Environment Variable Reading
In agent.py, load environment variables to securely store sensitive credentials:
from dotenv import load_dotenv
load_dotenv() # Loads variables from .env file
Add the necessary credentials to your .env file. You will need to populate the IPFLY, Langfuse, and OpenAI keys in later steps:
OPENAI_API_KEY=""
IPFLY_API_KEY=""
IPFLY_PROXY_ENDPOINT="http://[USERNAME]:[PASSWORD]@proxy.ipfly.com:8080"
LANGFUSE_SECRET_KEY=""
LANGFUSE_PUBLIC_KEY=""
LANGFUSE_BASE_URL=""
Step #3: Prepare Your IPFLY Account
IPFLY will power the agent’s web data collection by enabling SERP searches and regulatory site scraping. Here’s how to configure it:
- Log into your IPFLY account and generate an API key (under “Account Settings”).
- Note your proxy endpoint (provided in IPFLY’s dashboard), which includes your username, password, and port.
- For compliance use cases, select dynamic residential proxies (to avoid blocks on government/regulatory sites) or static residential proxies (for consistent access to trusted sources).
IPFLY offers several key benefits for this agent:
- 90M+ real-user IPs: Mimic human browsing to bypass anti-scraping tools, such as CAPTCHAs on GDPR.eu.
- 190+ country coverage: Scrape region-specific regulations, such as CCPA for California and GDPR for the EU.
- Multi-layer IP filtering: Ensures no blacklisted IPs are used, maintaining compliance with data collection rules.
- 99.9% uptime: Guarantees consistent access to critical regulatory data.
Step #4: Build IPFLY Tools for LangChain
Create custom LangChain tools to handle SERP searches and web scraping using IPFLY proxies. Add these to agent.py:
import requests
from bs4 import BeautifulSoup
from langchain.tools import Tool
class IPFLYSERPTool(Tool):
"""Tool to retrieve SERP data using IPFLY proxies for regulatory search queries."""
def __init__(self):
super().__init__(
name="ipfly_serp_search",
description="Searches Google for regulatory keywords (e.g., 'GDPR data retention') using IPFLY proxies. Returns top 5 search results (prioritizes government sites).",
func=self.run
)
self.proxy = os.getenv("IPFLY_PROXY_ENDPOINT")
def run(self, query: str) -> str:
"""Run SERP search with IPFLY proxy."""
params = {"q": query, "hl": "en", "gl": "us"} # Customize for regional regulations (e.g., "eu" for GDPR)
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"}
try:
response = requests.get("https://www.google.com/search",
params=params,
proxies={"http": self.proxy, "https": self.proxy},
headers=headers,
timeout=30)
response.raise_for_status()
soup = BeautifulSoup(response.text, "html.parser")
results = []
# Extract top 5 organic results (prioritize gov sites)
for g in soup.find_all("div", class_="g")[:5]:
title = g.find("h3").get_text(strip=True) if g.find("h3") else None
url = g.find("a")["href"] if g.find("a") else None
if title and url and ("gov" in url or "regulatory" in url):
results.append({"title": title, "url": url})
return json.dumps(results, indent=2)
except Exception as e:
return f"SERP search failed: {str(e)}"
class IPFLYWebScraperTool(Tool):
"""Tool to scrape regulatory sites using IPFLY proxies (returns Markdown-formatted content)."""
def __init__(self):
super().__init__(
name="ipfly_web_scraper",
description="Scrapes content from regulatory websites (e.g., government sites) using IPFLY proxies. Returns clean, Markdown-formatted text for LLM analysis.",
func=self.run
)
self.proxy = os.getenv("IPFLY_PROXY_ENDPOINT")
def run(self, url: str) -> str:
"""Scrape web page with IPFLY proxy."""
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"}
try:
response = requests.get(
url,
proxies={"http": self.proxy, "https": self.proxy},
headers=headers,
timeout=30)
response.raise_for_status()
soup = BeautifulSoup(response.text, "html.parser")
# Extract main content (remove ads/navigation)
for script in soup(["script", "style", "nav", "aside", "footer"]):
script.decompose()
text = soup.get_text(strip=True, separator="\n")
# Convert to Markdown (simplified)
lines = [line.strip() for line in text.split("\n") if line.strip()]
markdown = "\n\n".join(lines[:50]) # Limit to 50 lines for LLM context
return f"Source: {url}\n\n{markdown}"
except Exception as e:
return f"Web scraping failed: {str(e)}"
# Initialize IPFLY tools
ipfly_serp_tool = IPFLYSERPTool()
ipfly_scraper_tool = IPFLYWebScraperTool()
Step #5: Integrate the LLM
Add OpenAI (or your preferred LLM) to agent.py to power the agent’s analysis:
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
model="gpt-5-mini", # Replace with your LLM (e.g., gpt-4o)
api_key=os.getenv("OPENAI_API_KEY"))
Step #6: Define the Compliance AI Agent
Combine the LLM, IPFLY tools, and a system prompt to create the agent. Add this to agent.py:
from langchain.agents import create_agent
from langchain_core.prompts import PromptTemplate
# System prompt for compliance tracking
system_prompt = """
You are a compliance-tracking expert. Your role is to analyze internal documents for privacy/regulatory risks and validate findings with updated web data (via IPFLY proxies).
Follow these rules:
1. Analyze the input PDF to identify key regulatory aspects (e.g., data retention, deletion).
2. Generate 2-3 concise SERP queries (max 5 words) to find updated regulations.
3. Use ipfly_serp_search to get top regulatory sites (prioritize government sources).
4. Use ipfly_web_scraper to extract content from those sites.
5. Create a report with:
- Quotes from the internal PDF.
- Insights from scraped web data.
- Clear compliance recommendations.
6. Only use data from IPFLY-scraped sources and the input PDF—never make up information.
"""
# List of tools (IPFLY + LLM)
tools = [ipfly_serp_tool, ipfly_scraper_tool]
# Create the agent (LangGraph-powered)
agent = create_agent(
llm=llm,
tools=tools,
system_prompt=system_prompt
)
Step #7: Launch the Agent (Load PDF & Create Prompt)
Add logic to load internal PDF documents and generate a prompt for the agent. Add this to agent.py:
from langchain_community.document_loaders import PyPDFDirectoryLoader
import os
# Create input folder for PDFs
os.makedirs("./input", exist_ok=True)
# Load PDF documents
loader = PyPDFDirectoryLoader("./input")
docs = loader.load()
internal_doc = "\n\n".join([doc.page_content for doc in docs])
# Prompt template for the agent
prompt_template = PromptTemplate.from_template("""
Analyze the following internal document for compliance risks and validate with updated web data:
PDF CONTENT:
{pdf}
Generate a concise compliance report with quotes from the PDF and scraped regulatory insights.
""")
# Create final prompt
prompt = prompt_template.format(pdf=internal_doc)
Step #8: Set Up Langfuse for Observability
- Create a Langfuse account (free tier available) and navigate to “Project Settings” → “API Keys.”
- Generate public and secret keys and add them to your
.envfile (as shown in Step #2).
Step #9: Integrate Langfuse Tracking
Add Langfuse tracing to the agent to monitor every step (tool calls, LLM outputs, latency). Update agent.py:
from langfuse import get_client
from langfuse.langchain import CallbackHandler
# Initialize Langfuse client
langfuse = get_client(
secret_key=os.getenv("LANGFUSE_SECRET_KEY"),
public_key=os.getenv("LANGFUSE_PUBLIC_KEY"),
base_url=os.getenv("LANGFUSE_BASE_URL"))
# Create Langfuse callback handler
langfuse_handler = CallbackHandler()
Step #10: Final Code
Your complete agent.py file will look like this:
import os
import json
import requests
from dotenv import load_dotenv
from bs4 import BeautifulSoup
from langchain.tools import Tool
from langchain_openai import ChatOpenAI
from langchain.agents import create_agent
from langchain_core.prompts import PromptTemplate
from langchain_community.document_loaders import PyPDFDirectoryLoader
from langfuse import get_client
from langfuse.langchain import CallbackHandler
# Load environment variables
load_dotenv()
# ------------------------------
# Langfuse Setup
# ------------------------------
langfuse = get_client(
secret_key=os.getenv("LANGFUSE_SECRET_KEY"),
public_key=os.getenv("LANGFUSE_PUBLIC_KEY"),
base_url=os.getenv("LANGFUSE_BASE_URL"))
langfuse_handler = CallbackHandler()
# ------------------------------
# IPFLY Tools for LangChain
# ------------------------------
class IPFLYSERPTool(Tool):
def __init__(self):
super().__init__(
name="ipfly_serp_search",
description="Searches Google for regulatory keywords using IPFLY proxies. Returns top 5 government/regulatory sites.",
func=self.run
)
self.proxy = os.getenv("IPFLY_PROXY_ENDPOINT")
def run(self, query: str) -> str:
params = {"q": query, "hl": "en", "gl": "us"}
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"}
try:
response = requests.get("https://www.google.com/search",
params=params,
proxies={"http": self.proxy, "https": self.proxy},
headers=headers,
timeout=30)
response.raise_for_status()
soup = BeautifulSoup(response.text, "html.parser")
results = []
for g in soup.find_all("div", class_="g")[:5]:
title = g.find("h3").get_text(strip=True) if g.find("h3") else None
url = g.find("a")["href"] if g.find("a") else None
if title and url and ("gov" in url or "regulatory" in url):
results.append({"title": title, "url": url})
return json.dumps(results, indent=2)
except Exception as e:
return f"SERP search failed: {str(e)}"
class IPFLYWebScraperTool(Tool):
def __init__(self):
super().__init__(
name="ipfly_web_scraper",
description="Scrapes regulatory sites with IPFLY proxies. Returns Markdown-formatted content.",
func=self.run
)
self.proxy = os.getenv("IPFLY_PROXY_ENDPOINT")
def run(self, url: str) -> str:
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"}
try:
response = requests.get(
url,
proxies={"http": self.proxy, "https": self.proxy},
headers=headers,
timeout=30)
response.raise_for_status()
soup = BeautifulSoup(response.text, "html.parser")
for script in soup(["script", "style", "nav", "aside", "footer"]):
script.decompose()
text = soup.get_text(strip=True, separator="\n")
lines = [line.strip() for line in text.split("\n") if line.strip()]
markdown = "\n\n".join(lines[:50])
return f"Source: {url}\n\n{markdown}"
except Exception as e:
return f"Web scraping failed: {str(e)}"
# Initialize IPFLY tools
ipfly_serp_tool = IPFLYSERPTool()
ipfly_scraper_tool = IPFLYWebScraperTool()
# ------------------------------
# LLM Integration
# ------------------------------
llm = ChatOpenAI(
model="gpt-5-mini",
api_key=os.getenv("OPENAI_API_KEY"))
# ------------------------------
# Compliance AI Agent Definition
# ------------------------------
system_prompt = """
You are a compliance-tracking expert. Analyze internal PDFs for regulatory risks and validate with IPFLY-scraped web data.
1. Identify key privacy/regulatory aspects from the PDF.
2. Generate 2-3 concise SERP queries (max 5 words).
3. Use ipfly_serp_search to find top government/regulatory sites.
4. Use ipfly_web_scraper to extract content from those sites.
5. Create a report with PDF quotes, web insights, and compliance recommendations.
Only use PDF and IPFLY-scraped data—no made-up information.
"""
tools = [ipfly_serp_tool, ipfly_scraper_tool]
agent = create_agent(llm=llm, tools=tools, system_prompt=system_prompt)
# ------------------------------
# Load PDF & Create Prompt
# ------------------------------
os.makedirs("./input", exist_ok=True)
loader = PyPDFDirectoryLoader("./input")
docs = loader.load()
internal_doc = "\n\n".join([doc.page_content for doc in docs])
prompt_template = PromptTemplate.from_template("""
Analyze this internal document for compliance risks and validate with web data:
PDF CONTENT:
{pdf}
Generate a compliance report with PDF quotes and scraped regulatory insights.
""")
prompt = prompt_template.format(pdf=internal_doc)
# ------------------------------
# Run Agent with Langfuse Tracing
# ------------------------------
if __name__ == "__main__":
print("Running compliance AI agent with Langfuse tracing...")
for step in agent.stream({"messages": [{"role": "user", "content": prompt}]},
stream_mode="values",
config={"callbacks": [langfuse_handler]}):
step["messages"][-1].pretty_print()
Step #11: Run the Agent
- Place a compliance-related PDF (e.g.,
data-processing-workflow.pdf) in the./inputfolder. - Execute the agent by running
python agent.pyin your terminal.
The agent will then:
- Analyze the PDF to identify regulatory risks (e.g., “data retention”).
- Use IPFLY’s SERP tool to search for updated rules (e.g., “GDPR data retention”).
- Scrape top government sites (e.g., europa.eu) using IPFLY’s web scraper.
- Generate a comprehensive compliance report with relevant citations.
- Langfuse will automatically track every step, from IPFLY proxy calls to LLM outputs, providing complete observability.
Step #12: Inspect Agent Traces in Langfuse
- Log into your Langfuse dashboard.
- Navigate to the “Tracing” tab to see a new trace for your agent run.
- Click the trace to explore the following:
- Tool Calls: View IPFLY SERP and scraper requests, including proxy usage and response data.
- LLM Interactions: Inspect prompts, outputs, and latency.
- Metrics: Track scraping success rates, LLM costs, and total runtime.
Key insights you can gain from Langfuse include:
- Verify IPFLY proxy performance (e.g., 100% success rate for scraping government sites).
- Identify bottlenecks (e.g., latency in SERP searches, which could be addressed by adjusting the IPFLY proxy type to a data center proxy for increased speed).
- Audit compliance (e.g., confirm that the agent only used IPFLY-scraped government data).
Next Steps to Enhance the Agent
- Prompt Management: Utilize Langfuse’s prompt library to version-control compliance prompts, ensuring consistency and accuracy.
- Custom Langfuse Dashboards: Track IPFLY proxy success rates, LLM costs, and the overall quality of compliance reports generated by the agent.
- IPFLY Proxy Optimization: Employ static residential proxies for recurring scrapes (e.g., monthly GDPR updates) to enhance consistency and reliability.
- Report Export: Add logic to save compliance reports as PDFs for easy sharing and auditing purposes.
- Multi-Region Support: Leverage IPFLY’s regional IPs to scrape regulations for multiple countries, such as CCPA for the US and PIPEDA for Canada, ensuring global compliance coverage.
Integrating Langfuse with a LangChain AI agent powered by IPFLY proxies offers enterprise-grade observability and reliability, which is critical for compliance use cases. Langfuse provides the transparency needed to track every agent action, while IPFLY ensures unrestricted access to high-quality regulatory data. Together, these tools address the most significant challenges of building and deploying enterprise AI agents:
- Data Access: IPFLY’s extensive network of over 90 million global proxies bypasses blocks and geo-restrictions, ensuring access to the data needed.
- Observability: Langfuse traces every step of the process, providing comprehensive audit trails and facilitating optimization efforts.
- Compliance: Immutable records of data sources and agent logic ensure adherence to regulatory requirements and provide clear evidence of compliance.
Whether you’re building compliance agents, market research tools, or customer support bots, the combination of IPFLY, Langfuse, and LangChain creates a powerful, transparent, and scalable stack that can meet the demands of any enterprise AI application.
Ready to build your own observable AI agent? Start with IPFLY’s free trial and Langfuse’s free tier, and use the code provided in this guide to unlock the full potential of web data for enterprise AI applications. With these tools at your disposal, you can create AI agents that are not only powerful but also transparent, reliable, and compliant.