Building Autonomous Lead Enrichment Agents in Python

Growth teams are caught in an eternal tug-of-war. Marketing wants short, 2-field lead forms (Work Email + Website URL) to maximize conversion rates. Sales teams demand deep qualification data: Company Size, Annual Revenue, Current Tech Stack, Primary Pain Point, and Budget before taking a discovery call.

Forcing prospects to fill out lengthy 10-field surveys crushes landing page conversion rates. But handing reps empty email addresses wastes hours of manual LinkedIn and Google searching. The solution is an autonomous background enrichment agent.

1. The Friction Dilemma: Form Length vs. Sales Intelligence

Every additional form field beyond email and name reduces conversion rates by an average of 5% to 8%. However, unqualified leads flood sales calendars with tire-kickers.

By letting the user submit only their work email and domain name, an autonomous Python microservice can execute real-time scraping, DNS inspection, and LLM synthesis in the background within seconds.

2. Architectural Blueprint: Async Python Scraping & LLM Extraction

The enrichment pipeline operates as an asynchronous event-driven worker:

• Trigger: Inbound webhook from the landing page form or CRM.
• Domain Extraction: Derives the root company domain from the work email (e.g., john@acme.com → acme.com).
• Headless Web Scrape: An async HTTP client fetches the company's homepage, About Us page, and Pricing page text.
• Structured LLM Analysis: An LLM analyzes the text against predefined criteria to produce validated, typed JSON.
• CRM Ingestion: Updates the CRM contact custom fields and calculates an ICP fit score from 1 to 100.

3. Defining Type-Safe Pydantic Schemas

Using OpenAI's Structured Outputs with Pydantic ensures the LLM output is strictly validatable without runtime JSON decoding errors:

from pydantic import BaseModel, Field
from typing import List, Optional

class CompanyEnrichment(BaseModel):
    company_name: str
    industry: str
    core_value_proposition: str
    estimated_team_size: str = Field(description="e.g. 1-10, 11-50, 51-200, 200+")
    detected_technologies: List[str]
    b2b_or_b2c: str = Field(description="'B2B', 'B2C', or 'Hybrid'")
    icp_score: int = Field(description="Fit score from 0 to 100 based on B2B software fit")
    summary_for_rep: str = Field(description="2-sentence briefing for the sales rep")

4. Core Python Pipeline Implementation

Here is the core async enrichment pipeline using httpx and the OpenAI API:

import httpx
from openai import OpenAI

client = OpenAI()

async def enrich_company(domain: str) -> CompanyEnrichment:
    url = f"https://{domain}"
    async with httpx.AsyncClient(timeout=10.0, follow_redirects=True) as http_client:
        response = await http_client.get(url)
        html_snippet = response.text[:8000] # Clean text extraction

    completion = client.beta.chat.completions.parse(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are a senior B2B sales intelligence agent."},
            {"role": "user", "content": f"Analyze this company website text for {domain}:\n\n{html_snippet}"}
        ],
        response_format=CompanyEnrichment,
    )
    return completion.choices[0].message.parsed

5. Wiring the Agent to GoHighLevel or HubSpot

Once the Pydantic model parses successfully, the agent makes a PUT /contacts/{id} call to your CRM API:

• Fills custom fields: Industry, Tech Stack, and Estimated Team Size.
• Posts an internal pinned note with the summary_for_rep directly on the lead card.
• If icp_score >= 80, instantly alerts the enterprise account executive via Slack.
• If icp_score < 50, routes the lead to a self-serve automated nurture sequence without booking rep time.