Multi AI Agent Market Segmentation

Multi AI Agent Market Segmentation

Strategic Context

The VP of Engineering at Act-On Software asked me to investigate potential applications of AI agents within the company's product suite. As the data scientist with expertise in prototyping emerging technologies, I was tasked with identifying high-value opportunities and validating technical feasibility.

After initial prototyping of an email sequencing agent that didn't align with the current product roadmap, I shifted focus to the Segmentation Engine 2.0 development. Product had identified a competitive threat: a competitor's natural language interface for customer segmentation. Our existing form-based approach required users to navigate complex, time-consuming forms.

Problem Diagnosis

Analysis of the existing segmentation forms revealed several structural challenges:

Form Complexity: The interface required users to specify multiple dimensions of customer behavior and attributes across numerous fields, creating significant friction in the segmentation workflow.

Term Expansion Requirements: Users wanted to specify geographic regions ("Pacific Northwest"), industries ("medical"), or job levels ("C-suite executives") using natural language, but contact databases contained specific values like state names (Washington, Oregon), city names (Seattle, Portland), and exact titles (CEO, CTO, COO). The system needed to expand broad user queries into the specific database values.

Data Quality Issues: Contact lists contained widespread misspellings and inconsistent entries. Exact string matching would exclude valid customer records, undermining segmentation accuracy.

Dual Classification Problem: Some query terms were "fuzzy" (industry, location, title) requiring expansion and approximate matching, while others were "non-fuzzy" (opened 5 emails in last 3 months, filled out form in last month) requiring exact interpretation of behavioral criteria.

The technical challenge: build a system that could parse natural language queries, distinguish between fuzzy and non-fuzzy terms, expand fuzzy terms appropriately while handling misspellings, and generate structured output compatible with the segmentation engine under development.

Architecture Experimentation

Initial approach used a single LLM with comprehensive prompts to handle all aspects of query parsing and form completion. This proved fragile - the prompts became increasingly complex as we attempted to handle the full range of term types and edge cases through prompt engineering alone.

Next iteration used a single agent architecture. Performance remained inconsistent because a monolithic agent struggled to apply different reasoning strategies for fundamentally different term types.

The multi-agent architecture emerged from recognizing that fuzzy and non-fuzzy terms represented distinct computational problems requiring specialized handling:

  • Parser Agent: Identify and classify terms in the user query as fuzzy, non-fuzzy, or irrelevant
  • Fuzzy Agent: Expand geographic, industry, and demographic terms while handling misspellings via vector similarity search
  • Non-Fuzzy Agent: Parse behavioral criteria into exact specifications for the segmentation engine
  • Aggregation Agent: Combine outputs into valid JSON for the UI

This separation allowed each agent to use reasoning patterns appropriate to its problem domain, reducing prompt complexity and improving reliability.

Technical and Business Constraints

Vector Database Scoping: Initial design included all contact fields in the vector database for fuzzy matching. Cost analysis revealed this approach would create prohibitive storage costs. Worked with product to identify which fields users actually searched on versus fields that existed but were never queried (e.g., middle name). This analysis reduced the vector database scope to high-value fields, making the solution economically viable.

Timeline Management: Product needed a testable prototype within a specific development window to coordinate with the Segmentation Engine 2.0 release. Negotiated limiting the initial version to core fields, establishing a path to expand functionality in subsequent iterations once the core architecture was validated.

Integration Requirements: The segmentation engine team required specific JSON output formats. Coordinated with their development to ensure the aggregation agent produced compatible structured data while they built the UI components to consume it.

Outcome

Deployed a functional prototype to the development environment for product testing. The day after deployment, the company underwent a 10% workforce reduction due to a failed acquisition, and the project was canceled for funding reasons. The prototype successfully demonstrated the technical approach - parsing natural language queries into structured segmentation criteria while handling the dual challenges of term expansion and data quality issues.

Development Environment

  • Python
  • LangGraph
  • LangChain
  • OpenAI API
  • psycopg
  • PostgreSQL
  • pgvector
  • Snowflake
  • Docker
  • Git
  • Bitbucket
  • Jupyter Lab
  • Visual Studio Code
  • ChatGPT
  • Claude