Case Study: Multi-Agent Collaborative Content Creation System

Chapter 73: Case Study — Multi-Agent Collaborative Content Creation System

Chapter Introduction

Truly excellent content is never just "a good article." It requires accurate information, compelling narrative, precise SEO layout, platform-adapted distribution formats, and rigorous editorial review. A single person juggling all of this inevitably makes tradeoffs; a team juggling it incurs heavy coordination overhead. This chapter designs a multi-agent collaborative content creation system where four specialized agents — Researcher, Writer, Editor, SEO Expert — collaborate under an Orchestrator Agent. Given a topic, the system automatically produces: a complete article + image generation prompts + SEO metadata + social media versions.

73.1 Requirements & System Goals

Traditional vs. Automated Workflow

Traditional content creation (manual):
Topic → Research (1-2 days) → Write (2-4 hrs) → Edit (1-2 hrs)
→ SEO (30 min) → Social adaptation (30 min)
Total: ~1.5-2 days

Target with multi-agent automation:
Total: ~15-30 min (including quality score ≥ 80 iterations)

Output Specification

73.2 Multi-Agent Architecture

Agent Role Design

┌─────────────────────────────────────────────────────────────┐
│                     Orchestrator Agent                       │
│  (Task scheduling + Quality gating + Iteration control)      │
│  Responsible for: decompose → assign → aggregate → score     │
└────────────────────────────┬────────────────────────────────┘
                             │ coordinates
          ┌──────────────────┼──────────────────┐
          ▼                  ▼                  ▼
┌─────────────────┐ ┌─────────────────┐ ┌──────────────────┐
│  Researcher     │ │  Writer         │ │  Editor          │
│  Agent          │ │  Agent          │ │  Agent           │
│                 │ │                 │ │                  │
│ - Background    │ │ - Draft article │ │ - Fact-checking  │
│ - Key facts     │ │ - Narrative arc │ │ - Logic review   │
│ - Data points   │ │ - Case stories  │ │ - Style polish   │
│ - Keyword dig   │ │ - Image prompts │ │ - Edit feedback  │
└─────────────────┘ └─────────────────┘ └──────────────────┘
                              │
                              ▼
                   ┌─────────────────┐
                   │  SEO Expert     │
                   │  Agent          │
                   │                 │
                   │ - Keyword opt   │
                   │ - Meta tags     │
                   │ - Social rewrites│
                   │ - Publish strat │
                   └─────────────────┘

Data Flow

[Input: Topic]
    ↓
[Researcher] → Output: Research Brief (facts, data, angles, keywords)
    ↓
[Writer] reads brief → Output: Draft article + image prompts
    ↓
[Editor] reads draft → Output: Edit notes + revised article
    ↓
[SEO Expert] reads revision → Output: SEO metadata + social versions
    ↓
[Orchestrator] Quality evaluation →
    Score < 80 → request revisions
    Score ≥ 80 → output final content package

73.3 Full Implementation

Core Data Models

# content_system/models.py
from dataclasses import dataclass, field
from typing import List, Optional

@dataclass
class ResearchBrief:
    topic: str
    background: str
    key_facts: List[str]
    statistics: List[dict]      # [{data, source}]
    target_keywords: List[str]
    competitor_analysis: str
    recommended_angle: str
    sources: List[str]

@dataclass
class ContentPackage:
    topic: str
    article: str
    seo_meta: dict
    image_prompts: List[dict]
    twitter_thread: List[str]
    linkedin_post: str
    quality_report: dict
    generation_time_seconds: float

Orchestrator Agent

# content_system/orchestrator.py
import os, json, time
from openai import OpenAI

client = OpenAI(
    base_url=os.getenv("HERMES_BASE_URL", "http://localhost:11434/v1"),
    api_key=os.getenv("HERMES_API_KEY", "ollama"),
)
MODEL = os.getenv("HERMES_MODEL", "nous-hermes-2-mixtral-8x7b-dpo")

ORCHESTRATOR_PROMPT = """You are the coordinator of a multi-agent content creation system.

Your responsibilities:
1. Receive the topic and plan the creation workflow
2. Call specialized agents in sequence: Researcher → Writer → Editor → SEO Expert
3. Evaluate each stage's output quality
4. If quality < 80/100, request revisions from the relevant agent
5. Compile the final content package

Quality scoring (100 points total):
- Information accuracy (25): facts supported by sources
- Narrative flow (20): engaging, logical progression
- Original value (20): unique angle and insights
- SEO readiness (15): natural keyword integration
- Reader value (20): genuinely helps the reader

Minimum publish threshold: total ≥ 80, accuracy ≥ 20"""

ORCHESTRATOR_TOOLS = [
    {"type": "function", "function": {
        "name": "run_researcher",
        "description": "Invoke the Researcher agent to gather topic information",
        "parameters": {"type": "object",
            "properties": {"topic": {"type": "string"},
                          "research_focus": {"type": "array", "items": {"type": "string"}}},
            "required": ["topic"]}
    }},
    {"type": "function", "function": {
        "name": "run_writer",
        "description": "Invoke the Writer agent to draft the article",
        "parameters": {"type": "object",
            "properties": {"research_brief": {"type": "object"},
                          "style": {"type": "string", "default": "conversational"},
                          "target_length": {"type": "integer", "default": 2000}},
            "required": ["research_brief"]}
    }},
    {"type": "function", "function": {
        "name": "run_editor",
        "description": "Invoke the Editor agent to review and revise the draft",
        "parameters": {"type": "object",
            "properties": {"draft": {"type": "object"},
                          "research_brief": {"type": "object"},
                          "edit_focus": {"type": "array", "items": {"type": "string"}}},
            "required": ["draft", "research_brief"]}
    }},
    {"type": "function", "function": {
        "name": "run_seo_expert",
        "description": "Invoke the SEO Expert to generate metadata and social versions",
        "parameters": {"type": "object",
            "properties": {"article": {"type": "object"},
                          "target_keywords": {"type": "array", "items": {"type": "string"}},
                          "platforms": {"type": "array", "items": {"type": "string"}}},
            "required": ["article", "target_keywords"]}
    }},
    {"type": "function", "function": {
        "name": "evaluate_quality",
        "description": "Score the content across quality dimensions",
        "parameters": {"type": "object",
            "properties": {"article_content": {"type": "string"},
                          "research_brief": {"type": "object"}},
            "required": ["article_content", "research_brief"]}
    }},
    {"type": "function", "function": {
        "name": "compile_final_package",
        "description": "Compile and return the complete content package",
        "parameters": {"type": "object",
            "properties": {"article": {"type": "string"},
                          "seo_data": {"type": "object"},
                          "social_versions": {"type": "object"},
                          "image_prompts": {"type": "array"},
                          "quality_report": {"type": "object"}},
            "required": ["article", "seo_data", "quality_report"]}
    }}
]


def run_content_system(topic: str, style: str = "conversational") -> dict:
    start_time = time.time()
    messages = [
        {"role": "system", "content": ORCHESTRATOR_PROMPT},
        {"role": "user", "content": f"""Create a complete content package for:

Topic: {topic}
Style: {style}
Required outputs:
1. Full article (1500-3000 words)
2. 3-5 AI image prompts
3. SEO metadata (title/description/keywords)
4. Twitter thread (5-8 tweets)
5. LinkedIn post (800 words)
6. Quality score report

Coordinate the specialized agents and ensure quality score ≥ 80."""}
    ]

    final_package = None
    for _ in range(20):
        response = client.chat.completions.create(
            model=MODEL, messages=messages, tools=ORCHESTRATOR_TOOLS,
            tool_choice="auto", temperature=0.4
        )
        message = response.choices[0].message
        messages.append(message)

        if not message.tool_calls:
            break

        for tc in message.tool_calls:
            name = tc.function.name
            args = json.loads(tc.function.arguments)
            result = _dispatch(name, args)
            if name == "compile_final_package" and result.get("success"):
                final_package = result.get("package")
            messages.append({"role": "tool", "tool_call_id": tc.id,
                            "content": json.dumps(result)})

    if final_package:
        final_package["generation_time_seconds"] = time.time() - start_time
    return final_package or {"status": "failed"}

Specialized Agents (Compact)

# content_system/agents/writer.py
import os, json
from openai import OpenAI

client = OpenAI(base_url=os.getenv("HERMES_BASE_URL"), api_key=os.getenv("HERMES_API_KEY"))
MODEL = os.getenv("HERMES_MODEL")

WRITER_PROMPT = """You are a skilled content writer who transforms complex information into engaging articles.

Writing principles:
- Open strong: hook the reader in the first 3 seconds
- Every section must deliver value — no filler
- Use stories and cases to make abstractions concrete
- End with a clear call to action

Also generate AI image prompts (English, Midjourney/DALL-E format) for key sections."""

def run(research_brief: dict, style: str = "conversational", target_length: int = 2000) -> dict:
    messages = [
        {"role": "system", "content": WRITER_PROMPT},
        {"role": "user", "content": f"""Write a {target_length}-word {style} article based on:

{json.dumps(research_brief, indent=2)}

Output as JSON:
{{
  "title": "Article title",
  "hook": "Opening hook (first 100 words)",
  "sections": [{{"heading": "...", "content": "..."}}],
  "conclusion": "Conclusion text",
  "call_to_action": "CTA text",
  "image_prompts": [{{"position": "header", "prompt": "English prompt", "description": "..."}}]
}}"""}
    ]
    response = client.chat.completions.create(
        model=MODEL, messages=messages, temperature=0.7,
        max_tokens=4000, response_format={"type": "json_object"}
    )
    try:
        return {"success": True, "draft": json.loads(response.choices[0].message.content)}
    except Exception as e:
        return {"success": False, "error": str(e)}


# content_system/agents/editor.py
EDITOR_PROMPT = """You are a rigorous senior content editor.
Tasks: fact-check, logic review, language polish, quality gating.

Review checklist:
□ All statistics sourced?
□ Logic consistent, no contradictions?
□ Language natural and fluent?
□ Title compelling?
□ Opening hook strong enough?
□ Conclusion tied to body?"""

def run_editor(draft: dict, research_brief: dict, edit_focus: list = None) -> dict:
    messages = [
        {"role": "system", "content": EDITOR_PROMPT},
        {"role": "user", "content": f"""Review and revise this article:
Draft: {json.dumps(draft, indent=2)}
Research brief (for fact-checking): {json.dumps(research_brief, indent=2)}

Output JSON with: title, content (Markdown), editor_notes[], fact_check_results[],
quality_scores {{accuracy:0-25, narrative:0-20, originality:0-20, seo:0-15, value:0-20}},
total_score: 0-100"""}
    ]
    response = client.chat.completions.create(
        model=MODEL, messages=messages, temperature=0.2,
        max_tokens=5000, response_format={"type": "json_object"}
    )
    try:
        return {"success": True, "edited": json.loads(response.choices[0].message.content)}
    except Exception as e:
        return {"success": False, "error": str(e)}

73.4 Quality Scoring Engine

# content_system/quality.py

def evaluate_content_quality(article_content: str, research_brief: dict) -> dict:
    scores = {}

    # Accuracy (25 pts): key facts present in article
    key_facts = research_brief.get("key_facts", [])
    matches = sum(1 for f in key_facts if _fuzzy_match(article_content, f))
    scores["accuracy"] = min(25, int(25 * matches / max(len(key_facts), 1)))

    # Structure (15 pts)
    has_headings = article_content.count("##") >= 2
    has_conclusion = any(w in article_content.lower() for w in ["conclusion", "summary", "in summary"])
    scores["structure"] = 15 if (has_headings and has_conclusion) else 8

    # Length (10 pts)
    wc = len(article_content.split())
    scores["length"] = 10 if 400 <= wc <= 1200 else (6 if wc >= 200 else 3)

    # Keyword coverage (15 pts)
    keywords = research_brief.get("target_keywords", [])
    kw_hits = sum(1 for kw in keywords if kw.lower() in article_content.lower())
    scores["keyword_coverage"] = min(15, int(15 * kw_hits / max(len(keywords), 1)))

    # Reader value proxy (20 pts): presence of examples, numbers, actionable advice
    has_numbers = any(char.isdigit() for char in article_content)
    has_examples = any(w in article_content.lower() for w in ["for example", "case study", "such as"])
    scores["reader_value"] = 20 if (has_numbers and has_examples) else 12

    total = sum(scores.values())
    return {
        "scores": scores, "total": total,
        "grade": "A" if total >= 85 else "B" if total >= 70 else "C",
        "pass": total >= 80,
        "recommendations": _recommendations(scores)
    }

def _fuzzy_match(text: str, fact: str) -> bool:
    words = [w for w in fact.split() if len(w) > 3]
    return sum(1 for w in words if w.lower() in text.lower()) >= len(words) * 0.6

def _recommendations(scores: dict) -> list:
    recs = []
    if scores.get("accuracy", 0) < 18:
        recs.append("Add more specific data points and source citations")
    if scores.get("structure", 0) < 12:
        recs.append("Improve structure: add section headings and a clear conclusion")
    if scores.get("keyword_coverage", 0) < 10:
        recs.append("Naturally integrate more target keywords")
    return recs

Chapter Summary

This chapter designed and implemented a production-grade multi-agent content creation system:

• Role specialization: Researcher/Writer/Editor/SEO Expert each handling their domain prevents quality averaging
• Orchestrator pattern: Central coordinator handles scheduling, quality gates, and iteration control
• Data flow contracts: Clear interfaces between agents enable independent testing and replacement
• Quality feedback loop: Built-in scoring ensures output reaches publishable standards

The core value of multi-agent systems is not parallel speed (collaborative agents are often sequential) but quality gains from specialization — just as excellent content requires a researcher, writer, editor, and fact-checker working together rather than one person doing everything.

Discussion Questions

1. How do you design inter-agent communication protocols that are both flexible and lossless?
2. When the Editor's feedback conflicts with the Writer's stylistic choices, how should the Orchestrator arbitrate?
3. How would you extend this system to support video scripts, podcast content, and other non-text formats?
4. How do you design observability and debugging tooling for a multi-agent system?