How we structure, adopt, govern, and measure agentic operations.

Organizational Model

The Structural Shift

Traditional software teams organize in horizontal layers: frontend team, backend team, QA team, DevOps team. Work moves laterally through handoffs. This model breaks under agentic workflows because a single agent action can span all layers simultaneously — modifying frontend components, backend APIs, database schemas, and infrastructure configs in one task.

The shift is from horizontal silos to vertical cross-functional pods where small teams own features end-to-end, with AI agents acting as connective tissue between layers.

BEFORE (Horizontal)              AFTER (Vertical Pods)

Product --> Frontend --> Backend  Pod A: [PM + Designer + Architect + Engineers + Agent] --> Feature A
        --> QA --> DevOps         Pod B: [PM + Designer + Architect + Engineers + Agent] --> Feature B
        --> (handoff delays)      Pod C: [PM + Designer + Architect + Engineers + Agent] --> Feature C

Senior leads oversee pods delivering complete product slices. Agents handle bounded implementation within each pod. Handoffs between teams are replaced by direct ownership.

Roles with Explicit Authority

Nine roles that must carry real authority — not secondary responsibilities stacked on existing jobs. The distinction matters: when evaluation quality is “someone’s side project,” it doesn’t get done until an incident forces it.

Engineering Roles

AI Architect

Owns: End-to-end orchestration and structural decisions.

Responsibilities:

• Selects models and defines which model handles which task
• Designs data flow from input to output
• Decides orchestration pattern (single agent, multi-agent, workflow)
• Defines failure modes and recovery paths
• Makes the structural decisions the rest of the team builds on

AI Reliability Engineer

Owns: Observability, cost measurement, and failure recovery. The SRE equivalent for AI systems.

Responsibilities:

• Defines what to measure to know the system works
• Monitors cost per execution and flags unsustainable patterns
• Manages failure detection and recovery mechanisms
• Owns the guardrail stack implementation and enforcement
• Runs incident response for agent-related failures

For the full transformation story, see SRE / DevOps Engineer: Roles in the AI Era.

Evaluation Lead

Owns: Test coverage and evaluation strategy. Not unit tests for code — evaluation coverage for agent outputs.

Responsibilities:

• Defines “how do we know this is good enough to ship?”
• Designs eval suites for agent behavior (beyond standard test suites)
• Sets passing thresholds and quality bars
• Ensures evaluation runs before every ship decision
• Tracks quality metrics over time to detect drift

The Evaluation Lead emerges from the traditional QA role splitting in two. For the full transformation story, see QA Engineer / SDET: Roles in the AI Era.

Product Engineer

Owns: Feature velocity and integration.

Responsibilities:

• Runs the agent execution loop for scoped delivery tasks
• Creates and maintains task templates and agent instructions
• Integrates agent-generated output into the product
• Ensures agent output meets product requirements and UX standards
• Manages the Plan-Execute-Verify-Ship-Learn loop (see Practitioner Guide)

Platform Engineer

Owns: Infrastructure, model hosting, and inference serving. Optional early, required at scale.

Responsibilities:

• Manages compute infrastructure for agent execution
• Optimizes cost-efficiency at the infrastructure layer
• Handles latency and reliability of model inference
• Implements the governance layer (registry, access control, observability)
• Manages secrets, API keys, and secure agent-to-system connectivity

Engineering Manager

Owns: Team capability, adoption equity, and outcome-based measurement. The person who executes the Six Organizational Shifts day-to-day.

Responsibilities:

• Assesses and advances the team’s maturity level using the Maturity Model (Levels 1-5)
• Ensures adoption equity: the team’s effective level equals the least-adopted member in a critical-path role
• Defines and enforces the team’s operating rhythm around the Plan-Execute-Verify-Ship-Learn loop
• Restructures team workflows from horizontal silos to vertical cross-functional pods
• Establishes KPIs that measure product impact, not output volume
• Coaches engineers on judgment, review quality, and context engineering
• Manages the human side of AI adoption: resistance, identity shifts, and role redefinition

For the full transformation story, see Engineering Manager: Roles in the AI Era.

Product Roles

Product Manager

Owns: Problem definition, acceptance criteria, and product quality.

Responsibilities:

• Defines what to build and why — the “Plan” phase of the operating loop
• Writes acceptance criteria that agents can execute against
• Reviews agent output for product correctness (does it solve the user’s problem?)
• Shifts from writing detailed specifications to defining constraints and quality bars
• Tracks product outcome metrics alongside delivery metrics

For the full transformation story, see Product Manager: Roles in the AI Era.

Source: Deloitte (“Human-Agentic Workforce”), IdeaPlan (“Agentic AI for PMs”)

Product Designer

Owns: UX quality, design system, and interaction patterns.

Responsibilities:

• Maintains the design system that agents generate from (tokens, components, patterns)
• Reviews agent-generated UI for UX quality and design consistency
• Defines design tokens and component specifications as agent instructions
• Shifts from pixel-level execution to design governance and quality auditing
• Addresses the emerging discipline of Agent Experience (AX) — designing for both human and agent actors

For the full transformation story, see Product Designer: Roles in the AI Era.

Source: “From UX to AX” (The Atlantic), cases.media (“Five Product Shifts”)

QA Engineer

Owns: Product-level quality from the user’s perspective. Distinct from the Evaluation Lead.

Responsibilities:

• Translates acceptance criteria into testable assertions
• Builds evaluation suites that validate product behavior, not just code correctness
• Monitors quality drift from a user-facing perspective (UX regressions, accessibility, copy errors)
• Works with the Evaluation Lead on comprehensive quality coverage

Boundary with Evaluation Lead: The Evaluation Lead owns agent output correctness (did the agent follow instructions? is the code well-structured? does it pass technical evaluation?). The QA Engineer owns product correctness (does the shipped feature meet the user’s need? does it work as specified? does it meet accessibility and UX standards?). One judges the agent. The other judges the product.

For the full transformation story — how each of these roles evolved from traditional job descriptions and what to screen for when hiring — see Roles in the AI Era.

Scaling Path

Start (7-8 people):

• 1 AI Architect (leads)
• 2-3 Product Engineers
• 1 AI Reliability Engineer (may share duties with the Architect early on)
• 1 Engineering Manager
• 1 Product Manager (may be part-time)
• 1 Product Designer (may be shared across pods early on)

Scale (12-16 people):

• 1 AI Architect
• 3-4 Product Engineers (some specializing in different surfaces)
• 1-2 AI Reliability Engineers
• 1-2 Evaluation Leads
• 1 Platform Engineer
• 1-2 Engineering Managers (one per pod at scale)
• 1 Product Manager
• 1-2 Product Designers
• 1 QA Engineer

The key: roles exist with authority, not as hats stacked on other jobs. This applies equally to product roles — a PM who owns acceptance criteria must have the authority to reject agent output that doesn’t meet the bar.

Decision Rights Matrix

Ownership clarity alone is not enough. You also need clear decision rights so that every critical decision has a single owner, not a consensus process.

The single mistake most teams make: involving too many people in every decision, hoping consensus catches problems. Instead, they get slow, uncertain decision-making where nobody feels accountable when things go wrong.

Six Organizational Shifts

Adapted from McKinsey’s framework for the agentic organization, translated into concrete actions for tech teams:

Shift 1 — Workflows: Design for AI-first, not AI-assisted Stop asking “where can an agent help in this process?” Start asking “if we built this process from scratch with agents, what would it look like?” Redesign the workflow before automating it. This applies to product workflows too: how PMs define requirements, how designers create specifications, and how QA validates outcomes all need rethinking, not just how engineers write code.

Shift 2 — Leadership: From directing execution to defining constraints The leader’s job shifts from specifying how work gets done to defining what “good” looks like, what boundaries exist, and what must not happen. Leaders set quality bars and scope constraints. Agents and engineers handle execution within those bounds. PMs write constraints and quality bars, not step-by-step specifications. Designers define systems, not individual screens.

Shift 3 — Talent: From specialists to T-shaped integrators Individual contributors need breadth across the stack because agents blur layer boundaries. A backend engineer who cannot review agent-generated frontend code becomes a bottleneck. A designer who cannot evaluate agent-generated UI in code becomes a gatekeeper. A PM who doesn’t understand agent capabilities cannot write executable acceptance criteria. Hire and develop for judgment across domains, not just depth in one.

Shift 4 — Culture: Build continuous reinvention The tools, models, and patterns change quarterly. Teams that treat their current setup as permanent fall behind. Build a culture where workflows are versioned and revisited, rules files are living documents, and “the way we do things” is explicitly up for revision. This includes product processes: how PRDs are written, how design reviews work, and how quality is evaluated must all evolve alongside engineering practices.

Shift 5 — Structure: From functional teams to outcome-oriented pods Reorganize around outcomes (features, services, customer journeys) rather than functions (frontend, backend, QA). Each pod includes product roles (PM, Designer) alongside engineering roles, with the autonomy and capability to deliver end-to-end. Agents fill capability gaps within pods. Product and engineering co-own the outcome.

Shift 6 — People systems: Measure impact, not output volume Agent-assisted teams will produce more PRs, more designs, more docs, more tests. None of these are meaningful measures of contribution. Measure product outcomes (adoption, retention, satisfaction), quality, and decision quality. A PM who writes precise acceptance criteria that agents execute correctly on the first pass is more valuable than one who writes ten vague stories. A senior engineer who reviews 10 agent-generated PRs and catches a critical bug is more valuable than one who personally writes 10 PRs.

Source: Chrono Innovation (“Building AI Agents Without Organizational Chaos”), McKinsey (“The Agentic Organization,” “Six Shifts”), Deloitte (“Agentic AI Strategy”)

Maturity Model

The full five-level maturity model — with dimension tables, assessment criteria, prerequisites, and failure modes for each level — lives in the Practitioner Guide. Both guides share this model so practitioners and leaders use a common vocabulary.

Quick reference for leadership conversations:

Critical reminder: A team’s effective level is set by its least-adopted member in a critical-path role — see the Maturity Assessment in the Practitioner Guide for the full rationale.

Use the maturity level to calibrate the Adoption Roadmap phases and to set KPI targets in the Measurement section below.

Source: Eledath (“The 8 Levels of Agentic Engineering”), Gartner (Top Strategic Trends 2026, “Five Stages of Agentic Evolution”)

Adoption Roadmap

A four-phase rollout over 180 days. Each phase has defined goals, activities, exit criteria, and risk mitigations. The principle: staged adoption gives you speed without blind trust.

Phase 1: Contained Pilot (Days 1-30)

Goal: Establish baseline metrics, validate feasibility, build team confidence in a controlled environment.

Activities:

• Select one repository with moderate complexity (not the most critical, not a toy)
• Define 3 repeatable task types the pilot will focus on (e.g., API endpoint, test generation, refactor)
• Measure baseline metrics before agents touch anything:
  • PR cycle time (issue opened to merged)
  • Change failure rate
  • Test coverage
  • Bug rate per sprint
• Set up basic guardrails: scope definition per task, CI as gate, all agent PRs require senior review
• Every team member runs at least one agent-assisted task to establish shared experience
• Document what works, what fails, and what surprises
• Product: PM participates in defining the 3 repeatable task types. Designer reviews agent-generated UI in the pilot. Baseline product metrics recorded alongside delivery metrics.

Exit criteria (must meet ALL to proceed):

• Baseline metrics recorded for comparison
• At least 10 agent-assisted tasks completed and reviewed
• No critical quality incidents from agent output
• Team can articulate which tasks agents handle well and which they don’t
• Basic rules file created and shared across the team

Risk mitigations:

• Senior review on 100% of agent PRs — no exceptions in Phase 1
• Start with low-risk, well-bounded tasks only
• If a quality incident occurs, pause and retrospect before continuing

Phase 2: Expand Safely (Days 31-60)

Goal: Increase scope, introduce structured templates, and build the verification infrastructure.

Activities:

• Expand to 2-3 repositories
• Create task templates for each repeatable pattern (standardized plan documents)
• Introduce risk labels (low / medium / high) on every agent task
• Implement the full quality guardrail layer (Layer 2): formatting, linting, tests, type checks, static analysis
• Begin implementing policy guardrails (Layer 3): secret scanning, branch protection
• Automate post-merge summaries for auditability
• Start tracking adoption KPIs: % of PRs agent-assisted, CI first-pass rate
• Expand the rules file based on Phase 1 lessons (compounding engineering)
• Product: PM creates acceptance criteria templates for agent tasks. Designer contributes design tokens and component specs as agent instructions. Begin tracking design compliance rate.

Exit criteria (must meet ALL to proceed):

• Task templates exist for at least 3 common patterns
• Risk labeling is applied to all agent tasks
• Quality guardrails (Layer 2) are fully automated in CI
• Policy guardrails (Layer 3) are partially implemented
• Adoption KPIs are being tracked weekly
• No increase in change failure rate compared to baseline

Risk mitigations:

• Maintain senior review on medium and high risk tasks
• Low-risk tasks may move to sampling-based review (reviewer checks 1 in 3)
• Weekly retrospective on agent output quality

Phase 3: Standardize (Days 61-90)

Goal: Codify the operating model into an internal standard. Formalize governance. Train the full team.

Activities:

• Publish internal “Agentic Standard Operating Procedure” (SOP) documenting:
  • The Plan-Execute-Verify-Ship-Learn loop
  • Task classification matrix
  • When to use agents vs. when not to
  • Review requirements by risk level
  • Escalation procedures
• Implement remaining policy guardrails (Layer 3): PII filtering, relevance checking, moderation
• Add repo-level policy enforcement (which repos allow agent PRs, under what conditions)
• Train all team members on the SOP, task templates, and rules files
• Conduct a maturity self-assessment (Level 1-5) and set a target for the next quarter
• Establish the evaluation framework: how is agent output quality measured beyond CI?
• Define roles and decision rights (even if not all roles are filled by dedicated people yet)
• Product: Product team trained on SOP alongside engineering. Product-specific KPIs added to the dashboard. PM formally owns the Plan phase. Designer owns design system compliance checks.

Exit criteria (must meet ALL to proceed):

• Internal SOP is published and accessible to all team members
• All team members have completed SOP training
• Guardrail stack (Layers 1-4) is fully operational
• Maturity self-assessment completed; current level agreed; target level set
• Evaluation framework exists (beyond CI — how do we measure agent output quality?)
• Decision rights are documented (who owns what)

Risk mitigations:

• Ensure the SOP is a living document, not a one-time artifact
• Schedule quarterly SOP reviews
• Assign an SOP owner responsible for updates

Phase 4: Scale (Days 91-180)

Goal: Expand across teams. Build governance infrastructure. Move toward higher maturity levels.

Activities:

• Roll out to additional teams and repositories
• Implement governance layer (Layer 5): agent registry, access control, cross-team observability
• Build a shared skills/template library accessible across teams
• Establish cost budgeting per team and per agent workflow
• Begin experimenting with Level 4 capabilities (background agents, async PR generation) in controlled conditions
• Publish organizational metrics dashboard (see Measurement below)
• Conduct cross-team retrospectives to share learnings
• Evaluate dedicated role staffing (do we need a full-time AI Reliability Engineer? Evaluation Lead? QA Engineer?)
• Product: Product outcome metrics in the organizational dashboard. Evaluate dedicated QA Engineer staffing. PM acceptance criteria templates shared across teams. Design system fully instrumented for agent compliance checking.

Exit criteria (ongoing — reviewed quarterly):

• Multiple teams operating under the same SOP
• Governance layer (Layer 5) operational (at minimum: registry + cost tracking)
• Shared skills library in use across teams
• Organizational KPI dashboard published and reviewed weekly
• Change failure rate stable or improved relative to baseline
• Quarterly maturity assessment shows progression
• Roles and decision rights scaled to match organizational breadth

Risk mitigations:

• Do not skip Phase 3 standardization before scaling — scaling without standards multiplies chaos
• Start Level 4 experiments in a single pod before expanding
• Monitor cost carefully during scale-out; token spend can increase non-linearly

Source: vibecoding.app (“Agentic Engineering for Software Teams”), Deloitte (“Agentic AI Enterprise Adoption”), McKinsey (“Seizing the Agentic AI Advantage”)

Measurement and Failure Modes

KPI Dashboard

Track these metrics weekly. The goal is not to maximize agent usage — it is to deliver faster without quality regression.

Core Delivery KPIs

These are standard DORA-style metrics. The critical check: if lead time and deployment frequency increase but change failure rate and escaped defects also increase, agent adoption is creating the illusion of speed while degrading quality.

Adoption KPIs

Product Outcome KPIs

Quality KPIs

The Success Formula

Agent adoption is working when:

Lead time ↓  AND  Deployment frequency ↑  AND  Change failure rate ↔/↓  AND  Escaped defects ↔/↓

If speed metrics improve but quality metrics degrade, the adoption is creating accelerated technical debt. Pause and strengthen guardrails before continuing to scale.

Six Failure Modes

Each failure mode includes the symptom (how you detect it), the root cause (why it happens), and the mitigation (how to fix it).

Failure Mode 1: Automation Theater

Symptom: High volume of agent activity (many PRs, many tasks) with minimal measurable business or delivery impact. Leadership sees dashboards showing “AI adoption” but nothing ships faster or better.

Root cause: Tasks selected for agents are easy-to-automate busywork rather than genuine bottlenecks. The team optimizes for agent-friendly tasks rather than high-impact tasks.

Mitigation:

• Tie every agent workflow to a measurable delivery KPI (lead time, cycle time, deployment frequency)
• Require a “so what?” test: if this task is automated, what bottleneck does it remove?
• Review the task selection criteria quarterly — are we automating the right things?

Failure Mode 2: Review Bottlenecks

Symptom: Agents generate PRs faster than the team can review them. PR queue grows. Merge latency increases. Engineers feel overwhelmed by review volume.

Root cause: Agent output velocity exceeds the team’s review capacity. Often caused by large, unfocused agent PRs that are harder to review than manually written code.

Mitigation:

• Enforce smaller PR scope (one concern per PR, bounded by task template)
• Tighten acceptance criteria so PRs are more focused
• Scale review capacity: train more team members to review agent output
• Implement risk-based review: low-risk PRs get sampling-based review, freeing capacity for high-risk reviews
• Consider review automation: automated checks handle the mechanical aspects, human reviewers focus on logic and design

Failure Mode 3: Silent Quality Drift

Symptom: Code merges quickly. Sprint velocity looks good. But incident rate, bug reports, or customer complaints gradually climb. The connection between agent output and quality issues is not immediately visible.

Root cause: Verification gates are incomplete. Agent-generated code passes CI but introduces subtle issues (performance regressions, edge case mishandling, architectural erosion) that tests don’t cover.

Mitigation:

• Expand evaluation coverage beyond unit tests: include integration tests, performance benchmarks, architecture fitness functions
• Track post-release defect rate specifically for agent-generated code
• Implement regular “agent output audits” — periodically deep-review a sample of merged agent PRs
• Monitor the change failure rate as an early warning signal

Failure Mode 4: Prompt Tribal Knowledge

Symptom: One or two engineers get consistently better results from agents than the rest of the team. They know “the magic prompts” or have refined personal rules files. The rest of the team struggles with poor agent output and loses confidence.

Root cause: Effective agent interaction patterns are not captured and shared. Knowledge stays in individual heads or personal configuration files.

Mitigation:

• Convert individual prompts and techniques into shared task templates
• Maintain team-level rules files (not personal ones)
• Publish an internal “agentic SOP” with examples and patterns
• Pair programming sessions where skilled users demonstrate their approach
• Make the Learn phase of the operating loop mandatory: every insight gets codified

Failure Mode 5: Governance Gap

Symptom: Agent usage scales across teams, but nobody has a clear view of which agents exist, what they can access, what they cost, or what they’ve done. Compliance questions arise that nobody can answer. An agent accesses data it shouldn’t have.

Root cause: Governance infrastructure (Layer 5) was not built before scaling. Organizations skipped Phase 3 (Standardize) or Phase 4 (Scale) governance requirements.

Mitigation:

• Implement the governance layer before cross-team scaling (not after an incident)
• Start with minimum viable governance: an agent registry and cost tracking
• Add access control and audit trails as usage grows
• Assign an owner for the governance layer (AI Reliability Engineer or Platform Engineer)
• Review governance completeness quarterly

Failure Mode 6: Velocity Without Direction

Symptom: Team ships features 3x faster but product metrics (adoption, retention, satisfaction) don’t improve or decline. Agents solved the “how to build” problem but the “what to build” problem is unchanged. Leadership celebrates delivery speed while customers see no difference.

Root cause: Agent adoption accelerated delivery without improving problem selection. Product definition didn’t keep pace with engineering velocity. The team is building the wrong things faster.

Mitigation:

• Tie agent task selection to product outcome metrics, not just delivery speed
• Require PM sign-off on every task plan — agents should not build features nobody defined
• Measure feature adoption and user satisfaction alongside lead time and deployment frequency
• Apply the same “redesign, don’t automate” principle to product discovery, not just delivery

Source: Deloitte (“Human-Agentic Workforce”), IdeaPlan (“Agentic AI for PMs”)

Anti-Pattern Quick Reference

Source: vibecoding.app (“Agentic Engineering for Software Teams”), Deloitte (“State of AI in the Enterprise 2026”), Gartner (Top Strategic Trends 2026), McKinsey (“The State of AI in 2025”)