Workflow Design: From Manual to Auto-Search

THE QUESTION THIS PAGE ANSWERS

ANSWER FIRST

What is the key idea behind “Workflow Design: From Manual to Auto-Search”?

AI Scientist / ADAS / AFlow — using MCTS and Meta-Agents to search for optimal workflows

DECISION RULE

Make the claim earn its place. Use this page as a decision aid, not a definition to memorize. Connect the idea to one real task, one observable result, and one failure that would change your mind.

TRY NEXT

Write one question you could answer with evidence after trying this idea.

WATCH FOR

A conclusion that sounds complete but leaves the key assumption untested.

Hand-Designed Workflows

💬 Plain talk: A workflow is like a task checklist for the AI. Like onboarding a new employee, you have to tell them "do A first, then B, and if something goes wrong check with C." These checklists used to be written entirely by hand, and the two examples below represent the most powerful hand-crafted checklists ever built.

Lu et al. 2026

AI Scientist

Building a complete scientific research automation pipeline

Core idea: Hand the entire scientific research process to an Agent system to complete automatically: execute a full research cycle from scratch.

Pipeline:

Generate research idea
→
Write code
→
Run experiments
→
Analyze results
→
Write paper
→
Peer review

Key innovation: Every step is LLM-driven, forming an end-to-end automated research system. The review step introduces LLM-as-judge for quality control.

The complete AI Scientist pipeline: from research ideation to paper writing and peer review, fully automated by Agents. (Lu et al. 2026)

Kulikov et al. 2026

Autodata

Data Scientist Agent: synthesizing training data at exactly the right difficulty

Core idea: Through multi-role collaboration, automatically synthesize training data with precisely controlled difficulty: solvable by a strong solver, unsolvable by a weak solver.

Role system:

ChallengerQuestion setter

Weak SolverWeaker solver

Strong SolverStronger solver

Verifier / JudgeQuality arbiter

Key innovation: Uses difficulty gap as a data quality signal: only questions that a strong solver can solve but a weak solver cannot are retained, ensuring the synthesized data has maximum value for improving the model.

Autodata's multi-role collaboration: Challenger sets questions, Strong/Weak Solvers attempt them, Verifier arbitrates data quality. (Kulikov et al. 2026)

ADAS: Automated Design of Agentic Systems

💬 Plain talk: Writing checklists by hand is exhausting. What if we could let AI design checklists for AI? That's ADAS: hire a "designer AI" and let it continuously draft new checklists, run them, score them, and keep the good ones. It automates the work that used to require a senior expert mentoring a junior one.

Hu et al. 2025

ADAS — Automated Design of Agentic Systems

Treating Agent design itself as an optimization problem

Core idea: Let a "Meta-Agent" automatically search the Agent design space for the optimal solution, replacing hand-crafted Agent architectures. This is "meta-agent search".

How it works:
1. Meta-Agent generates a high-level description (Agent architecture, tool usage, reasoning strategy)
2. Translates the description into executable code
3. Uses self-refine to check for novelty: ensures it's not simply repeating an existing solution
4. Evaluates on benchmark tasks, keeping the best-performing designs

Key innovation: Uses code as the representation of the Agent design search space, enabling designs to be automatically generated, modified, and evaluated.

ADAS Meta-Agent Search: Meta-Agent generates candidate Agent designs → implements as code → evaluates → iteratively optimizes. (Hu et al. 2025)

AFlow: MCTS-Based Workflow Search

💬 Plain talk: ADAS lets AI freestyle and come up with new solutions. AFlow goes further, using the same board game simulation technique (MCTS, Monte Carlo Tree Search) that powers game-playing AIs. Just like AlphaGo simulates hundreds of moves ahead — "if I play here, what will my opponent do?" — AFlow simulates hundreds of workflow variations in its head and digs deeper down whichever path scores highest.

Zhang et al. 2025

AFlow

Representing Agent workflows as graphs and optimizing automatically with MCTS

Core idea: Represent workflows as a directed graph: nodes are LLM action calls, edges are logical operations in code (conditional branches, loops, data passing). Then use Monte Carlo Tree Search (MCTS) to automatically find the optimal workflow in this graph space.

MCTS optimization process:

Initialize the starting workflow as the root node of the search tree
Use soft mixture of score and uniform exploration to select the node to expand (balancing exploitation and exploration)
Let the LLM generate modified workflow variants (add/remove/modify nodes and edges)
Execute and evaluate the new workflow's performance on the target task
If there is improvement, add it back to the search tree as a new candidate
Repeat until the top-k average score stabilizes or the compute budget is exhausted

Key innovation: Transforms the workflow design problem into a tree search problem, letting MCTS's exploration-exploitation balance mechanism automatically discover good workflow structures — no human hand-crafting required.

AFlow's graph representation and MCTS search: workflows are represented as directed graphs of nodes (LLM actions) and edges (logical operations). (Zhang et al. 2025)

AFlow Experimental Results

Method	Design approach	Search strategy	Core advantage
Manual design	Human expert iteration	None (intuition-based)	High interpretability
ADAS	Meta-Agent + code	Self-refine	Automated design
AFlow	Graph representation + MCTS	Monte Carlo Tree Search	Systematic search + stable convergence

AFlow performance on QA, code generation, and mathematical reasoning tasks: systematic search outperforms manual design and ADAS. (Zhang et al. 2025)

Step-by-Step: AFlow's MCTS Search Process

Ready

Initialize

W₀: Plan → Execute Score 52

Start with the simplest single-step workflow. No reflection, no verification, no parallelism.

Expand: LLM Proposes Variants

W₁: +Reflect61 W₂: +Verify58 W₃: +Decompose55

Let the LLM generate three modification proposals based on W₀, adding reflection, verification, and task decomposition respectively.

Evaluate & Select Parent Node

W₁: 61 ← Selected

W₁ has the highest score (61 > 58 > 55), so it's selected as the parent node for the next round of expansion.

Expand Again

W₄: Reflect×265 W₅: Reflect+Verify78 🏆

Expand two variants from W₁: double reflection vs. reflection + verification combo. The latter wins by a wide margin!

Prune & Converge

W₃: 55 ✗ W₆: 56 ✗ W₅: 78 ✓ Optimal

Workflows below the threshold are pruned. W₅ (Plan → Execute → Reflect → Verify → Output) is confirmed as the optimal solution found. 50% better than manual design!

Core Insights

Enormous search space: The combinatorial possibilities for workflows far exceed what humans can manually explore — hand design only scratches the surface
Design is search: Treat workflow design as a search problem; use algorithms (like MCTS) instead of intuition to find good solutions
Code is the universal language: A Harness is fundamentally code that orchestrates prompts, tool calls, sub-Agents, control flow, memory, and workflow logic
The evolution from ADAS to AFlow: Automation of Agent design keeps advancing — from having an LLM design it to having algorithms systematically search for it

Why “When the workflow itself becomes the search space, design shifts from art to engineering” depends on the operation

“AI Scientist / ADAS / AFlow — using MCTS and Meta-Agents to search for optimal workflows” makes the structure concrete. The useful comparison is not which name sounds more advanced, but how the data is arranged and how far the most common operation has to travel.

Read a structure through access and change

“AI Scientist / ADAS / AFlow — using MCTS and Meta-Agents to search for optimal workflows” exposes a trade-off that is easy to miss: reading by position, looking up by key, adding at either end, inserting in the middle, and traversing relationships do not favor the same organization. A structure that is fast for one operation is not automatically fast for all of them.

Generate research idea
Initialize the starting workflow as the root node of the search tree
Use soft mixture of score and uniform exploration to select the node to expand (balancing exploitation and exploration)

Count scale and update frequency together

Use “AI Scientist / ADAS / AFlow — using MCTS and Meta-Agents to search for optimal workflows” as a boundary check. Write down the data size, the dominant operation, and the latency you can accept before deciding whether an AI-generated structure actually fits.

From “When the workflow itself becomes the search space, design shifts from art to engineering” to “Hand-Designed Workflows”

“When the workflow itself becomes the search space, design shifts from art to engineering” grounds the problem in “From fully hand-crafted pipeline design to using MCTS to automatically discover optimal workflows, the design paradigm for Agent systems is undergoing a fundamental shift. Code is the universal language for def…”. “Hand-Designed Workflows” then moves it toward “💬 Plain talk: A workflow is like a task checklist for the AI. Like onboarding a new employee, you have to tell them "do A first, then B, and if something goes wrong check with C." These checklists used to be w…”. Together, they show that the lesson is not just a conclusion to remember, but a claim with conditions.

Carry the judgment into the next situation

When you meet a new data structure, do not begin by memorizing its definition. Write down the most frequent operation, estimate scale and update behavior, and check whether the structure satisfies all three conditions.

“When the workflow itself becomes the search space, design shifts from art to engineering”: From fully hand-crafted pipeline design to using MCTS to automatically discover optimal workflows, the design paradigm for Agent systems is undergoing a fundamental shift. Code is the universal language for def…
“Hand-Designed Workflows”: 💬 Plain talk: A workflow is like a task checklist for the AI. Like onboarding a new employee, you have to tell them "do A first, then B, and if something goes wrong check with C." These checklists used to be w…
“The closing point”: Execute and evaluate the new workflow's performance on the target task

The final “The closing point” brings the discussion to “Execute and evaluate the new workflow's performance on the target task”. The useful thing to carry forward is knowing which judgments must be revisited when input, scale, or risk changes.

Mark as learned Your reading progress updates automatically

ARTICLE DISCUSSION

Leave one useful thought here.

Keep the idea that clicked, the question that stayed open, or a small note for the next learner.

↗

Discussing Workflow Design: From Manual to Auto-Search When the Harness Improves Itself

3discussionsArticle discussion · synced with the Circle

View in the learning circle

Asha MorganContent editorAug 20, 2026

INSIGHTField note

I turned one judgment from this article into a small experiment I could run today. Knowing what to observe next is more useful than simply remembering the conclusion.

ARTICLE DISCUSSION7 helpful

Lin HarperIndie developerAug 18, 2026

INSIGHTInsight

After reading this, I first looked for the conditions behind the idea instead of copying the method into a project. That order made the later trade-offs much clearer.

ARTICLE DISCUSSION5 helpful

Kiki MooreProduct operationsAug 15, 2026

QUESTIONQuestion

When this judgment reaches real work, which constraint should be added first? I am curious which step matters most between reading and the first practical attempt.

ARTICLE DISCUSSION4 helpful