Workflow

A workflow in AI systems represents a structured approach where Large Language Models (LLMs) and tools are orchestrated through code paths. These paths may be entirely fixed, or may be determined dynamically by the LLM in response to the task, its observations, and the environment. Workflows are designed for tasks that can be broken down into a series of steps.

Key Characteristics

• Potentially dynamic execution paths
• Structured tool integration
• Variable degrees of LLM autonomy over the workflow

Spectrum of Agency in Workflows

Workflows can be classified by their level of agency, or the degree to which the LLM controls the workflow:

• Workflow Agentic 1 (Least Agentic): Purely linear or DAG-based execution with no feedback loops or dynamic adaptation. The path is entirely predetermined. For example, a workflow might use an LLM to generate marketing copy in English, then send the output to a second LLM to translate it into Spanish.
• Workflow Agentic 2: Incorporates basic branching or conditional logic based on predefined rules. The workflow can adapt to certain inputs or conditions, but the overall structure remains fixed. For example, a workflow might use an LLM to classify customer support requests, then route them to different LLMs or tools based on the classification.
• Workflow Agentic 3: Includes limited feedback loops, allowing for minor adjustments based on the output of previous steps. The Evaluator-Optimizer workflow, with its iterative refinement cycle, could fall into this category. For example, a workflow might use one LLM to generate code and another to evaluate it, looping until the evaluator is satisfied.
• Workflow Agentic 4 (Most Agentic): Workflows with more sophisticated feedback mechanisms and the ability to dynamically choose tools or sub-tasks based on "ground truth" assessment. The workflow itself might guide the agent's actions, but the agent retains significant autonomy in how it interacts with the environment and adapts to unforeseen circumstances. For example, an LLM might assess a coding task and then decide which files to edit, which tools to use, and what changes to make, based on its understanding of the task and the codebase.

Common Workflow Patterns

• Prompt Chaining: Decomposing a task into a sequence of LLM calls, where each call processes the output of the previous one.
• Routing: Classifying an input and directing it to a specialized followup task.
• Parallelization: Running multiple LLM calls simultaneously on different parts of a task or the same task, and aggregating the outputs programmatically.
• Orchestrator-Workers: A central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results.
• Evaluator-Optimizer: One LLM call generates a response while another provides evaluation and feedback in a loop.

Best Practices

• Design clear processes: Whether the steps are fixed or determined dynamically, the workflow should have a clear objective and a logical flow.
• Implement proper error handling: The workflow should be able to handle unexpected inputs, errors, and exceptions gracefully.
• Optimize for the appropriate level of agency: Choose the level of LLM autonomy that is appropriate for the task and the desired balance of flexibility and control.
• Maintain clear documentation: The workflow should be well-documented so that it can be understood, maintained, and debugged by others.
• Regular performance monitoring: The workflow's performance should be monitored and evaluated regularly to ensure that it is meeting its objectives.