Why workflows are here to stay

Workflows vs. agents seems to be a topic of discussion that pops up every so often. It happened again in the aftermath of last week’s Agent Builder release by OpenAI which we discussed here. We argue that this discussion is missing the point. You can just embed agents as steps or nodes in workflows and combine the best of both worlds!

Note: to avoid any confusion, we use the commonly adopted definition that an agent is an LLM running tools in a loop to achieve a goal^[1].

Workflows let you take control of the trade-offs

It’s clear by now that LLMs can power use cases in almost every business function: from document processing to lead generation and everything in between. However, each use case has specific trade-offs around flexibility, reliability, cost and latency. While document processing might follow a well-defined sequence of steps, lead generation requires more flexibility to find and extract the right information. These trade-offs rarely favour open-ended AI agents, especially for high-volume use cases. Luckily, workflows allow you to control them!

An AI agent hammering a nail — Each use case has its own requirements. Don't use AI agents like a hammer and every use case as a nail!

Flexibility vs. reliability — pick two!

Agents are just nodes within workflows. How much agency you give each node is up to you. While some nodes are simple LLM calls (e.g. to extract information from a document), others require tools like search and access to external data (e.g. to find requested items in a product catalog). Give more agency to nodes which solve open ended tasks, restrict those which are narrow and well-defined. This puts you firmly in control of the flexibility vs. reliability trade-off for each step in your workflow so you can combine the best of both worlds.

Tool-cooling agents and simple LLM calls being combined in the same workflow — Combine tool-calling agents and simple LLM calls in the same workflow.

Workflows are cheaper and faster?!

Agents which require multiple reasoning steps are slower and burn a lot of extra tokens compared to a well-defined workflow. There are a few reasons for this:

By defining the workflow upfront, the agent doesn’t have to spend as much time checking the correctness of each reasoning step and fixing possible mistakes.
Because each workflow step has a well-defined context, we can cache a lot of it in advance and avoid burning a lot of tokens for intermediate results, reducing context bloat.
Not every step has to be AI-powered! Think about all the business logic or formulas embedded in spreadsheets which are basic data transformations. You can often include them as workflow steps instead of executing them via agentic function calling, saving even more tokens.

The debugging experience: agents vs. workflows

Anyone who has had the pleasure of debugging opaque AI agents knows how tricky this can be. Since there are many points of failure, the user needs to comb through reasoning traces, tool calls and model predictions to figure out the issue. Is the model not calling the right tool? Is the search tool not finding the right documents? Is the model not extracting the information from the retrieved documents correctly? Or is it a combination of factors?

Workflows allow you to disentangle these steps if needed and isolate points of failure. This makes it a lot easier to identify and fix issues, both during workflow building and debugging. You can run the workflow up until the problematic node, and focus on improving that node’s performance.

But don’t workflows get messy quickly?

The reality is that workflow diagrams can reflect the complexity of the process being automated. But be aware: this complexity is still there when moving from a no-code solution to a coding framework! It is just hidden under another layer of abstraction, less transparent for the domain expert, only accessible to the developer.

The big difference with the days of Yahoo! Pipes or Windows Workflow Foundation is that the workflow builder can now choose how much functionality is abstracted away in each node, and that they can communicate this in natural language. What used to be an entangled mess of 20 single-purpose nodes, can now be grouped into 1, drastically reducing complexity.

A complex workflow being simplified — We can now distill complex diagrams into simpler workflows.

Putting it all together in a reliable decision engine

Of course you only get all these benefits if they are deeply embedded in your workflow platform, which requires careful engineering, from intelligent caching to extensive context engineering, structured output generation, providing best-in-class tools like multi-modal search and much more! And once you move from prototyping to production, you’ll also need enterprise level security and governance controls around data management, deployment options, access and encryption.

This is what Decision Computing delivers: an integrated platform that lets your domain experts focus on automating key business processes without sacrificing flexibility or performance.