Flow Engineering in AI Systems
In this blog, we will explore what Flow engineering is and why it is significant. Unlike traditional prompt engineering, which focuses on crafting individual inputs to elicit desired outputs, flow engineering involves designing comprehensive processes that systematically and iteratively guide large language models (LLMs) through complex tasks. By the end of this blog, you will understand why Flow engineering represents a natural evolution of Prompt engineering in the development of AI applications and how increased Flow engineering can reduce reliance on Prompt engineering.
Why Do We Need Flow Engineering?
Let’s take an example use case of stock analysis of a particular company. Typically, an investment firm performs three kinds of analysis:
Technical Analysis
Fundamental Analysis
Sentimental Analysis
Of course, an investing firm does much more than that, but we will keep it simple for this blog.
A basic approach to solve this is to pass a prompt to a Large Language Model (LLM) and provide it with a web search tool. A web search tool is required to look for current price trends, news articles, etc.
Below is a sample prompt:
Perform a comprehensive analysis of ABC company by evaluating its price from three perspectives: fundamental analysis, technical analysis, and sentiment analysis. Provide actionable insights based on your findings.Instructions 1. Fundamental Analysis - Evaluate the financial health of the company using available metrics (e.g., P/E ratio, EPS, revenue growth, profit margins, and debt levels). - Compare the company’s performance with industry peers. - Discuss recent quarterly results, if applicable. - Highlight any notable changes in the company’s business model, leadership, or market conditions.2. Technical Analysis - Analyze historical stock price trends and key technical indicators (e.g., moving averages, RSI, MACD, and Bollinger Bands). - Identify support and resistance levels. - Highlight patterns or signals such as breakouts, reversals, or trends.3. Sentiment Analysis - Assess public and market sentiment around the stock using news headlines, social media trends, or market reports. - Discuss the tone of recent coverage and its potential impact on the stock price. - Include any significant events or controversies affecting investor sentiment.4. Conclusion - Synthesize the above analyses into a clear and actionable recommendation (e.g., buy, hold, sell). - Justify the recommendation with key points from the analysis. Use concise language and include relevant data or examples where applicable.
You implement this approach; it works great, and your client loves it. Now, let’s assume your client asks for the following:
In the fundamental analysis, include annual results and Price-to-Book (P/B) ratio and Earnings Per Share (EPS).
In the technical analysis, specifically include 50-day and 200-day moving averages and detect candlestick patterns.
In the sentimental analysis, only look for X (formerly Twitter) posts.
Based on this feedback, you go ahead and adjust the prompt to accommodate the above requirements.
-
Perform a comprehensive analysis of ABC company by evaluating its price from three perspectives: fundamental analysis, technical analysis, and sentiment analysis. Provide actionable insights based on your findings.
Instructions
1. Fundamental Analysis
- Evaluate the financial health of the company using available metrics (e.g., P/E ratio, EPS, revenue growth, profit margins, and debt levels).
- Compare the company’s performance with industry peers.
- Discuss recent quarterly and annual results, if applicable.
- Highlight any notable changes in the company’s business model, leadership, or market conditions.2. Technical Analysis
- Analyze historical stock price trends and key technical indicators such as 50-day and 200-day moving averages, RSI, MACD, and Bollinger Bands.
- Identify support and resistance levels. - Highlight patterns or signals such as breakouts, reversals, or trends.3. Sentiment Analysis
- Assess public and market sentiment around the stock using only X (formerly Twitter) posts.
- Discuss the tone of recent coverage and its potential impact on the stock price.
- Include any significant events or controversies affecting investor sentiment.4. Conclusion
- Synthesize the above analyses into a clear and actionable recommendation (e.g., buy, hold, sell).
- Justify the recommendation with key points from the analysis.Use concise language and include relevant data or examples where applicable.
Now you observe that the response is completely different. You note that it has accommodated new changes, but it has failed to respond with certain details that were previously present and are now missing.
Don’t Blame the LLM!
Let’s take a closer look at the fundamental characteristics of LLMs and prompts.
LLMs are inherently probabilistic, meaning their outputs can vary based on input phrasing or randomness. In creative industries, this variability can be an asset. However, in business operations, precise and consistent outputs are critical.
Prompts are notoriously brittle—a small change in the prompt can drastically alter outputs.
So the more instructions you give, the less likely that it will follow all the instructions.
As more specific and complex requirements start coming, the above approach fails to provide consistent and reliable outputs.
Apart from the LLM’s inherent limitations features, there are other issues from the perspective of software system design:
Hard to test: Due to the high variance characteristics of large language models (LLMs), comprehensively testing all possible scenarios presents a challenge. Much like how we typically perform unit tests on a function or class, it is essential to address a wide array of potential queries that users may pose within our evaluation test set.
Tightly Coupled: It is hard to optimize and scale each analysis separately. Modifying an instruction for fundamental analysis will affect the output of technical or sentimental analysis.
To solve for the above problem, we use Flow engineering technique to break down the complex problem into multiple simple steps.
Understanding Flow Engineering
At its core, Flow engineering seeks to emulate human problem-solving by breaking down intricate tasks into manageable steps. This approach aligns with "System 2" thinking, characterized by deliberate and analytical reasoning, as opposed to the intuitive and rapid "System 1" thinking. (“System 1” and “System 2” thinking is based on the book Thinking Fast & Slow by Daniel Kahneman).
From the AI systems perspective, it can be defined as follows:
Flow engineering is the systematic process of effectively splitting, separating, and simplifying large language model (LLM) prompts and tasks. This method involves forcing the application to operate within clearly defined states and transitions, akin to the principles of a finite state machine. By doing so, it enhances clarity and efficiency in handling complex tasks, allowing for more structured interactions and improved performance.
Yes, in the software engineering domain (or any other domain for that matter), it is not a new concept. We have seen this pattern at the lowest level of splitting code to multiple functions/classes to higher levels of system design such as micro-services architecture.
Now, let’s modularize our use case into a state machine. We will create 4 components:
Technical analysis component
Fundamental analysis component
Sentimental analysis component
Summarizer component
Each analysis component will have its own LLM, prompt, and web search tool. The responses from each component will be sent to a summarizer that also has its own LLM and prompt. This approach allows for the iterative evaluation and optimization of each component independently, resulting in a more robust and dependable Compound AI System.
With the technique of Flow engineering, we can easily customize and adapt to various future requirements and optimizations easily such as:
Add a new analysis component like Risk Analysis
Use different Language Models (LMs) for different components
Add a new feedback component to refine the final summarized response further. An example of cyclic flows.
Add a router component to route user query to specific analysis component
Summarizing Advantages Over Traditional Prompt Engineering
Reducing Iterative Cycles: By designing comprehensive workflows, Flow engineering minimizes the need for repetitive prompt adjustments, leading to more efficient development processes.
Enhancing Output Accuracy: Structured workflows allow for systematic testing and validation at each step, resulting in more precise and reliable outputs.
Facilitating Complex Task Automation: Flow engineering enables the automation of intricate processes that would be challenging to achieve through prompt engineering alone.
Implementing Flow Engineering in Practice
To effectively implement Flow engineering in LLM-based applications, consider the following steps:
Define the Workflow: Clearly outline the sequence of steps required to accomplish the task, specifying the inputs, processes, and expected outputs at each stage.
Develop Modular Components: Create self-contained units (flows) that handle specific tasks within the workflow, promoting modularity and ease of integration.
Incorporate Iterative Testing: Implement testing and validation mechanisms at each step to ensure accuracy and identify areas for improvement.
Leverage Tools: Utilize tools and platforms that support Flow engineering principles, such as Langgraph, to streamline development and deployment processes.
Conclusion
Flow engineering represents a significant advancement in the development of LLM-based applications, offering a structured and systematic approach to complex problem-solving. By embracing Flow engineering principles, developers can enhance the efficiency, accuracy, and scalability of AI-driven systems, paving the way for more sophisticated and reliable AI applications in the future.
At Innowhyte, we employ Flow engineering principles and other software patterns to build robust AI systems, taking the AI solution to the last mile. Remember, it’s easy to demo an AI solution, but hard to productionize it. Reach out to us if you need any help in taking your AI solution to the last mile.