Does AI work as a team? Does it really work?
Everyone is talking about LLMs (Large Language Models) now, right? These intelligent language models are getting better every day. Create articles smoothly and come up with reasons. It's no surprise that there are so many cool apps out there.
What's more interesting is that we're starting to use LLMs as "agents" or intelligent personal assistants that can help us do difficult tasks for us, whether it's writing code, summarizing large amounts of data, analyzing complex things, or even helping us make important decisions. These agents will work according to the "prompts" or instructions we enter, which will tell us what to do. What are your goals?
But the top thing is that we don't use just one agent! We can also take multiple LLMs and work together as a team, or what we call a Multi-Agent System (MAS). This kind of teamwork is often much better than solo screenings because it gives you a more diverse perspective, helps each other think, and helps each other do it.
But wait... Building an AI team can be a headache!
Sounds good, right?
But let's face it, to design an AI or MAS team to work perfectly. Especially with new problems that have never been encountered before, it is not easy. The first problem I encountered was that the size of a single agent still had a personal "stitch", which was very sensitive to prompts. Suddenly, the performance may drop in a puzzled way. When I have to work as a team. This problem seems to be even more chaotic.
In addition to the prompt, it is important to design which agent should talk to whom. Who do you send work to? "Team Structure" (Topology) It's like having to hold a needle in the ocean. It takes a lot of trial and error. There are millions and eight options, including endless adjustable prompt formats, and flexible team structures.
Of course, there are people trying to find a solution to this problem, such as DSPy that helps find good examples to create prompts, ADAS that uses LLM to help come up with new team structures, or AFlow that uses clever techniques to help find a good structure. How do Prompt and Topology work together (or do they contradict?) to make our AI team really better?
So, what's the key to building an AI team?
In the research that has been shown to Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies talked about the collaborative design of agents in an interesting way.
This article intends to delve into this by looking at 3 main factors:
- Prompt: The commands we use tell each agent what to do.
- Number of Agents: How many do you use to be healthy?
- Topology: What kind of team should you work with?
We wanted to know how these 3 things affect the overall performance of the AI team, and we found that "Prompt" is the real hero!
Having a good, clear, and appropriate instruction for the job is the most important factor. As for topology or team structure, there are only a few that really make sense.
"MASS": AI Team Wizard for Gods!
When we knew this, we invented a tool called Multi-Agent System Search (MASS) Think of it as a personal "coach" that will make it easier and better to build our AI team!
MASS doesn't do anything complicated, just combines how to improve the prompt with how to find the best team structure, and organize the process into a simple 3-step system:
- Block-Level Prompt Optimization: It is like intensive tutoring for each player to excel in their own position first.
- Find the Right Gameplay Plan (Workflow Topology Optimization): The team structure or topology that seems to work the best.
- Workflow-Level Prompt Optimization: Adjust the prompt again, this time focusing on agents working together seamlessly in the team. Pass the ball to each other exactly.
With MASS, we can actually build AI teams that are better than the traditional methods, and we can also get good guidelines to use to further design AI teams in the future.
A simple summary of what this research has to offer:
- It helps us better understand what are the "secrets" to building a good AI team.
- There is a new tool (MASS) to help AI designers make their lives much easier.
- Prove that MASS can really help build better AI teams!
So how do we design an AI team?
Okay, now let's go into a little more detail about what MAS design is, and what are the importance of Prompt and Topology?
We will call the "structure" of work "Topology", and the "process" of all work is "Workflow", so the design of an AI or MAS team will have 2 major levels:
- Block-Level Design: See what each agent needs to do best. This is mainly focused on "Prompt".
- Workflow-Level Design: See how agents work best together. This is the focus on "Topology".
1. Tune each agent with Prompt (Block-Level)
At the heart of each agent is a "prompt", it's like both a "role" (e.g., you're a "bug analyst") and a "work guide" (e.g., "Think step by step", "Take this example").
Cool tools Nowadays (Automatic Prompt Optimization - APO) can help us find both the best suggestions and examples automatically, but when it comes to applying it to the MAS team, it's not as easy as peeling a banana in your mouth.
- The AI team works in succession. Continue to pass the stick.
- We don't have an "answer" on the way about how well each agent works.
- The bigger the team, the more complex it is. Finding the best prompt for everyone is even more difficult.
This is why most MAS research still uses prompts written by people. In this event, we will try to see if we can use APO to help adjust the prompt, is it better than increasing the number of stubborn agents?
2. Design a game plan for the team (Workflow/Topology)
Let's talk about team management or Topology Optimization. This is a relatively new concept, but it is becoming more and more popular.
But it is worth noting that most of the focus is on "how to find" the best game plan (Search Method), but we forget to pay attention to the "game plan options" that are available (Search Space), which is actually equally important!
This is similar to when the Neural Architecture Search (NAS) industry started to boom, at that time it focused on how to find complex AI structures, but later I discovered that it was much more important to have a good "structure choice" from the beginning.
So we think that the game plan or topology that we think of ourselves may not be the best option. With the help of a computer (Automatic Topology Optimization), we should be able to find something that works better, so we set the commonly used "Building Blocks" as an option for the search system. As follows:
- Brainstorming (Aggregate): Many people help each other to think and then vote/summarize the best answer.
- Reflect: One person does it, the other checks it, and recommends corrections (like someone reviewing the work).
- Debate: Many people come to debate. Exchange views, find the best conclusions
- Custom Agents: Agents with special duties can be added, such as data summarizers.
- Tool-Use: Allows agents to use external tools, such as searching the net or running test code.
How does MASS work?
Imagine that we are "coaches" who have to build an AI team to solve difficult problems. We have a lot of players (agents) in our team. Each person may be good at different things. How do we plan to be the best at our team? You have to think about both the "duties" of each person (Prompts) and the "game plan" of the team (Topology), and there are so many ways to combine them!
MASS is the "assistant coach" who will help us here. There are 3 main steps:
1. Block-Level Prompt Optimization:
- Why do it? We have to mold each player to be good in their position first. The striker must shoot sharply. The defender had to intercept accurately.
- MASS will help you adjust the prompts and find good examples for each agent to practice until they are good at their assigned tasks.
- How important is it? If each player is not good at their specialty. Playing as a team won't survive, right?
2. Choose a Workflow Topology Optimization:
- Why do it? Once the players are good, they have to find the best game plan for the team and the opponent.
- MASS will help you find the topology or game plan (e.g. Aggregate, Reflect, Debate) that will work the most.
- How important is it? Having a good plan helps players who are already good at working together even better.
3. Workflow-Level Prompt Optimization:
- Once you have a plan, you have to practice to get it together. Passing the ball with his heart, running at the right time.
- MASS will adjust the agent's prompt again, but this time the emphasis is on "pick-up-delivery". Information between agents on the map we have chosen. It's the smoothest, the teamwork is smooth.
- How important is it? Even if the players are good and have a good plan, if they don't play together, it will be difficult to win. This step of tuning helps to increase the power of "teamwork".
The result?
The results are very satisfying! AI teams built with MASS are obviously better than teams built in other ways. Especially teams that don't improve the prompt well enough.
Whether it is math problems (MATH), reading comprehension (HotpotQA), or coding (HumanEval), teams that have been "coached" with MASS score significantly better.
We also tried to "disassemble" the MASS one at a time ( Ablation Study To see how important each step is, I found that every step really contributes to making the team better, not just one step.
Conclusion: Building a good AI team is no longer difficult?
There are a lot of challenges in building an AI team or a Multi-Agent (MAS) system to work well together, including writing the prompt correctly (which we found to be extremely important from our research!). And choosing the topology or team structure that is really suitable for the job.
Therefore, this research presents MASS as a "coach" or framework that helps to simplify the design of AI teams with 3 main steps. From choosing a team structure to tuning in to work together seamlessly.
The results are clear that MASS has helped us build AI teams that are not only better, but actually collaborate more effectively. Who is working on this kind of project? Let's apply these principles and tools!