Optimizing Multi‑Agent LLM Collaboration via Communication‑Flow Control

Multi-agent systems based on Large Language Models (LLMs) have shown promise in automating complex engineering tasks, including code generation for Finite Element Analysis (FEA). However, despite models demonstrating the capability to produce correct solutions, task success rates remain low. This study investigates the mechanisms limiting the effectiveness of multi-agent LLM collaboration through two complementary perspectives: agent role composition and communication flow dynamics.

We evaluate seven agent combinations across four increasingly complex FEA tasks through 1,120 controlled experimental trials using the AutoGen framework with GPT-3.5-Turbo. From the agent-role perspective, we find that neither adding more reviewers nor varying team composition yields consistent performance gains; instead, agents designed to provide critical evaluation fail to do so in practice. From a communication-flow perspective, we identify early agreements (i.e., approval messages such as ‘looks good’ or ‘execution succeeded’ that may falsely indicate task completion) as a dominant failure mechanism that prematurely halts iterative code refinement, leaving the model’s coding capability systematically underutilized.

Based on these findings, we suggest an approach to optimize communication flow: removing misleading agreements while preserving necessary guidance. This approach significantly improves task success rates (Task 2: 10.7% to 77.5%; Tasks 3-4: 0.7% to 45.0%) without requiring additional training or external information. These results establish that communication dynamics, rather than agent quantity or role diversity, are the primary determinant of multi-agent collaboration effectiveness in engineering code generation.