\section{Multi-Agent Inference: \workforce}\label{method}

\begin{figure}[t]
    \centering
    \includegraphics[width=\linewidth]{images/workforce.pdf}
    \caption{Overview of the \workforce framework. The system consists of a Planner Agent for task decomposition, a Coordinator Agent for orchestrating subtasks, and multiple specialized Worker Nodes equipped with domain-specific toolkits to execute assigned tasks.}
    \label{fig:workforce}
\end{figure}

\subsection{\workforce} \label{sec:workforce}

% \subsubsection{Architecture and Task Flow}
% \label{method:architecture}

\noindent \textit{\textbf{Motivation.}}
Contemporary multi-agent systems are critically limited by domain specificity and architectural rigidity, requiring complete redesign and retraining for each new application domain.
We introduce \workforce, which addresses this fundamental challenge through modular architecture, particularly the separation of domain-agnostic planning from domain specific execution. More details about \workforce can be found in Appendix~\ref{app:workforce_details}.

\noindent \textit{\textbf{Architecture.}} 
As illustrated in Figure~\ref{fig:workforce}, \workforce comprises three core components:
\textit{(i)} \textbf{Planner Agent} analyzes incoming tasks and decomposes them into subtasks based on worker capability registry;
\textit{(ii)} \textbf{Coordinator Agent} serves as the central orchestration mechanism, managing worker assignments and task dependencies while integrating intermediate results;
\textit{(iii)} \textbf{Worker Nodes} consist of one or more specialized agents equipped with specific capabilities and toolkits that execute assigned subtasks and post results.
This modular architecture provides inherent flexibility, allowing the framework to be deployed across diverse applications by simply modifying the worker nodes while maintaining its core planning and coordinating mechanism. 

\noindent \textit{\textbf{Communication Mechanism.}}
Communication in \workforce operates through a shared \textit{task channel} that serves as a central hub. The coordinator posts tasks and assignments to this channel. Upon completion, workers post only their final results back to the channel, while the detailed execution context of tool calls remains isolated within each subtask's scope. This maintains a clean context for each worker, who only has access to the current subtask details and concise previous subtask results. This centralized approach simplifies system management and enhances scalability by eliminating direct agent-to-agent messaging.

% \subsubsection{Task Flow}
% \label{method:taskflow}

\noindent \textit{\textbf{Task Flow.}}
The task processing workflow within \workforce follows a structured pipeline:
\textit{(i)} The Planner analyzes incoming overall tasks and then decomposes them into a set of subtasks according to the capabilities of available worker nodes and the overall task's complexity;
\textit{(ii)} The Coordinator assesses the capabilities of available worker nodes and dispatches subtasks accordingly;
\textit{(iii)} The Worker Nodes perform the assigned subtasks using their specialized tools;
\textit{(iv)} Results from worker nodes are posted to the shared \textit{task channel};
\textit{(v)} The Coordinator manages task dependencies and integrates results, ultimately forwarding them to the Planner;
\textit{(vi)} The Planner analyzes the results of each subtask and synthesizes the final output.


\noindent \textit{\textbf{Replanning Mechanism.}}
During task execution, workers self-assess whether the assigned subtask is failed. When a worker determines that a subtask has failed, it posts failure information to the task channel.
The task channel then detects this failure, and the planner will be prompted to generate new subtasks based on the feedback information. 
This replanning mechanism enables test-time scaling by dynamically adjust its approach to increasingly complex tasks, which is validated in Section~\ref{sec:analysis}.


\subsection{Generalist Multi-Agent Assistance}
\label{method:agent_specialization}

To construct a generalist multi-agent assistant capable of processing diverse real-world tasks, we instantiated \workforce with three worker agents, each equipped with domain-specific toolkits:
\textit{(i)} A \textbf{Web Agent}, capable of performing web searches, extracting webpage content, and simulating browser actions;
\textit{(ii)} A \textbf{Document Processing Agent}, designed to process documents and multimodal data, including text, images, audio, video, spreadsheets, and various file formats; and
\textit{(iii)} A \textbf{Reasoning/Coding Agent}, which handles analytical reasoning and code execution tasks.
The details of worker agents and corresponding toolkits could be found in Appendix~\ref{app:worker_agent_details}.

% This modular design enables the \workforce system to handle a wide range of tasks in heterogeneous modalities, while maintaining efficiency, extensibility, and interpretability.
% The description of each worker node plays a crucial role in the system, as the coordinator agent uses these descriptions to make informed decisions about task assignment, ensuring that tasks are routed to the most appropriate worker nodes based on their specialized capabilities.

\subsection{Experiments}


% \begin{revisionbox}{Experiments: Inference [YR]}
%   \item Clearly state why each baseline is included and what it demonstrates.
%   \item Quantify all improvements; highlight key numbers in text and tables.
%   \item Emphasize how your approach closes the gap with proprietary systems.
%   \item Add statistical significance if feasible (confidence intervals, significance tests).
%   \item Use summary tables/figures to make key results visible.
%   \item \textbf{Quick checks:}
%     \begin{itemize}
%       \item [ ] Are all baselines justified?
%       \item [ ] Are improvements/gaps quantified?
%       \item [ ] Are summary tables/figures present and referenced?
%     \end{itemize}
% \end{revisionbox}

% \subsubsection{Experimental Setup}

% \noindent \textit{\textbf{Benchmark}}
% We evaluate OWL on the GAIA~\citep{gaia}, a comprehensive question answering benchmark designed to assess General AI Assistants in realistic environments. 
% GAIA tests fundamental abilities essential for general-purpose AI assistants, including multi-modal understanding, web browsing, reasoning, and sophisticated problem-solving capabilities.
% Based on the different tools required and the number of steps needed to complete tasks, GAIA tasks are further categorized into three distinct levels.
% We selected the validation set of GAIA for evaluation, which comprises 165 realistic agentic tasks.
% % It consists of 466 questions, stratified across three distinct difficulty levels. 
% % This stratification allows for nuanced evaluation of agent capabilities across varying task complexities.


% \noindent \textit{\textbf{Evaluation Metrics}}
% We employ accuracy as our primary metric, which first post-processes the LLM-generated answer with rules and then calculates its exact match with the ground truth answer.
% % This metric, which measures the percentage of tasks successfully completed, provides a clear indicator of \workforce's effectiveness in handling real-world, human-like challenges that often require multiple steps of reasoning and tool interaction.

\noindent \textit{\textbf{Baselines.}}
We selected a comprehensive set of baseline systems, categorized into four main groups:
\textit{(i)} \textbf{Proprietary frameworks} establish upper-bound commercial performance, including commercial agentic systems like OpenAI's Deep Research~\citep{openai2024deepresearch}, h2oGPTe Agent~\citep{h2oai2025enterpriseh2ogpte}, etc.
% : Recently, numerous companies have released general AI assistant products. For example, OpenAI introduced Deep Research~\citep{openai2024deepresearch} for completing multi-step research tasks, while other commercial entities have developed similar proprietary solutions.
\textit{(ii)} \textbf{Open-source frameworks} reveal the community progress, including strong baselines like HuggingFace's Open Deep Research~\citep{roucher2025deepresearch}, Trase Agent~\citep{trasesystems2025}, etc.
% Many open-source initiatives have aimed to construct frameworks with performance comparable to proprietary systems. For instance, Hugging Face's smolagents developed an Open Deep Research~\cite{roucher2025deepresearch} implementation based on CodeAct~\citep{wang2024executable} agent architecture to replicate the capabilities of Deep Research.
\textit{(iii)} A \textbf{Single Agent} baseline utilizing multi-step tool calling.
% capabilities to establish a performance reference point for our multi-agent approach.
\textit{(iv)} \textbf{Role Playing}~\cite{camel} comprising two agents (user agent and assistant agent) that collaborate through structured dialogue to accomplish tasks.
Note that to control experimental variables, both the Single Agent and Role Playing baselines utilize identical tool sets to \workforce. 

\noindent \textit{\textbf{Implementation Details.}}
\label{sec:implementation_details}
For our implementation, we access all models via APIs, eliminating the need for GPUs. 
To ensure reproducibility, we configure API inference with greedy decoding. 
The default replanning threshold is set to 2.
For evaluation methodology, we employ pass@3 sampling for \workforce with GPT-4o~\citep{gpt4o} and pass@1 for \workforce with Claude-3.7-sonnet~\citep{claude37}. Since some gold answers from GAIA had been leaked online, we blocked several websites to ensure fair comparison~\citep{openai2024deepresearch}.


% \subsubsection{Main Results}

\noindent \textit{\textbf{Main Results.}}
Several conclusions could be drawn from Table~\ref{tab:main-results}:

\textit{(i)} \textbf{Workforce achieves state-of-the-art performance among open-source frameworks.} 
    Our \workforce achieves \textbf{69.70\%}, consistently outperforming previous open-source frameworks across all difficulty levels. 
    Under strictly controlled settings with the same model and toolkits, our GPT-4o-based \workforce achieves \textbf{60.61\%} accuracy - \textbf{23.03\%} higher than the Single Agent and \textbf{6.06\%} higher than the multi-agent baseline Role Playing.
    % Notably, under strictly controlled conditions (with identical available toolkits and models), \workforce significantly outperforms the single agent baseline (60.61 vs. 37.58) and surpasses the previous state-of-the-art role playing approach (60.61 vs. 58.18).
    
\textit{(ii)} \textbf{Workforce demonstrates comparable or even superior performance relative to commercial proprietary frameworks.} 
    While previous open-source frameworks have exhibited a substantial performance gap compared to closed-source alternatives, \workforce considerably narrows this divide. 
    % For instance, \workforce exceeds OpenAI's Deep Research~\citep{openai2024deepresearch} in average score (69.70 vs. 67.36) and surpasses the current state-of-the-art Langfun Agent v2.1 on Level 1 score (84.91 vs. 83.02).
    To our knowledge, \workforce is the first open-source system to exceed OpenAI's Deep Research, achieving a \textbf{2.34\%} improvement, and also establishes a new Level 1 best by outperforming Langfun Agent v2.1 by \textbf{1.89\%}.

% \begin{itemize}
%     \item \textbf{Workforce achieves state-of-the-art performance among open-source frameworks.} 
%     Our \workforce achieves \textbf{69.70\%}, consistently outperforming previous open-source frameworks across all difficulty levels. 
%     Under strictly controlled settings with the same model and toolkits, our GPT-4o-based \workforce achieves \textbf{60.61\%} accuracy - \textbf{23.03\%} higher than the Single Agent and \textbf{6.06\%} higher than the multi-agent baseline Role Playing.
%     % Notably, under strictly controlled conditions (with identical available toolkits and models), \workforce significantly outperforms the single agent baseline (60.61 vs. 37.58) and surpasses the previous state-of-the-art role playing approach (60.61 vs. 58.18).
    
%     \item \textbf{Workforce demonstrates comparable or even superior performance relative to commercial proprietary frameworks.} 
%     While previous open-source frameworks have exhibited a substantial performance gap compared to closed-source alternatives, \workforce considerably narrows this divide. 
%     % For instance, \workforce exceeds OpenAI's Deep Research~\citep{openai2024deepresearch} in average score (69.70 vs. 67.36) and surpasses the current state-of-the-art Langfun Agent v2.1 on Level 1 score (84.91 vs. 83.02).
%     To our knowledge, \workforce is the first open-source system to exceed OpenAI's Deep Research, achieving a \textbf{2.34\%} improvement, and also establishes a new Level 1 best by outperforming Langfun Agent v2.1 by \textbf{1.89\%}.
% \end{itemize}

\begin{table}[t]
\caption{Performance comparison of agent frameworks on GAIA~\citep{gaia} validation set with accuracy score (\%) as the evaluation metric. 
% We adopt the original accuracy score proposed by GAIA as the evaluation metric.
Scores of open-source and proprietary frameworks were obtained from the~\href{https://huggingface.co/spaces/gaia-benchmark/leaderboard}{official leaderboard.}
% \footnote{https://huggingface.co/spaces/gaia-benchmark/leaderboard}.
The best-performing proprietary and open-source frameworks are highlighted in \textbf{bold}.}
\centering
\scalebox{0.9}{
    \begin{tabular}{lclccc}  
        \toprule
        Agent Name & Base Model & Level 1 & Level 2 & Level 3 & Average \\
        \midrule
        \rowcolor{gray!15} \multicolumn{6}{c}{\textit{Proprietary Frameworks}} \\
        \midrule
        % AgentIM v1.1 & GPT-4-Turbo & 50.94 & 40.70 & 15.38 & 40.00 \\
        % LRC-Huawei & - & 52.83 & 43.02 & 7.69 & 40.61 \\
        % Agents-On-the-fly & QwQ-32B & 52.83 & 43.02 & 7.69 & 40.61 \\
        % qbnlp & O1 & 52.83 & 45.35 & 23.08 & 44.24 \\
        DRP-val-v.1.0 & - & 56.60 & 48.84 & 15.38 & 46.06 \\
        omne & O1-Preview$^\dagger$ & 60.38 & 44.19 & 23.08 & 46.06 \\
        Barcelona v0.1 & Claude-3.5-Sonnet$^\dagger$ & 62.26 & 50.00 & 26.92 & 50.30 \\
        Ormind v0.1 & Claude$^\dagger$ & 69.81 & 54.65 & 26.92 & 55.15 \\
        desearch & GPT-4o & 71.70 & 58.14 & 23.08 & 56.97 \\
        Anges & Claude$^\dagger$ & 66.04 & 65.12 & 30.77 & 60.00 \\
        h2oGPTe Agent v1.6.8~\citep{h2oai2025enterpriseh2ogpte} & Claude-3.5-Sonnet & 67.92 & 67.44 & 42.31 & 63.64 \\
        Deep Research~\citep{openai2024deepresearch} & O3 & 78.66 & \textbf{73.21} & \textbf{58.03} & 67.36 \\
        Trase Agent v0.3~\citep{trasesystems2025} & Claude$^\dagger$ & \textbf{83.02} & 69.77 & 46.15 & 70.30 \\
        Langfun Agent v2.1$^*$~\citep{peng2023langfun} & Claude-3.7-Sonnet$^\dagger$ & \textbf{83.02} & 68.60 & 57.69 & \textbf{71.52} \\
        \midrule
        \rowcolor{gray!15} \multicolumn{6}{c}{\textit{Open-Source Frameworks}} \\
        \midrule
        FRIDAY~\citep{wu2024copilot} & GPT-4-Turbo & 45.28 & 34.88 & 11.54 & 34.55 \\
        Multi-Agent Exp v0.1~\citep{microsoft2024gaiamultiagent} & GPT-4-Turbo & 54.72 & 38.37 & 11.54 & 39.39 \\
        HuggingFace Agents~\citep{smolagents} & GPT-4o & 58.49 & 43.02 & 19.23 & 44.24 \\
        Magnetic-One~\citep{fourney2024magenticonegeneralistmultiagentsolving} & O1$^\dagger$ & 56.60 & 46.51 & 23.08 & 46.06 \\
        AutoAgent~\citep{tang2025autoagent} & Claude-3.5-Sonnet & 71.70 & 53.49 & 26.92 & 55.15 \\
        Open Deep Research~\citep{roucher2025deepresearch} & O1 & 67.92 & 53.49 & 34.62 & 55.15 \\
        TapeAgents~\citep{bahdanau2024tapeagents} & Claude-3.7-Sonnet & 71.70 & 53.49 & 30.77 & 55.76 \\
        Single Agent & GPT-4o & 52.83 & 34.88 & 15.38 & 37.58 \\
        Role Playing~\citep{camel} & GPT-4o & 75.47 & 52.33 & 19.23 & 54.55 \\
        Role Playing$^{\ddagger}$~\citep{camel} & GPT-4o$^\dagger$ & 81.14 & 54.65 & 23.08 & 58.18 \\
        \midrule
        \rowcolor{gray!15} \multicolumn{6}{c}{\textit{Ours}} \\
        \midrule
        \rowcolor{blue!10} \workforce & GPT-4o & 81.14 & 58.14 & 26.92 & 60.61 \\
        \rowcolor{blue!10} \workforce & Claude-3.7-Sonnet & \textbf{84.91} & \textbf{68.60} & \textbf{42.31} & \textbf{69.70} \\
        \bottomrule
    \end{tabular}
}
\label{tab:main-results}
\end{table}

\footnotetext{
    \noindent $^\dagger$ Indicates systems that use multiple models, and only the primary model is listed. \\
    $^*$ Indicates frameworks that have released an early version but have not provided the latest version that can replicate these results. \\
    $^{\ddagger}$ Indicates an optimized version of role-playing where we introduced an LLM-based classifier that switches to O3-mini when tasks require more reasoning and coding, and uses GPT-4o otherwise. 
    }


% 
% 
% ------------------------------ Training --------------------------------
%


\section{Multi-Agent Training: \owl}

\subsection{Training Strategy}
\label{sec:training_strategy}

\noindent \textit{\textbf{Motivation.}}
The \workforce architecture separates domain-agnostic planning from domain-specific execution, allowing us to adapt to new domains by simply adding or replacing Worker Nodes while preserving the core planning mechanism. We introduce \textbf{\owl (OWL)}, which focuses on enhancing a generalizable Planner Agent capable of handling diverse real-world scenarios. This design significantly reduces training overhead since only the Planner Agent requires intensive optimization, while Worker Nodes can leverage existing domain-specific tools with minimal adaptation. This ``stable core, variable periphery'' approach enables efficient knowledge transfer across domains and eliminates the need to retrain entire systems for new applications, substantially reducing computational costs while maintaining consistent performance.

\noindent \textit{\textbf{Implementation.}}
More specifically, we adopt a two-phase training paradigm:
\textit{(i)} In the first phase, we employ supervised fine-tuning (SFT) to initialize the Planner Agent with fundamental task decomposition skills derived from expert demonstrations. 
\textit{(ii)} Subsequently, we utilize reinforcement learning to further optimize the SFT-initialized Planner Agent. We select Direct Preference Optimization (DPO)~\citep{rafailov2023direct} as our optimization algorithm of choice, as this phase enhances the quality of decomposition strategies beyond mere imitation of demonstrations, enabling the Planner to develop more sophisticated decision-making capabilities.

% The Planner's capabilities effectively determine the upper bound on overall system performance, as even highly capable worker agents cannot compensate for poor initial task decomposition.

\subsection{Task Curriculum}

\noindent \textit{\textbf{Motivation.}}
The core innovation of \workforce is its architectural separation of domain-agnostic planning from domain-specific execution. For this design to be effective, the Planner Agent must possess robust generalization capabilities across diverse problem domains. 
% While Worker Nodes can be specialized and swapped for different applications, the Planner represents the system's critical cognitive core. 
This creates a fundamental tension: the Planner must simultaneously maintain a deep understanding across disparate domains while avoiding overfitting to specific task patterns or domains.
To address this challenge, we developed a strategically balanced task curriculum that deliberately spans multiple capability dimensions required for general intelligence. Our curriculum design is guided by two key principles:
\textit{(i)} \textbf{Capability Coverage}: Exposing the Planner to diverse reasoning patterns and problem structures.
\textit{(ii)} \textbf{Transfer Learning}: Prioritizing tasks that develop complementary cognitive skills which can transfer across domains, rather than domain-specific knowledge.

\noindent \textit{\textbf{Implementation.}}
More specifically, as is shown in Table~\ref{tab:dataset_capabilities}, we carefully selected four datasets, each targeting distinct agent cognitive capabilities dimensions: 
\textit{(i)} \textbf{HotpotQA}~\citep{yang2018hotpotqa}: This dataset requires multi-hop reasoning based on online information, challenging the Planner to orchestrate complex information-seeking behaviors. 
\textit{(ii)} \textbf{WikiTableQuestions}~\citep{wtq}: This dataset requires the Planner to formulate strategies for navigating, filtering, and operating over tabular information. 
\textit{(iii)} \textbf{Math-related Problems}: This is a custom-curated collection of mathematical problems that require reasoning or coding to solve, covering various mathematical domains. They help the Planner develop skills in logical reasoning and computational problem-solving.
\textit{(iv)} \textbf{Infinity-MM}~\citep{infinitymm}: As a multimodal dataset, Infinity-MM challenges the Planner to orchestrate multimodal information processing., including vision, text, and structured data. 


\newcommand{\agenticon}[1]{\includegraphics[height=1.2em]{#1}}

\begin{figure}[t]
  \centering
  \begin{minipage}[c]{0.64\textwidth}
    \centering
    \captionof{table}{%
      The statistics of the dataset.
      We use icons to represent different agent capabilities:%
      \includegraphics[height=1em]{images/multi_agent_training_pdf/Web-Browsing}\,Web Browsing; 
      \includegraphics[height=1em]{images/multi_agent_training_pdf/Reasoning}\,Reasoning; 
      \includegraphics[height=1em]{images/multi_agent_training_pdf/Document_Processing}\,Document Processing; 
      \includegraphics[height=1em]{images/multi_agent_training_pdf/coding}\,Coding; 
      \includegraphics[height=1em]{images/multi_agent_training_pdf/multimodal1}\,Multimodal Handling.
      More details of training data can be found in Appendix~\ref{app:trajectory_details}.
    }
    \resizebox{\linewidth}{!}{%

      \begin{tabular}{lccccc}
        \toprule
        \textbf{Dataset}    & \textbf{\# Total} & \textbf{\# SFT Filtering} & \textbf{\# DPO Filtering} & \textbf{Agent Capabilities} \\
        \midrule
        HotpotQA            &  998       & 495          &    354   & 
          \agenticon{images/multi_agent_training_pdf/Web-Browsing}
          \agenticon{images/multi_agent_training_pdf/Reasoning}
          \agenticon{images/multi_agent_training_pdf/multimodal1} \\
        WikiTableQuestions  &  869       & 577          &   311    & 
          \agenticon{images/multi_agent_training_pdf/Document_Processing}
          \agenticon{images/multi_agent_training_pdf/Reasoning} \\
        Math-related        & 1100       & 487          &    297   & 
          \agenticon{images/multi_agent_training_pdf/Reasoning}
          \agenticon{images/multi_agent_training_pdf/coding} \\
        Infinity-MM         &  499       & 40           &   47    & 
          \agenticon{images/multi_agent_training_pdf/Reasoning}
          \agenticon{images/multi_agent_training_pdf/multimodal1} \\
        \midrule
        Total/Average       & 3466       & 1599         &  1009   & \\
        \bottomrule
      \end{tabular}
    }

    \label{tab:dataset_capabilities}
  \end{minipage}%
  \hfill%
  \begin{minipage}[c]{0.34\textwidth}
    \centering
    \subfloat[Ablation study on trajectory filtering\label{fig:ablation}]{
      \includegraphics[width=\linewidth]{images/ablation_study_of_training.pdf}
    }
  \end{minipage}
\end{figure}



% \noindent \textit{\textbf{Dataset Construction}}
\subsection{Trajectory Synthesis}
\label{sec:trajectory_synthesis}

\noindent \textit{\textbf{Supervised Fine-tuning.}}
We employed our \workforce approach (\S \ref{sec:workforce}) with GPT-4o-mini to synthesize expert trajectories, which consist of planner-generated subtasks and worker-generated execution traces.
To filter out low-quality data, we applied different evaluation metrics across datasets: 
For HotpotQA and WikiTableQuestions, we used accuracy metrics; 
For Infinity-MM, we used text cosine similarity with a 0.7 threshold between ground truth and generated answers;
For math-related problems, we implemented LLM-as-a-judge to compare ground truth answers with workforce-generated solutions.
Ultimately, as shown in Table~\ref{tab:dataset_capabilities} and~\ref{tab:more_synthesis_stats}, we obtained 1,599 filtered trajectories for supervised fine-tuning, with each trajectory containing an average of 3.41 subtasks.

\noindent \textit{\textbf{Reinforcement Learning.}}
We use the SFT-initialized model to generate pair-wise trajectories for DPO~\citep{rafailov2023direct}. 
Specifically, for each question in our collected dataset, we roll out $n=4$ distinct trajectories.
The evaluation of these generated trajectories is the same as in the SFT stage. 
Then, we construct preference pairs from the $n$ trajectories generated for each question based on their evaluation outcomes.
For Math, HotpotQA, and WikiTableQuestions tasks, trajectories that are correct are labeled as ``chosen'', while incorrect answers are labeled as ``rejected''. 
For Infinity-MM dataset, trajectories whose final text cosine similarity scores exceeded the same threshold as in the SFT stage (0.7) are labeled as ``chosen'', while those below the threshold are labeled as ``rejected''.
As shown in Table~\ref{tab:dataset_capabilities}, we collect 1009 filtered trajectories pairs.
% In our experiments, we set $n$ to 4.
% \mk{Rl details}


\subsection{Experiments}
\label{sec:exp-training}


\noindent \textit{\textbf{Baselines.}}
We compare our approach against multiple proprietary and open-source models as baselines, including the GPT-4o series~\citep{gpt4o}, Claude-3.7-Sonnet~\citep{claude37}, and the Qwen2.5 series~\citep{qwen2}. 
These models represent the current state-of-the-art in language model capabilities across different scales and architectures.

% \noindent \textit{\textbf{Training Setup}}

\noindent \textit{\textbf{Implementation Details.}}
% \mk{hyper-parameters of training}
Our model training is conducted on a computing cluster equipped with 8 NVIDIA H100 GPUs. 
We use the LlamaFactory~\citep{zheng2024llamafactory} framework for managing and executing our training procedures. 
Specifically, for all the models we trained, the input sequences are truncated to a maximum length of 32,768 tokens and the learning rate is set to $10^{-5}$.
All models are trained for a total of two epochs. 
To optimize memory usage and training efficiency, we use bfloat16 mixed-precision training. 
The effective batch size is 12, achieved through 1 per-device batch size combined with 12 gradient accumulation steps.


% \subsubsection{Main Results}

\noindent \textit{\textbf{Main Results.}}
Table~\ref{tab:planner_comparison} reveals multiple important findings:

\textit{(i)} \textbf{\shortowl significantly enhances planner capabilities, enabling open-source models to outperform proprietary alternatives.} 
    The OWL-trained Qwen2.5-32B-Instruct model demonstrates remarkable improvements with a substantial \textbf{16.37\% }gain.
    OWL enables open-source model reaches \textbf{52.73\%} performance, surpassing both the proprietary GPT-4o-mini (47.27\%) and the larger Qwen2.5-72B-Instruct (49.09\%).
    While GPT-4o (60.61\%) still maintains an advantage, our OWL-trained model achieves comparable performance to GPT-4o (26.92\%) on the more challenging Level 3 tasks.
    
\textit{(ii)} \textbf{Reinforcement learning significantly boosts planner generalization.} 
    While supervised fine-tuning (SFT) alone improves performance on simpler tasks (Level 1 and 2), it shows a regression on the most complex tasks (Level 3), with performance dropping by 3.85\%. 
    However, when combined with DPO, our approach not only recovers this performance but significantly exceeds the base model across all difficulty levels, with a \textbf{7.69\%}improvement on Level 3 tasks.

\begin{table}[t]
    \centering
    \caption{Performance comparison between different planners. Each result is based on the Workforce framework with \texttt{GPT-4o} as the worker foundation model.}
    \scalebox{0.9}{
    \label{tab:planner_comparison}
    \begin{tabular}{lllll}
        \toprule
        \textbf{Planner} & \textbf{Level 1} & \textbf{Level 2} & \textbf{Level 3} & \textbf{Average} \\
        \midrule
        \midrule
        % \multicolumn{5}{l}{\textit{Proprietary Models}} \\
        % \midrule
        GPT-4o-mini~\citep{gpt4o} & 64.15 & 45.34 & 19.23 & 47.27 \\
        Qwen2.5-72B-Instruct~\citep{qwen2} & 60.30 & 51.16 & 19.23 & 49.09 \\
        Claude-3.7-Sonnet~\citep{claude37} & 81.13 & 53.49 & 34.61 & 59.39 \\
        GPT-4o~\citep{gpt4o} & 81.14 & 58.14 & 26.92 & 60.61 \\
        \midrule
        \midrule
        % \multicolumn{5}{l}{\textit{Ours}} \\
        % \midrule
        Qwen2.5-32B-Instruct~\citep{qwen2} & 49.05 & 33.72 & 19.23 & 36.36 \\
        \rowcolor{blue!10} ~~~~\textit{w.}  OWL (SFT) & \better{56.60}{7.55} & \better{39.53}{5.81} & \worse{15.38}{3.85} & \better{41.21}{4.85} \\
        \rowcolor{blue!10} ~~~~\textit{w.}  OWL (SFT + DPO) & \better{67.92}{18.87} & \better{51.16}{17.44} & \better{26.92}{7.69} & \better{52.73}{16.37} \\
        \bottomrule
    \end{tabular}
    }
    % \vspace{-0.7cm}
\end{table}
% $\Delta$


% \subsubsection{Ablation Study}
\noindent \textit{\textbf{Ablation Study.}}
We evaluated the impact of trajectory filtering on model performance. 

As shown in Figure~\ref{fig:ablation}, models trained on filtered trajectories consistently outperform those trained on unfiltered data, which underscores that data quality is more critical than quantity for effective planner training. 
