Can Autoresearch also cause pointless overtime? Microsoft tests a “one‑click clear conversation” feature.

Does Autoresearch also involve ineffective overtime? Microsoft tries “one-click clear conversation”

Keep the already written code and experimental results, then let the Agent clear the conversation and explore again; the outcome may be better than just running continuously.

For the same algorithm optimization problem, let Agents with identical configurations run independently from the same starting point eight times; how much can the results differ?

The best run scored 100 points, while the worst was only 68.3 points.

The underperforming Agent keeps tweaking code and running experiments, occasionally boosting its score a bit. Even with continued heavy budget consumption, it never catches up with those high-scoring runs.

After Autoresearch takes off, a single research task can keep an Agent running continuously for dozens of hours, even several days. But does doing more experiments really lead to more new ideas?

After analyzing these long‑term experiments, Microsoft Research discovered that some agents become confined to a particular class of solutions early on, and subsequent optimization revolves around it. The research team also found a way to directly visualize this confinement.

〓 Distribution of performance curves and solution space for 8 independent runs

Paper Title:

Fork-and-Flush: Escaping Idea Basins in Autoresearch Agents

Paper link:

https://arxiv.org/abs/2610.07447

Agent Stuck in Old Thinking

The agent continuously proposes solutions, runs tests, and then adjusts based on the scores. The research team likens this process to gradient descent, calling it Idea Descent.

But just looking at the scores, you still can’t tell whether it has truly changed its solution. The two programs may have similar results, yet their underlying algorithms differ; or the code may have changed a lot while the core strategy remains the same.

The researchers simply had the LLM perform more than 30,000 solution comparisons.

The research team first deduplicated and sampled the submitted programs, then had the LLM organize them into a unified functional specification, preserving algorithms, data structures, and key parameters while reducing interference from coding style.

Then give the LLM three proposals A, B, C, and have it determine which of B or C has an implementation strategy closer to A.

Based on these judgments, the research team mapped the different approaches onto a two-dimensional space. In the p171 task, the eight independently running submission approaches were concentrated in five distinct regions, and subsequent submissions also mostly remained near their respective regions.

Researchers call these areas Idea Basins, i.e., “Idea Basins”.

Across five tasks, the 2D mapping and hold‑out validation LLM similarity judgments achieved an average agreement rate of 85%, while the random baseline was only 50%. After switching to another embedding method, the main clustering phenomenon still persisted.

Even though conversation history is periodically compressed, the Agent may still keep adjusting around the algorithm selected early on.

The score sometimes still improves, but the attempts may always be of the same type of solution.

Clear the conversation, try several approaches

The research team then tried a new execution method, called Fork-and-Flush.

When reaching a scheduled node, the system copies the current workspace and launches multiple new branches. They inherit the existing code, experiment files, and historically best results, but any previously accumulated dialogue context is cleared.

After each branch explores independently for a while, the system selects the best-performing one to continue running, and repeats this process at the next scheduled node.

In this way, the previously accumulated code and experimental results remain, but the new branch doesn’t have to continue thinking along the original conversation. It can build upon existing results for optimization, and also has the opportunity to re-examine the current algorithm and try alternative solutions.

〓 Process for copying a workspace, clearing context, and preserving the optimal branch

Give the entire budget to a single agent, which may iteratively refine the existing plan over a long period; let four agents each start from scratch and then select the best, but each trajectory can only receive a quarter of the budget.

Fork-and-Flush tries to balance two strategies: it avoids starting from scratch each time, while also reducing the risk of continuously optimizing along a single solution over the long term.

The research team also tried to decide the forking timing based on whether performance stalled, but found that determining if an Agent is truly stuck is not easy.

Even when continuously tweaking around the same type of solution, the Agent may still improve the score slightly through minor optimizations, causing the stagnation timer to restart. In 25 early tests, 15 of them did not trigger a fork until the budget was exhausted.

Therefore, the main experiment was changed to branch at fixed budget nodes. Initially, start four independent branches and select the best, then branch again when about one‑third and two‑thirds of the budget have been consumed.

The entire process is scheduled by Agent Harness, without needing to modify model weights.

The research team also tried directly reminding the agent in the prompt to “think differently,” but in this set of experiments, this reminder did not significantly increase the probability of a branch being selected.

13 Long-Term Tasks Tested

In the Polyomino multi-tile arrangement task, a single mid‑process fork already caused a clear divergence.

Four branches each inherited an identical workspace. Three remained near the original solution area, while the other entered the new area on its first commit after clearing the context, and was finally selected to continue running.

In the two‑dimensional solution space, the distance of this jump reached 43 times the median displacement of adjacent submissions in a single run. This branch ultimately achieved a score of 93.9, while the median baseline score for a single run was 84.6.

All four branches cleared their conversation history, and only one actually entered the new solution area.

Clearing the context does not guarantee that the Agent will change its approach. The value of Fork-and-Flush lies in letting multiple branches try separately, then allocating subsequent budget to the more promising direction.

〓 Winning branches enter different solution areas

This study tested a total of 13 long-running tasks, covering algorithm optimization, cryptanalysis, groundwater pollution source backtracking, model compression, and kernel optimization. Among them, the baseline median runtime for a single run of the order-add task reached 107 hours.

The experiments were run uniformly via the GitHub Copilot CLI, with 11 tasks using GPT-5.6-terra and the other 2 using GPT-5.5.

Under the condition that each task uses the same model and token billing budget, the research team compared three strategies: a single run, selecting the best of four independent runs, and Fork-and-Flush.

After normalizing the scores of all tasks, the average scores of the three methods are 0.47, 0.54 and 0.78.

Fork-and-Flush improves the average scores over the first two strategies by 66.0% and 44.4%, achieving the highest average performance in 12 out of 13 tasks.

The highest scores of the single‑run baseline across all tasks, with a normalized average of 0.75, still slightly lower than the average score of Fork‑and‑Flush 0.78.

In addition to improved performance, the results are also more stable.

According to the normalized metric, the average standard deviation per run is 0.26, which drops to 0.16; among the 13 tasks, 10 show less performance variation than a single run.

For research tasks that need to run continuously for dozens of hours or even days, this also has practical value. With the same model and budget, choosing different solutions early on can lead to vastly different final results. Multi-branch exploration can reduce this randomness to some extent.

〓 Overall performance comparison of three execution strategies

To distinguish the contributions of context clearing and parallel forking, the research team also conducted ablation experiments on 11 tasks.

In terms of normalized average score, a single run is 0.45, clearing only the context reaches 0.55, only forking reaches 0.57. Using both operations together further improves the score to 0.71.

Clearing a conversation alone or adding a branch can both improve overall performance, but the improvement is more pronounced when the two actions are combined.

The effect of Fork-and-Flush comes from the entire mechanism; clearing the conversation history is just one part of it.

〓 Ablation Study of Forking and Context Clearing

However, this approach does not have an advantage on all tasks. LWE-rec is an exception; its average performance per run is actually higher than the full Fork-and-Flush.

Forking itself also consumes budget. If the new branch fails to find a better direction for a long time, that budget could have continued to be used to improve the existing solution.

When it is worth expanding the exploration and when it is more appropriate to continue refining the current solution still requires weighing.

Result Differences at the Edge of Capability

The research team further examined the relationship between task difficulty and the variance of results across multiple runs. They tasked the 4B, 9B, and 27B models of Qwen3.5 with solving MAX-3-SAT problems of varying sizes.

When the task is relatively simple, each run usually yields good results; when the task difficulty far exceeds the model’s capability, performance is generally unsatisfactory. The differences between runs tend to be greatest near the model’s capability boundary.

Compared with random guessing and a reference solver, the performance fluctuation forms an inverted U shape. The variation between runs is largest when the Agent’s score is around the midpoint between the two.

〓 The variance in execution results peaks near the capability boundary

This also provides a clue to explain the performance divergence in long-term Autoresearch.

Open research often has to tackle challenges that are near the limits of model capabilities. Agents have made some progress, but they may not consistently find high‑scoring solutions. The choices made early on can affect results for a long period thereafter.

Keeping a single research trajectory running continuously may not be enough to bridge this gap.

The research team also proposed a speculation: as model capabilities improve, today’s challenges may gradually become easy, but new research tasks will continue to approach the model’s capability boundaries.

The performance divergence between different runs may not naturally disappear as the model gets stronger.

Conclusion

The performance of Fork-and-Flush on 13 long-term tasks shows that even without changing the model, adjusting Harness’s execution strategy can still yield better results.

Of course, these experiments still rely on verifiable evaluation metrics. When facing more open real-world research tasks, how to compare the value of different research directions and how to enable effective collaboration among multiple branches remain subjects for further study.

For long-term Autoresearch, the hardest-to-detect bottleneck may be that the agent is still making some progress, yet has long been stuck in old thinking.

Read more

·