田渊栋新公司首个模型亮相！AI全程参与、不到4个月从零造

Machine Heart Editorial Team

Since May this yearofficially announced, the new company Recursive_SI co‑founded by Tian Yuandong has finally delivered its first model result!

Flint-40B-A2B is a fast, locally‑hardware‑optimized agent programming model, built entirely with Recursive’s self‑developed automation research system Eureka Machine.

As a mixture-of-experts (MoE) model, Flint has a total of 40 billion parameters, but only 2.5 billion parameters are activated for each generated token. With the joint effort of a small staff and AI agents, the model was built from scratch in less than four months.

On agent programming benchmarks such as SWE‑Bench Pro v2, Terminal‑Bench 2.1, and SWE‑Bench Verified, Flint outperforms other MoE models with fewer than 50 billion parameters. Its programming ability is comparable to dense models with 25 billion to 40 billion parameters, while running more than five times faster.

The inference engine has also been specially optimized. With short contexts, Flint generates over 125 tokens per second on a MacBook Pro equipped with an M4 Pro chip, and exceeds 150 tokens per second on an NVIDIA RTX Spark with 64 GB of memory. Compared to similar models, speed can be up to twice as fast.

Flint on the MacBook Pro M4 Pro takes the balance between speed and capability of local models to a new level. 

“Bro…Recursive just released a programming model that can run locally on a MacBook Pro, with a generation speed of up to 125 tokens per second!”

Developing Flint is also a practical test of the large‑scale automated AI research and development system, Eureka Machine. This system helped Recursive complete almost every stage of the Flint model building process, including data processing, training strategies, infrastructure, and inference optimization.

Currently, Flint’s model weights are publicly available on Hugging Face under an Apache 2.0 license. After downloading the model, you can run it locally, keeping your data on your own device, and fine‑tune it to suit specific application needs.

• Hugging Face URL: https://huggingface.co/RecursiveSI/flint

• rec project URL: https://github.com/recursive-oss/rec

Flint also supports multiple tools and frameworks, including llama.cpp, Ollama, Unsloth, LM Studio, vLLM, and SGLang. Additionally, the open‑source lightweight inference engine rec is specifically optimized for Flint on M4 Pro and M5 Pro/Max chips, and provides a simple configuration process to help users get started quickly.

A developer using a MacBook Pro M5 Max with 64 GB of RAM measured that Recursive Flint generates at a speed of 114–132 tokens per second.

Image source: @WangCankun

Another developer managed to run this 40‑billion‑parameter programming model on an RTX 3060 with only 12 GB of VRAM, supporting a 256K context window and achieving a generation speed of about 46 tokens per second.

Local agent programming capability reaches SOTA

In agent programming benchmarks, Flint is the best-performing MoE model with a total parameter count below 50 billion. Its performance rivals top dense models of comparable scale, while its runtime is noticeably faster.

Specifically, Flint achieves the highest scores on IFBench and IFEval, and leads the listed other sparse MoE models on Terminal‑Bench 2.1 and SWE‑bench Pro v2. It also remains highly competitive on SWE‑bench Verified and LiveCodeBench v6. However, some dense models still attain higher scores on programming benchmarks.

Flint is suitable as a daily programming model, running tasks locally without sending data to a remote server. When handling complex tasks, it is recommended to break the work into smaller sub‑tasks and provide clear instructions. Flint already has quite good capabilities, but compared with the large proprietary programming models most people are familiar with, you may need to give it more guidance.

Flint supports a context window of up to 512k tokens. For programming and other tasks, Recursive recommends keeping the context length within 256k tokens for better results.

To translate Flint’s computational efficiency into a faster programming experience, Recursive has optimized the entire inference pipeline, improving everything from the representation of model weights to the low‑level kernels that execute computation on local hardware.

Local AI devices run faster

Flint brings strong programming capabilities to the user’s own hardware. You can use it in an offline environment, keeping data on your own computer, and you don’t have to pay for API calls.

With an optimized inference engine, Flint can run up to twice as fast as comparable models on Apple Silicon and NVIDIA RTX Spark. Compared to dense models with 25 billion to 40 billion parameters that have similar programming benchmark scores, Flint is more than five times faster.

The Automated Research and Development Behind Flint

Currently, Eureka Machine can automatically complete an entire research workflow: generating ideas, writing implementations, running tests, and then deciding what to try next based on the test results. It can explore multiple technical paths simultaneously, leverage results from existing experiments to keep progressing, and combine the best-performing solutions from different experiments.

Recursive has previously introduced how this system works and its performance on the initial test tasks.

More details: https://recursive.com/updates/first-steps-toward-automated-ai-research

The entire development of Flint leveraged the Eureka Machine. From data, optimizers, and training schemes to infrastructure and inference performance, this system was involved in the R&D. The improvements it uncovered helped a small team build Flint from scratch and, in less than four months, achieve leading performance on the target benchmark.

Flint’s local inference speed also demonstrates how these improvements stack across different stages of model development.

Model Architecture

The first step was to find a model architecture better suited to local hardware, aiming for each token to activate fewer parameters and thus increase generation speed.

Eureka Machine started from a baseline model with a total of 35 billion parameters, activating 3 billion parameters per token, and explored various expert configurations and network widths. After multiple rounds of exploration, the system predicts that among the configurations it tested, the finally selected Flint architecture better balances inference speed and programming capability: a total of 40 billion parameters, with only 2.5 billion parameters activated per token.

Quantization

After model training is completed, Eureka Machine continues to search for suitable quantization schemes, aiming to reduce Flint’s memory footprint while preserving the model’s capabilities as much as possible.

Unlike common practice, the system also quantizes the output layer to 4 bit. Since Flint activates only 2.5 billion parameters per token, the output layer constitutes a large proportion of the weights read when generating each token. Quantizing this layer further reduces inference overhead, but it may also affect model accuracy.

To compensate for precision loss, the system adds an offline distillation step: first sample answers from the automatically discovered core sample set, then let the quantized model train using these answers, with the 16‑bit model serving as the teacher model.

Compared with the previous quantization‑aware training baseline, this method reduces the number of training steps by almost half. The resulting NVFP4 model’s scores on all benchmarks differ from the 16‑bit model by no more than 0.7 points.

Inference Kernel

Next, Eureka Machine further optimizes the execution of quantized models for Apple Silicon and NVIDIA RTX Spark.

It develops dedicated kernels for low‑bit projection computation, MoE operations, attention computation, and prompt processing, and adjusts them according to model architecture and hardware characteristics.

On the M4 and M5 chips, the system also developed a large fused kernel based on Metal that merges the entire forward computation of the decoding stage into a single kernel execution, reducing extra overhead during computation.

Flint’s current inference speed is the result of continuous improvements by Eureka Machine across various stages. Recursive also adopts the same approach in data processing, model training, and infrastructure, combining effective solutions discovered in different experiments to further enhance the final model’s performance.

Next Steps

Recursive aims to create a superintelligent system capable of recursive self‑improvement, automating the knowledge discovery process. Starting with model development, we hope to establish a research workflow that continuously improves with each generation of models.

Flint is the starting point of this effort. A small team built this model from scratch in less than four months. The Eureka Machine is responsible for generating ideas, attempting implementations, and testing the effectiveness of these approaches at every stage of model development.

Recursive’s hypothesis is that a more capable model can make the Eureka Machine a better researcher, helping to develop the next generation of models faster and achieve greater capability at the same cost.

Going forward, Recursive will continue to develop more powerful local models and the infrastructure needed to run them, extending this work to a broader range of AI research and development.

Blog address: https://recursive.com/updates/introducing-recursive-flint

© THE END 

For reprints, please contact this public account for permission

Submit articles or request coverage: liyazhou@jiqizhixin.com