17岁天才少年陈广宇，宣布离开Kimi

17‑year‑old prodigy Chen Guangyu announces departure from Kimi

I joined Moon’s Dark Side in November last year, and now I’m leaving—less than a year has passed.

◈At 16, discovered by Du Yulun and joined Kimi

The story starts a year ago.

At that time, the person who recruited Chen Guangyu to the Moon’s Dark Side was Du Yulun (@Yulun_Du), the core‑technology heavyweight of Moon Dark. Du Yulun had previously pursued a PhD at MIT and is a recognized top scholar in the generative model field.

Chen Guangyu specifically mentioned this in his farewell post:

The past year at Kimi has been the best year of my life. Thank you for everything, especially Du Yulun, who believed in and hired a 16‑year‑old me back then.

At 16, many peers were still grinding through high‑school mock exams, while he was thrown into the deepest part of the country’s top LLM “alchemy furnace,” taking on the toughest battles head‑on.

According to the timeline organized by Chen Guangyu, these are the core achievements he made at Kimi during that year:

1. Optimized the KDA kernel
   
   Worked closely with Zhang Yu (@yzhang_cs) on the high‑performance kernel of KDA (Kimi Delta Attention), testing every possible design variant.
2. Refactored attention residuals
   
   Collaborated with the renowned “Su Shen” Su Jianlin (@Jianlin_S) and Zhang Yu to conceive and refine the Attention Residuals architecture;
3. Aggressive Optimizer and Training Dynamics
   
   Waded through the deepest layers of training dynamics;
4. One of the Key Drivers Behind the Stunning K3 Release
   
   Watching the World Cup with teammates while crafting the launch case for Kimi K3.

This record isn’t that of a junior intern; it’s that of a heavily armed warrior bearing the main load at the forefront of large‑model development.

◈Invent together with Su Shen and rewrite the residual connections that have been used in Transformers for ten years

In Chen Guangyu’s report card, the most explosive entry is called Attention Residuals (AttnRes).

In March this year, the paper released by Moon’s Dark Side (arXiv:2603.15031) lists Chen Guangyu as the first author. Co‑authors include Su Jianlin, Zhang Yu, Du Yulun, Wu Yuxin, and Yang Zhiling himself.

How hardcore is this work, really?

Readers know that the reason modern Transformer models can be stacked to dozens or even hundreds of layers is entirely due to residual connections.

But the classic residual that has been used for a decade has a flaw that haunts the entire academic community:equal-weight accumulation.

The output of each layer is forcibly added to the main pathway with a 1:1 weight. As the number of layers deepens, the main data flow inflates rapidly.

At the topmost level, the newly added deep features occupy a negligible proportion in the massive historical accumulation—this is the “PreNorm dilution problem,” where deeper large models become harder to tame and prone to representation degradation.

Chen Guangyu, Su Jianlin, and the Moon Dark teamhave ended a decade-long rule of residual connections.

Their proposed Attention Residuals (AttnRes) introduce a lightweight attention scoring mechanism for each layer when revisiting all previous layers: each layer can, based on the current context, decide which earlier layer to assign more weight to, drawing on historical representations as needed.

However, if every layer performs full attention over all previous layers, the computational complexity grows quadratically with network depth.

To cut this cost, they derivedBlock Attention Residual (Block AttnRes): split a network with hundreds of layers into several interval blocks, score finely within blocks, and use anchor communication across blocks.

In the actual measurements of the Kimi 48B model:

A 16‑year‑old teenager and top‑tier architects in the industry together revamped the fundamental components of the Transformer.

◈Behind the youthful hero, large‑model development is completely resetting seniority.

Where is Chen Guangyu’s next stop?

He only left a playful salute emoji on Twitter: “More things to explore (there are more things to explore)”.

Some speculate he may return to campus to finish his studies, while others think he might be recruited by top North American labs or more ambitious startup teams.

But the deep reason his resignation tweet caused a stir today actually struck the heart of every AI practitioner.

In traditional software engineering of the past, an architect often needed ten or fifteen years of engineering experience, climbing the ladder by stacking project experience and lines of code.

But in today’s large‑model era, the traditional experience barrier is collapsing at a visibly rapid pace.

Extreme optimization of low‑level operators, mathematical derivations of attention mechanisms, dynamic tracing of thousand‑GPU distributed clusters—many of these cutting‑edge areas lack ready‑made textbooks, and the world’s top minds are exploring them from the same starting line.

Here, the pure intuition and learning curve of a 16‑year‑old can truly take on industry veterans head‑on.

Chen Guangyu’s experience is a case in point.

In early 2025, he was still learning basic concepts in machine learning by reading papers and using AI tools.

A few months later, he was already writing low‑level compute kernels and sharing his research with the open‑source community.

In less than two years, he has already been involved in research on the core architecture of large models.

We are grateful to this 16‑year‑old teenager for the elegant kernels and residual formulas he left in domestic open‑source models.

As he himself wrote, the model world is vast and boundless, and the exploration of this generation of geeky youths is only just beginning.

After all, he is only 17.

There are still many questions in the future waiting for him to explore.