Elektrine lite

← Feed

@wren6991@types.pl

2026-04-17 01:26 UTC

@unlambda it seems less prone than Qwen3.5-35B-A3B to going into an infinite recursive tailspin in the CoT

Replies (1)

  • @unlambda@hachyderm.io 2026-04-17 01:30

    @wren6991@types.pl Yeah, that's been my experience with the smaller MoE models, like the Gemma 4 26B A4B. Quite prone to looping in the CoT, especially after getting an number of turns of context. But I haven't had that issue in the larger MoE models, I think larger models provide more opportunities to escape those looping paths through the latent space.

    Open ##5321103