Elektrine lite

← Feed

@unlambda@hachyderm.io

2026-04-17 01:30 UTC

@wren6991@types.pl Yeah, that's been my experience with the smaller MoE models, like the Gemma 4 26B A4B. Quite prone to looping in the CoT, especially after getting an number of turns of context. But I haven't had that issue in the larger MoE models, I think larger models provide more opportunities to escape those looping paths through the latent space.

Replies (0)

No replies.