A walk through of the DeltaNet family of linear attention variants

Posted 4 hours ago by AnhTho_FR
266 points

https://blog.doubleword.ai/you-could-have-come-up-with-kimi-delta-attention

30 comments

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..