Dust: Pretraining Transformers Without Backpropagation

Posted 1 day ago by E-Reverance
273 points

https://qlabs.sh/research/dust

24 comments

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..

Loading..