3 hasil
Menampilkan 1–3 · 0,04 detik · RSS
Journal of Statistical Mechanics Theory and Experiment
Scopus
ISSN 1742-5468 · Institute of Physics · 3 artikel di indeks
-
Artikel Jurnal Teks lengkap
ConViT: improving vision transformers with soft convolutional inductive biases*
Abstract Convolutional architectures have proven to be extremely successful for vision tasks. Their hard inductive biases enable sample-efficient learning, but come at the cost of a potentially lower performance ceiling. Vision transformers rely on more flexible self-attention layers, and have…
-
Artikel Jurnal Teks lengkap
Deep double descent: where bigger models and more data hurt*
Abstract We show that a variety of modern deep learning tasks exhibit a ‘double-descent’ phenomenon where, as we increase model size, performance first gets worse and then gets better. Moreover, we show that double descent occurs not just as a function of model size, but also as a function of the…
-
Artikel Jurnal Teks lengkap
Wide neural networks of any depth evolve as linear models under gradient descent *
Abstract A longstanding goal in deep learning research has been to precisely characterize training and generalization. However, the often complex loss landscapes of neural networks (NNs) have made a theory of learning dynamics elusive. In this work, we show that for wide NNs the learning dynamics…