Biblio Discovery — Demo Discovery Service

3 hasil

Menampilkan 1–3 · 0,04 detik · RSS

Jurnal

Journal of Statistical Mechanics Theory and Experiment

Scopus

ISSN 1742-5468 · Institute of Physics · 3 artikel di indeks

  1. Artikel Jurnal Teks lengkap

    ConViT: improving vision transformers with soft convolutional inductive biases*

    Stéphane d’Ascoli; Hugo Touvron; Matthew L. Leavitt dkk.

    Journal of Statistical Mechanics Theory and Experiment · 2022 · Inggris · OpenAlex Scopus

    Abstract Convolutional architectures have proven to be extremely successful for vision tasks. Their hard inductive biases enable sample-efficient learning, but come at the cost of a potentially lower performance ceiling. Vision transformers rely on more flexible self-attention layers, and have…

  2. Artikel Jurnal Teks lengkap

    Deep double descent: where bigger models and more data hurt*

    Preetum Nakkiran; Gal Kaplun; Yamini Bansal dkk.

    Journal of Statistical Mechanics Theory and Experiment · 2021 · Inggris · OpenAlex Scopus

    Abstract We show that a variety of modern deep learning tasks exhibit a ‘double-descent’ phenomenon where, as we increase model size, performance first gets worse and then gets better. Moreover, we show that double descent occurs not just as a function of model size, but also as a function of the…

  3. Artikel Jurnal Teks lengkap

    Wide neural networks of any depth evolve as linear models under gradient descent *

    Jaehoon Lee; Lechao Xiao; Samuel S Schoenholz dkk.

    Journal of Statistical Mechanics Theory and Experiment · 2020 · Inggris · OpenAlex Scopus

    Abstract A longstanding goal in deep learning research has been to precisely characterize training and generalization. However, the often complex loss landscapes of neural networks (NNs) have made a theory of learning dynamics elusive. In this work, we show that for wide NNs the learning dynamics…