Deep Learning

📚 Leituras Recomendadas

Para Backpropagation

Capítulo Capítulo 2 — NIELSEN, M. A. Neural Networks and Deep Learning. Determination Press, 2015. neuralnetworksanddeeplearning.com ↗

Artigo RUMELHART, D. E.; HINTON, G. E.; WILLIAMS, R. J. Learning representations by back-propagating errors. Nature, v. 323, n. 6088, p. 533–536, 1986.

Artigo RUMELHART, D. E.; HINTON, G. E.; WILLIAMS, R. J. Learning internal representations by error propagation. In: Parallel Distributed Processing, vol. 1. MIT Press, 1986.

Para Gradient Descent

Capítulo Capítulos 4 e 8 — GOODFELLOW, I.; BENGIO, Y.; COURVILLE, A. Deep Learning. MIT Press, 2016. deeplearningbook.org ↗

Capítulo Capítulo 6 — PRINCE, S. J. D. Understanding Deep Learning. MIT Press, 2023. udlbook.github.io ↗

Para Funções de Custo

Capítulo Capítulo 5 — PRINCE, S. J. D. Understanding Deep Learning. MIT Press, 2023. udlbook.github.io ↗

Para Inicialização de Parâmetros

Capítulo Capítulo 7 — PRINCE, S. J. D. Understanding Deep Learning. MIT Press, 2023. udlbook.github.io ↗

Para Regularização

Capítulo Capítulo 9 — PRINCE, S. J. D. Understanding Deep Learning. MIT Press, 2023. udlbook.github.io ↗

📚 Leituras Recomendadas

Para Introdução a Redes Convolucionais

Capítulo Capítulo 10 — PRINCE, S. J. D. Understanding Deep Learning. MIT Press, 2023. udlbook.github.io ↗

Artigo LECUN, Y. et al. Backpropagation applied to handwritten zip code recognition. Neural Computation, v. 1, n. 4, p. 541–551, 1989.

Para Conexões Residuais

Capítulo Capítulo 11 — PRINCE, S. J. D. Understanding Deep Learning. MIT Press, 2023. udlbook.github.io ↗

📚 Leituras Recomendadas

Para Introdução a Redes Recorrentes

Capítulo Capítulo 10 — GOODFELLOW, I.; BENGIO, Y.; COURVILLE, A. Deep Learning. MIT Press, 2016. deeplearningbook.org ↗

Blog Post The Recurrent Net — Pablo Insente. pabloinsente.github.io ↗

Para Aplicações

Blog Post Language Modeling — VOITA, Lena. NLP Course for You. lena-voita.github.io ↗

📚 Leituras Recomendadas

Para Transformers e Self-Attention

Capítulo Capítulo 12 — PRINCE, S. J. D. Understanding Deep Learning. MIT Press, 2023. udlbook.github.io ↗

Tutorial The Annotated Transformer — Harvard NLP. nlp.seas.harvard.edu ↗

Blog Post The Illustrated Transformer — ALAMMAR, Jay. jalammar.github.io ↗

Blog Post Transformers from Scratch — RYAN, Brandon. e2eml.school ↗

Para Aplicações

Artigo VASWANI, A. et al. Attention is all you need. Advances in Neural Information Processing Systems, v. 30, 2017.

Artigo DEVLIN, J. et al. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805, 2018.

Artigo RADFORD, A. et al. Improving language understanding by generative pre-training (GPT-1). OpenAI, 2018.

Artigo RADFORD, A. et al. Language models are unsupervised multitask learners (GPT-2). OpenAI Blog, v. 1, n. 8, p. 9, 2019.

📚 Leituras Recomendadas

Para GANs

Capítulo Capítulo 15 — PRINCE, S. J. D. Understanding Deep Learning. MIT Press, 2023. udlbook.github.io ↗

Artigo GOODFELLOW, I. et al. Generative adversarial nets. Advances in Neural Information Processing Systems, v. 27, 2014.

Artigo GULRAJANI, I. et al. Improved training of Wasserstein GANs. Advances in Neural Information Processing Systems, v. 30, 2017.

Artigo KARRAS, T.; LAINE, S.; AILA, T. A style-based generator architecture for generative adversarial networks (StyleGAN). In: CVPR, 2019, p. 4401–4410.

Para Normalizing Flows

Capítulo Capítulo 16 — PRINCE, S. J. D. Understanding Deep Learning. MIT Press, 2023. udlbook.github.io ↗

Artigo REZENDE, D.; MOHAMED, S. Variational inference with normalizing flows. In: ICML. PMLR, 2015, p. 1530–1538.

Artigo KINGMA, D. P.; DHARIWAL, P. Glow: Generative flow with invertible 1×1 convolutions. Advances in Neural Information Processing Systems, v. 31, 2018.

Para VAE

Capítulo Capítulo 17 — PRINCE, S. J. D. Understanding Deep Learning. MIT Press, 2023. udlbook.github.io ↗

Artigo BANK, D.; KOENIGSTEIN, N.; GIRYES, R. Autoencoders. Machine Learning for Data Science Handbook, p. 353–374, 2023.

Artigo KINGMA, D. P. et al. An introduction to variational autoencoders. Foundations and Trends in Machine Learning, v. 12, n. 4, p. 307–392, 2019.

Artigo KINGMA, D. P. Auto-encoding variational Bayes. arXiv:1312.6114, 2013.

Para Diffusion Models

Capítulo Capítulo 18 — PRINCE, S. J. D. Understanding Deep Learning. MIT Press, 2023. udlbook.github.io ↗

Artigo HO, J.; JAIN, A.; ABBEEL, P. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, v. 33, p. 6840–6851, 2020.

Artigo HO, J.; SALIMANS, T. Classifier-free diffusion guidance. arXiv:2207.12598, 2022.