2
$\begingroup$

I am searching for an academic (i.e. with maths formulae) textbook which covers (at least) the following:

  • GAN
  • LSTM and transformers (e.g. seq2seq)
  • Attention mechanism

The closest match I got is Deep Learning (2016, MIT Press) but it only deals with part of the above subjects.

Improve this question
$\endgroup$
0
2
$\begingroup$

There are a few more books that were published after 2016 that cover some of the topics you are interested in. I've not read any of them, so I don't really know whether they are good or not, but I try to summarise if they cover some of the topics you may be interested in.

  • Deep Learning with Python (2017), by Francois Chollet (author of the initial Keras library), which covers GANs in section 8.5 (p. 305), but it does not seem to cover transformers and attention mechanisms, although it covers other intermediate/advanced topics (not sure to which extent), such as text generation with LSTMs, DeepDream, Neural Style Transfer and VAEs

  • Grokking deep learning (2019), by Andrew Trask, which seems to cover some intermediate/advanced topics (such as LSTMs and related tasks), but no transformers or GANs (unless I missed them); you can find the accompanying code here

  • Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play, by David Foster, which covers many variants of GANs, VAEs and other stuff

The first transformer was published in 2017, so I guess there may not yet be a book that extensively covers it and other related models, such as the GPT models (if you're interested in CV, check this blog post, although it seems to list books that cover mostly traditional CV techniques). The attention mechanisms are older and can probably be found in textbooks that cover machine translation topics (such as seq2seq models with LSTMs), such as this one.

Improve this answer
$\endgroup$
2
$\begingroup$

I recommend Introduction to Deep Learning by Eugene Charniak ISBN 978-0-262-03951-2 (MIT 2018). It mentions GAN & LSTM & Attention (all three occurs in the index).

But read also Pitrat's last book: Artificial Beings: The Conscience of a Conscious Machine - it does cover machine learning (but not in the "deep learning" sense) but was published before 2016.

And see also RefPerSys. If you speak French, also see this and more generally the AFIA organization.

Improve this answer
$\endgroup$
11
  • $\begingroup$ Why the downvote? Charniak book mention deep learning and is after 2016! $\endgroup$ – Basile Starynkevitch Dec 21 '20 at 20:28
  • $\begingroup$ And it seems that Blincer lives (like I do) in France $\endgroup$ – Basile Starynkevitch Dec 21 '20 at 20:42
  • $\begingroup$ The question is about books that cover GANs and transformers. I downvoted because you're promoting other books here, which have little to do with the question. Remove those books and I will remove my downvote. Also, you should explain if the DL book you're suggesting contains chapters (or at least some sections) dedicated to these topics. It should contain, otherwise, you should not be suggesting it. So, you should specify which chapters are dedicated to GANs and transformers. $\endgroup$ – nbro♦ Dec 21 '20 at 20:48
  • 2
    $\begingroup$ Then the question should be improved $\endgroup$ – Basile Starynkevitch Dec 21 '20 at 21:06
  • 1
    $\begingroup$ Introduction to Deep Learning is a pretty good match thank you! The only issue is that it covers attention in only a specific case. I don't see how the rest of the answer is relevant. $\endgroup$ – Blincer Dec 21 '20 at 22:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.