-1
$\begingroup$

I am beginning to learn deep learning. I recently spoke with an expert in the field. He suggested that I start with pytorch because of these reasons:

  1. Keras abstracts the stuff a lot that we will not actually understand what is going on behind. This would be the case especially if we are experimenting with some models.
  2. Tensorflow is good, but, pytorch is much more flexible. You can do much more with it and thus it is more powerful.
  3. When you code stuff in pytorch, you can actually understand what it is, but, tensorflow can make it confusing.

I understood some things that he said. But, since I have not had experience with these frameworks, I wanted to know which to start with if I want to go in depth and understand the background concepts.

$\endgroup$
  • 2
    $\begingroup$ If you want to understand concepts in depth you should learn mathematics behind it, not the frameworks. If you understand the mathematical concepts you can implement it in any framework, it's not very relevant. Good place to start is to read deep learning book. Also, I wouldn't say that you can do much more in Pytorch and that it is more powerful than Tensorflow, maybe it's easier to use though $\endgroup$ – Brale_ Jul 2 at 11:32
  • $\begingroup$ Pytorch. It's much more clear and easy to understand then TF $\endgroup$ – mirror2image Jul 2 at 12:21
  • $\begingroup$ No experience in pytorch, but tensorflow 2.0 is much more user friendly than older versions of tensorflow. You should have no problem beginning with tensorflow. I can't say if one is more powerful than the other but tensorflow has, generally speaking, a larger community (twice as many contributors, 4 times as many stars in github) and is supported by google which in terms of companies is maybe the largest contributors to open-source software worldwide. $\endgroup$ – Djib2011 Jul 2 at 21:35
  • $\begingroup$ @Brale_ Thank you for the suggestion. I am learning the mathematics behind it. I just wanted to know which framework would be better to implement the mathematics (and, in some sense, which framework is more straightforward). Thanks for the resources! $\endgroup$ – Akhilesh Ravi Jul 3 at 6:24
0
$\begingroup$

Tensorflow 2 is dynamic, similar to PyTorch, because by default it works in eager execution. So the worries your friend had were probably for 1.x. Also, I don't think any framework will affect your understanding. I get that some abstractions can hide things from you, but that shouldn't be what deters you from checking out the theory behind it on the side.

Let's look at some of the pros/cons of all 4.

TensorFlow 1.x

Pros

  • Heavy amounts of documentation/tutorials
  • A lot of ready-made projects and I would say most repositories I've had to deal with were in TF 1.x (others experiences may differ)
  • A lot of nice convenient ways to do parallelism

Cons

  • static when not in eager execution (and generally you're not), so harder to prototype quickly

PyTorch

Pros

  • Heavy amounts of documentation/tutorials
  • The second most amount of ready-made projects and I would say I've had to deal with were in PyTorch (others experiences may differ)
  • Dynamic (convenient for prototyping)
  • a lot of nice convenient ways to do parallelism

Cons

  • community is much smaller than TF 1.x (you'll notice a difference when checking out published work, asking questions, etc) (note that the PyTorch community is still huge!)

Keras

Pros

  • Has multiple backends
  • Can use abstract frameworks like 1.x in a way that prototyping feels easy

Cons

  • high-level abstraction can degrade control, but in most cases through custom layers, and maybe clever tricks you can sometimes get it back
  • not the best usage of backends capabilities for parallelism

TensorFlow 2.x

Pros

  • now is dynamic by default for quick prototyping
  • can still process your model in graph mode when needed but done internally
  • integrates tf.keras to give you the high-level abstraction feel but still give you power and control necessary to create different models/training environments
  • parallelism is still easy to do
  • has the support of the TF team

Cons

  • its new and in beta. As much as I like TF 2, it's new and so the community hasn't grabbed to it (i know for sure my company still will only use 1.x for the time being)
  • is not backward compatible with 1.x (they have a conversion script, but it's still not ideal)

If I was forced to recommend one right now, it would be TF 2 but that's because I've taken a liking to it and its super convenient in a multitude of manners.

Note that some of these were subjective. so take that into account. I hope this helped

$\endgroup$
  • $\begingroup$ Hi. If you want me to remove my downvote, first format your answer more properly (using markdown when appropriate). Your statement "Also i dont think any framework will affect your understanding." is also very arguable. $\endgroup$ – nbro Jul 3 at 13:55
  • $\begingroup$ @nbro im notoriously bad at formatting, i tried helping it with code-formatting on the pros/cons but no idea what else to do-- feel free to keep the down vote there... And out of those 4, i guess its arguable but thats my experience. The abstraction of the different level languages doesn't change that in ALL of them you can write top of the line models with little understanding by just duplicating the model description. The abstractions just change how much boiler plate code is necessary + what level of control you have. $\endgroup$ – mshlis Jul 3 at 14:20
  • $\begingroup$ @nbro if you want to understand whats going on, the framework isnt the factor, your ability to go into the literature and backtrack the piece you dont get until you do understand is. Your ability to change the model/training in a way and understand the outcome. $\endgroup$ – mshlis Jul 3 at 14:21
  • $\begingroup$ TensorFlow 2.x use eager execution? How? I have had to follow a tremendously steep learning curve for TF, compared to that PyTorch was easy and on point but the trade-off Probably was speed of execution. Also PyTorch lacks many activations I think although you can implement themselves I guess. $\endgroup$ – DuttaA Jul 3 at 22:39
  • $\begingroup$ @mshlis thanks for summing all that up. I have a good idea now. Thanks for the comments as well! $\endgroup$ – Akhilesh Ravi Jul 4 at 6:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.