Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
I've been using PyTorch to do research for a while and it seems to be quite easy to implement new things with. Also, it is easy to learn and I didn't have any problem with following other researchers code so far.
However, I wonder whether TensorFlow has any advantage over PyTorch. The only advantage I know is, it's slightly faster than PyTorch.
In general, does TensorFlow have any concrete advantages over PyTorch apart from performance, in particular for research purposes?
In the past, I have used TensorFlow (1 and 2), Keras and PyTorch, so I will give an answer based on my experience. Currently, I use TF 2 and Keras (the version shipped with TF 2).
In my (but not only) opinion, TF 1 is really ugly and painful, given that it involves sessions, placeholders and, in general, you need to define the computational graph before executing anything (even the simplest programs). With TF 2, you don't need sessions and placeholders anymore, and this is a very big improvement because you don't need to think that you first need to define a computational graph and then feed it with the input. Essentially, in TF 2, you can write TF code and you can almost be sure that your TF 2 code will look like any other Python code.
TF and Keras have some bugs, but I think that also PyTorch must have some bugs (even though I don't remember having encountered them).
TF comes with TensorBoard. You can also easily use TensorBoard from Keras. About one year ago, I had written a blog post about visualization tools for PyTorch. At the time, the tools didn't seem as developed as TensorBoard, but there were already a few options.
Debugging in TF 2 isn't still perfect, but I would say that it has improved with respect to TF 1 (mainly because there is less boilerplate code). To debug in TF 2.0, I mainly use tf.print (which behaves similarly to Python's print function), but there may be more advanced tools (e.g. I think there's a TF debugging plugin for PyCharm), even though I've never used them. TensorBoard could also be used as a debugging tool (for example, to understand how your models are learning, etc).
The simplicity of PyTorch is comparable to that of Keras. TF now comes with an integrated version of Keras, which is optimized for TF, so TF code can look as simple as Keras code.
In general, my impression is that, if you need to, TF provides more flexibility, but it's been a while since I used PyTorch. Furthermore, with TF 2.0, the code looks simpler, more consistent and the framework is better organized. On the other hand, PyTorch may still be easier to learn (because it doesn't have the past of TF, i.e. there are still many TF code that uses sessions, etc.).
Both TF and PyTorch are supported by big companies (respectively, Google and Facebook), but TF is more mature and there are functionalities/libraries that exist for TF that still do not exist for PyTorch (AFAIK). I doubt that there is a functionality for PyTorch that doesn't also exist for TF or Keras.
