Why does C++ seem less widely used than Python in AI?

Question

I just want to know why do machine learning engineers and AI programmers use languages like Python to perform AI tasks and not C++, even though C++ is technically a more powerful language than Python.

Answer 1

You don't need a powerful language for programming AI. Most of the developers are using libraries like Keras, Torch, Caffe, Watson, TensorFlow, etc. Those low level libraries are highly optimized and handle all the tough work. They are built with high-performance languages, like C, C++. Python is just there for high level task like describing the neural network layers, load data, launch the processing, and display results.

Using C++ for high level task instead of Python would give barely any performance improvement, but it would be harder for non-developers as it requires to care for memory management. Also, several AI people may not have a very solid programming or computer science background.

Another similar example would be game development, where the engine is coded in C/C++, and, often, all the game logic scripted in a higher-level language.

Answer 2

C++ is actually one of the most popular languages used in the AI/ML space. Python may be more popular in general, but as others have noted, it's actually quite common to have hybrid systems where the CPU intensive number-crunching is done in C++ and Python is used for higher level functions.

Just to illustrate:

http://mloss.org/software/language/c__/

http://mloss.org/software/language/python/

Answer 3

It depends how flexible it needs to be: if you have a fully-fledged system ready for production, which is not going to need much adjusting, then C++ (or even C) might be fine. You need to put a lot of time into building the software, but then it should run pretty fast.

However, if you're still experimenting with settings and parameters, and maybe need to adjust the architecture, then C++ will be clumsy to work with. You need a language like Python which makes it easier to change things. Changing the code is easier, as you can generally code faster in languages like Python. The price you pay is that the software does usually not perform as well.

You need to decide how that trade-off works best for you. It is usually better to spend less time on coding, and not worry too much about longer run-time. If you take a day less to get your code done, that's a lot of time the C-coded version needs to catch up. Most of the time it's just not worth it.

A common approach seems to be hybrid systems, where core libraries are implemented in C/C++, as they don't need much changing, and the front-end/glue/interfaces are in Python, as there you need flexibility and speed is not that critical.

This is not an issue specific to AI, by the way, but a general question of interpreted vs compiled languages. With AI a lot of systems are still focused on research rather than application, and that is where speed of development trumps speed of execution.

Answer 4

You claim that

C++ is technically a more powerful language than python.

But that claim is wrong (or does not mean much). Remember that a programming language is a specification (often some document written in English). For example, n3337 is a late draft of the C++ specification. I don't like Python, but it does seems as powerful than C++ (even if C++ implementations are generally faster than Python ones): what a good Python programmer can code well in Python, another good C++ programmer can code well in C++ and vice versa.

Theoretically, both C++ and Python are Turing-complete (on purpose) programming languages.

And Python is as expressive as C++ is. I cannot name a programming language feature that Python has but not C++ (except those related to reflection; see also this answer and be aware of dlopen - see my manydl.c program -, of LLVM, of libgccjit, of libbacktrace, and consider some meta-programming approach with them, à la Bismon or like J.Pitrat's blog advocates it).

Maybe you think of a programming language as the software implementing it. Then Python is as expressive as C++ is (and seems easier to learn, but that is an illusion; see http://norvig.com/21-days.html for more about that illusion). Python and C++ have a quite similar semantics, even if their syntax is very different. Their type system is very different.

Observe that sadly, many recent major machine learning libraries (such as TensorFlow or Gudhi, both mostly coded in C++) are in practice easier to use in Python than in C++. But you can use TensorFlow or Gudhi from C++ code since TensorFlow and Gudhi are mostly coded in C++ and both provide and document a C++ API (not just a Python one).

C++ enables multi-threaded programming, but the usual Python implementation has its GIL, is bytecoded, so is significantly slower than C++ (which is usually compiled by optimizing compilers such as GCC or Clang; however you could find C++ interpreters, e.g. Cling). Some experimental implementations of Python are JIT-compiled and without GIL. But these are not mature: I recommend investing a million euros to increase their TRL.

Observe also that C++ is much more difficult to learn than Python. Even with a dozen years of C++ programming experience, I cannot claim to really know most of C++.

Sadly, most recent books teaching AI software engineering (e.g. this one or that one) use Python (not C++) for their examples. I actually want more recent AI books using C++ !

BTW, I program open source software (like this one, or the obsolete GCC MELT) using AI techniques, but they don't use Python. My approach to AI applications is to start designing some DSL in them.

Some AI approaches involve metaprogramming, e.g. generating some (or most, or even all) the code of a system by itself. J.Pitrat (he passed away in October 2019) pioneered this approach. See his blog, his CAIA system, read his Artificial Beings, the conscience of a couscious machine book (ISBN 978-1848211018) and the RefPerSys project (whose ambition is to generate most -and hopefully all- of its C++ code).

On operating systems such as Linux you could in practice generate C++ (or C) code at runtime and compile it (using GCC) into a plugin, then later dlopen(3) that generated plugin, and retrieve function pointers by their name using dlsym(3). See the manydl.c example (on a powerful desktop in 2020, you would be able to generate and load half a million of plugins, if you run that example several days). With dladdr(3) and Ian Taylor's libbacktrace, you can also inspect some of the call stack.

AFAIK major corporations such as Google use C++ internally for most of their AI-related code. Look also into MILEPOST GCC or the H2020 Decoder project for an application of machine learning techniques to compilers. See also HIPEAC.

Of course, you can code AI software in Haskell, in Common Lisp (e.g. with SBCL), or in Ocaml. Many machine learning frameworks can be called from them. Number crunching libraries could use OpenCL.