What does AI software look like, and how is it different from other software?

Question

What does AI software look like? What is the major difference between AI software and other software?

Answer 1

Code in AI is not in principle different from any other computer code. After all, you encode algorithms in a way that computers can process them. Having said that, there are a few points where your typical "AI Code" might be different:

• A lot of (especially early) AI code was more research based and exploratory, so certain programming languages were favoured that were not mainstream for, say, business applications. For example, much work in early AI has been coded in Lisp, and probably not much in Fortran or Cobol, which were more suited to engineering or business. Special languages were developed to make it easy to program with symbols and logic (eg Prolog).

• The emphasis was more on algorithms than clever/complex programming. If you look at the source code for ELIZA (there are multiple implementations in many different languages), it's really very simple.

• Before the advent of neural networks and (statistical) machine learning, most AI programming was symbolic, so there hasn't been much emphasis on numerical computing. This changed as probabilities and fuzziness were increasingly used, but even if using general purpose languages there would be fewer numerical calculations.

• Self-modifying code is inherently complex; while eg Lisp made no difference between code and data (at least not in the same way as eg C or Pascal), this would just complicate development without much gain. Perhaps in the early days this was necessary when computers had precious little memory and power and you had to work around those constraints. But these days I don't think anybody would use such techniques anymore.

• As modern programming languages evolved, Lisp and Prolog (which were the dominant AI languages until probably 20 to 30 years ago) have been slowly replaced by eg Python; probably because it is easier to find programmers comfortable in an imperative paradigm rather than a functional one. In general, interpreted languages would be preferred over compiled ones due to speed of development, unless performance is important.

The move to deep learning has of course shifted this a lot. Now the core processing is all numeric, so you would want languages that are better with calculations than symbol handling. Interpreted languages would now mainly make up the 'glue' code to interface between compiled modules, and be used for data pre-processing. So current AI code is probably not really that different from code used in scientific computing these days.

There is of course still a difference between R&D and production code. You might explore a subject using an interpreted language, and then re-code your algorithm for production in a compiled language to gain better performance. This depends on how established the area is; there will for example be ready-made libraries available for neural networks or genetic algorithms which are well-established algorithms (where performance matters).

In conclusion: I don't think AI code is any more complex than any other code. Of course, that's not very exciting to portray in a film, so artistic licence is used to make it more interesting. I guess self-modifying code also enables the machines to develop their own conscience and take over the world, which is even more gripping as a story element. However, given that a lot of behaviour is nowadays in the (training/model/configuration) data rather than the algorithm, this might even be more straight forward to modify.

Note: this is a fairly simplified summary based on my own experience of working in AI; other people's views might vary, without either being 'wrong'.

Update 2021: I now work at a company that extracts business information/events from news data on a large scale using NLP methods. And we're using Lisp... so it's still in active, commercial use in AI.

Answer 2

Oliver Mason's answer is quite good, but I think it can be expanded upon a bit.

I think there are extra factors that could be popularly interpreted as making AI code difficult to read (as compared to other code):

1. AI code actually is more complex than most code that is written. When we work in AI, we often lose sight of this, but most code ever written does one of two things: turn data in one standard format into data in another standard format; display something to a user. Both of those are conceptually easy to understand. Neither of them is likely to require knowledge of mathematics. This is very unlike most code written in AI, where understanding what was written, and why, requires extensive knowledge beyond the knowledge needed to read and write computer programs. So, reading AI code requires more knowledge of mathematics or of complicated AI-focused Algorithms.
2. The "programs written by AIs" are really our models in the modern context. Our algorithms "program" a template model to make it work for a specific application. This is especially true if you think of it in the senses in which programming is also used in "linear programming", "quadratic programming", and even "dynamic programming". Our models really are hard to understand. Often even their creators cannot explain or characterize the model's behavior on specific inputs without running the model. The reason for this is that our models do not represent simple enough concepts that humans can easily understand or simplify them.
3. Self-modifying code is rare, but does exist within AI. However, as with other AI-generated models, AI-generated code tends to be comparatively difficult for humans to interpret, because (unlike most human-generated code), it is not written with the intention that humans are going to try to read and understand it. There actually are some efforts to generate code that conforms to human styles, but usually the code that is generated does not work well.

Answer 3

This may be a much simpler explanation than you're looking for, but in Machine Learning Zero to Hero, Google engineer Laurence Moroney summarized it in a way that I thought was brilliant. Paraphrasing from a presentation slide:

In traditional programming, you input rules and data and the program outputs answers. In machine learning, you input data and answers and the program outputs rules.

There's an algebra-like symmetry to this. And the program doesn't even know what it's coming up with rules for. It just randomly evolves the rules until the data produces the correct answers. You can then take those rules, apply them to different data, and hopefully get correct answers.

Answer 4

AI has been redefined recently to machine learning.

All programming except machine learning (and we'll come back to this) is embodying human knowledge in terms a computer can follow.

EG A text editor has user interface rules, user expectations, a contract with the OS that it has to follow. A programmer puts it all together. This applies to text editors, expert medical systems, banking software, accounting software (and the programmer needs to know accounting to program it).

Machine learning is training software with data and outputs allowing it to determine the link between them. No human knowledge. Nor can it explain what it is doing.

Of course they actually work far better when human knowledge surrounds them as part of their data. A AI that routes incoming invoices etc works better when told where things should actually go (accounts payable).