What are some books/papers that deal with fundamental and philosophical issues of ML and relate it to the global discourse of AIs?

Question

In my experience, most of the time, when people talk about AI nowadays they mostly mean machine learning. Despite this, ML is usually seen as a mere technique to build high-performance software.

I rarely see people discuss the foundational questions of it, such as: from which "philosophy" of AI did ML emerge? Why is ML compelling in AI research, if not by its performance? What are the fundamental differences between statistical/probabilistic AI and logical AI? For reference, this hasn't even been mentioned in my master-level course on machine learning. Even myself I used to have a distaste for ML because I thought it was just mindless data-crunching.

But, lately, I've been reading through "Probability Theory: The Logic Of Science" and I'm starting to appreciate the theoretical side of ML, for instance, how Bayesian probability can be seen as a model of plausible reasoning in humans, and how probability theory extends logic (motivating, maybe, why probabilistic AIs were the next logical [no pun intended] step after logical AI). I would like now to delve deeper into the topic.

What are some books/papers that deal with fundamental and philosophical issues of ML and relate it to the global discourse of AIs?

Answer 1

I'll recommend two sources:

1. The venerable Russell & Norvig book, which is a common text in AI courses. Russell & Norvig end each chapter with a summary of the history of the developments of the techniques they have just discussed. These sections are often skipped by novice readers, but are almost exactly what you are looking for. The ones in the back half of the book should, together, give you a good sense of the order in which developments occurred, why techniques were developed, and what advances techniques enabled.

2. R&N does a good job of covering what happened, but not always why. For that, you want a philosophy of AI book. I recommend Mind Design II as a starting place. This book is a chronologically organized collection of papers and academic essays by the big thinkers in philosophy of mind and in AI research. Often, the papers are responses to one another. With some side reading about the history of each author, you can begin to get a good sense of the big philosophical movements in the field over the last 70 years, and why things have changed.

If you don't want to read the book, I can give you a summary (spoilers ahead!):

1. 1920's: behaviorists, and others, propose that Mind = Brain, and specifically focus on intelligent behavior.
2. 1950: Turing proposes that a computer could be programmed to exhibit intelligent behavior.
3. 1960's: Cognativists in AI, Psych, and Linguistics argue that behavior is not enough. Minds think, and thought takes the form of reasoned algorithms. The lynchpin of their argument is domains like language understanding, which they claim cannot be modeled without logic. Their work produces search algorithms, and early rule-based planning and language systems.
4. 1970's Searle argues that computers can't think because of Phenomenology. Most AI folk ignore him, and go on working, but Philosophy & Psych take greater notice.
5. 1970's Dryfus (and others) argue compelling that logic and reason cannot explain human thought, by dissecting the rule based systems of the day through the Frame Problem. AI researchers take some notice.
6. The Connectionists, especially Hinton (AI) & Churchland (Philosophy) propose the first post-Cognativist theories of mind. These focus on the idea that Mind is in the Connections (specifically, the firing patterns) of neurons, not in the brain itself. This view of minds spurs (re)-development of Neural Networks. In the 1980's, this is mostly ignored.
7. During the 1990's, Connectionists, and others working on probabilistic methods in AI demonstrate systems for language that are substantially better than rule based methods. Cognativists begin to decline, because the domains they claimed needed reasoned algorithms are actually better handled by statistical algorithms. Connectionist views gain support in AI and elsewhere.
8. Today, with rising computational power, statistical techniques come to the forefront of many AI domains. Simultainiously, the Churchlands, Rodney Brooks, and others propose further Post-Cognativist schools of thought based around dynamical systems theory and embodied cognition, which were somewhat influential in robotics. Cognativism continues to enjoy some support within the AI, Psych, and Linguistics communities, but this is much diminished. Some hybrid systems use a mix of statistical and rule-based techniques.