Also, why is assisted machine learning seen as an opportunity and unassisted machine learning seen as a threat?
Assisted intelligence can be thought of as a carpenter working with an intelligent hammer to build a better house. This would be a great opportunity.
Some people think that Artificial Intelligence might be like an intelligent hammer building the house on its own putting the carpenter out of a job. This would be a big threat.
Assisted Intelligence, Augmented Intelligence, or Intelligence Augmentation all refer to the same concept a technology which enhances an autonomous system that had already proven to function (humans). In the carpenter example, even a tape measure is an example of such a technology as it enhances the carpenters ability to measure distances more accurately and faster.
There has been a long history of success on the front of Intelligence Augmentation (IA) and a few terms have come up defining the different techniques such as "Extended Mind", "Distributed Cognition" which I leave up to the reader to research.
The opportunity and thus optimism behind IA comes from this long history of success and many proponents of IA hopes it will influence the trend of research to augment humans to perform better than replace them entirely.
Artificial Intelligence on the other hand attempts to build a human-like intelligence in the form of an autonomous technological system such as a computer or robot. Many fundamental problems, practical and theoretical, have been encountered from job replacement (like the carpenter), existential crisis (If no human carpenter can build a better house than a robot, why bother?), misaligned goals (how does a robot know what the best house looks like?), and many many more which I leave to the reader to research.
These problems create the threat seen in AI and has been the source many academic debates as well as many great scifi works.
Although there doesn't seem to be standardization of the terminology involving assisted learning, I've been noticing this concept, and the distinction, popping up in articles recently, surely driven by important milestones in unassisted machine learning.
My understanding is that Algorithmic Intelligence has traditionally been based on complex decision-making algorithms that are trained to some degree by the programmers. (In essence, given a bit of a head start by utilizing human knowledge in regard to the problem.)
By contrast, unassisted machine learning is an Algorithmic Intelligence that learns only by its own analysis of models and problems. No human knowledge goes into the automata, which develops intelligence entirely on its own.
A passage from Matthew Lai's Giraffe Chess paper may provide some insight:
"This report presents Giraffe, a chess engine that uses self-play to discover all its domain-specific knowledge, with minimal hand-crafted knowledge given by the programmer. Unlike previous attempts using machine learning only to perform parametertuning on hand-crafted evaluation functions, Giraffe’s learning system also performs automatic feature extraction and pattern recognition. The trained evaluation function performs comparably to the evaluation functions of state-of-the-art chess engines - all of which containing thousands of lines of carefully hand-crafted pattern recognizers, tuned over many years by both computer chess experts and human chess masters."
DeepMind, for instance, is setting automata on video games and letting the automata learn with no human guidance whatsoever, and getting very good results.
As to why unassisted machine learning is worrisome relates to the issue of human/AI value alignment and the control problem: if we don't know what an an automata is thinking, or even how smart it has become, how can homo sapiens sapiens be assured of positive outcomes for our species?