Is there a way to teach reinforcement learning in applications other than games?
The only examples I can find on the Internet are of game agents. I understand that VNC's control the input to the games via the reinforcement network. Is it possible to set this up with say a CAD software?
You will see a lot of game examples in reinforcement learning literature, because game environments can often be coded efficiently, and run fast on a single computer that can then contain the environment and the agent. For classic games, such as backgammon, checkers, chess, go, then there are human experts that we can compare results with. Certain games or simplified game-like environments are commonly used to compare different approaches, much like MNIST handwritten digits are used for comparing supervised learning approaches.
Is there a way to teach reinforcement learning in applications other than games?
Yes. Informally you could apply reinforcement learning approaches whenever you can frame a problem as an agent acting within an environment where it can be informed of the state and a goal-influencing reward value. More formally, reinforcement learning theory is based upon solutions to Markov Decision Processes, so if you can fit your problem description to a MDP then the various techniques used in RL - such as Q-learning, SARSA, REINFORCE - can be applied. This fit to theory does not need to be perfect for the resulting system to work, for instance you can often treat unknown or imperfectly observed state as effectively random to the agent, and consider this part of a stochastic environment.
Here are some examples of possible uses for reinforcement learning outside of recreational games:
Control logic for motorised robot, such as learning to flip pancakes and other examples. Here the environment measurements are made by physical sensors on the robot. The rewards are given for completing a goal, but may also be adjusted for smoothness, economic use of energy etc. The agent chooses low-level actions such as motor torque or relay position. In theory there can be nested agents where higher level ones choose the goals for the lower-level ones - e.g. the robot might decide at a high level between doing one of three tasks that require moving to different locations, and at a lower level might be decisions on how to control motors to move the robot to its chosen goal.
Self-driving cars. Although a lot of focus on sensor interpretation - seeing road markings, pedestrians etc, a control system is required in order to select accelerator, brake and steering.
Automated financial trading. Perhaps a game to some, there are clear real-world consequences. The reward signal is simple enough though, and RL can be adjusted to prefer long or short term gains.
is it possible to set this up with say a CAD software?
In theory yes, but I do not know what might be available to do this in practice. Also you need one or more goals in mind that you code into the agent (as reward values that it can observe) before giving it a virtual mouse and setting a task to draw something. Computer games come with a reward scheme built in as their scoring system, and provide frequent feedback, so an agent can gain knowledge of good vs bad decisions quickly. You would need to replace this scoring component with something that represents your goals for the CAD-based system.
CAD does not have anything suitable built-in, although CAD tools with simulations, such as various physics engines or finite element analysis, could allow you to score designs based on a simulated physical measure. Other possibilities include analysis of strain, non-wasteful use of material, whatever metrics the CAD/CAM system can provide for a partial or completed design. The tricky part is constraining a design to its goal or purpose and either arranging for that to be rewarded, or building the constraints into the environment; giving an RL agent full unconstrained control of CAD process and rewarding on lowest strain will likely result in something very uninteresting such as a small cube.
One of the cool examples of reinforcement learning is an autonomous flying helicopter. I had a chance to learn some of the stuff done by Andrew Ng and others recently. Here is the research article paper. There are other similar papers too. You can google them if you want to learn more.
You can also see it in action in in this youtube video.
Here is another completely different application in finance apparently.
Reinforcement learning (and, in particular, bandit) algorithms have been and can be used to solve problems other than games, such as
In general, any problem that can be modelled as the maximization of some notion of reward, where you need to interact with some environment (with some states) by taking some actions, can, in principle, be solved by reinforcement learning. Take a look at this pre-print paper (2019) for other applications.
However, note that there are several obstacles that prevent RL algorithms from being widely adopted to solve real-world problems, starting from poor sample complexity (i.e. they require many samples to reach a good performance) or the partial inability to evaluate their performance online without affecting the users.
