How can a neural network work with continuous time?

Question

I have an ANN model that receives an input and produces an output. The output is an action that interacts with the environment and changes the input accordingly. The network has a desired environment state which, in any turn, decides the desired response and trains the network on that basis.

Currently, the network works in discrete time. How can I make this network work in a continuous manner? Can you provide some resources and links if there is any past or current research on continuous AI?

To be more concrete, the system starts with the current environment state, for example, [1 1 1], then produces an output. In current system, the next step takes the final state of the system as input, for example, [1 2 2], but we know that such a thing doesn't happen in physical world and the system goes from [1 1 1] to, for example, [1 1 2], and then to [1 2 2], and that middle step is something that a discrete-time AI can't figure out.

The very case that I'm working on is the simulation for an autopilot cart in which the model is incapable to take subtle things like "the maximum speed that you can turn the steering wheel" into consideration. I don't want to add these complexities to the model since if the model is perfect, then the result is deterministic and there is no need for AI. I want the AI to be able to make a decision in each step based on the current state of the system in continuous time.

Answer 1

By the way you have explained things above, it seems more like a problem with your code and not the something to do with the environment. The term discrete and continuous is used to define, how the outside environment is acting, rather than how your code is taking its steps. These are some lines from the book, Artificial Intelligence: A Modern Approach:

The discrete/ continuous distinction applies to the state of the environment, to the way time is handled, and to the percepts and actions of the agent. For example, the chess environment has a finite number of distinct states (excluding the clock). Chess also has a discrete set of percepts and actions. Taxi driving is a continuous state and continuous-time problem: the speed and location of the taxi and of the other vehicles sweep through a range of continuous values and do so smoothly over time. Taxi-driving actions are also continuous (steering angles, etc.). Input from digital cameras is discrete, strictly speaking, but is typically treated as representing continuously varying intensities and locations.

So, continuous or discrete is not something that should be talked about as a problem of the code. It is basically, what an environment is. Your concern with the device is regarding the code. I will suggest that you upload the code on git and ask people to improve it.

I hope this helps!

Answer 2

I don't think the transformation of [1 1 1] into [1 2 2] needs a middle step. Actuators can work simultaneously and they do not have to wait for each other to complete their job. I must even note that, if your next output is [1 2 2], then performing [1 1 2] is so wrong in the case of following a trajectory (if it's your case). So, I guess the middle step in your example is [1 1.5 1.5]. Think of a line segmentation. When you segment a line, you still have your slope, and you do not create "steps". So what you are following in your neural network-based controller is exactly the pattern you need. Your problem is probably the closed-loop frequency of your controller. Better NN performance leads to quicker response and then better actuation.