Tracking control is a technique which is similar to PID control and has the aim to implement a robust feedback loop for generating action-signals. In contrast to a line following robot, tracking control is oriented in a time-action space. On the x-axis the timecode is presented, for example 0 seconds, 0.5 seconds and 1 seconds, while on the y-axis the value of the signal is plotted. The value can be the position of a robot, the angle of an inverted pendulum or the x-position of an object. Even in simple problems like the inverted pendulum, the number of parameters on the y-axis is greater than 1. The spline of all parameters of the time are equal to a movement pattern.

The problem is how to summarize the different values to a single signal. This is needed for determine the similarity in a “Learning from demonstration” experiment. The human-operator is doing an action, and the aim of the robot is to reproduce the movement pattern. I've searched a bit the topic in Google Scholar and found a paper: A humanoid robot standing up through learning from demonstration using a multimodal reward function but I'm a bit unsure, because there is so much math and it is also dedicated to ZMP biped walking. What I'm searching is more a general idea of how to compress different splines into one reward function.

Description of the bug

Tracking a single spline which is plotted in a diagram is easy. The difference between the current value and the desired value is measured and the feedback-controller reduces the difference. If the number of splines is 2 this concept fails. For example, spline #1 represents the angle and spline #2 the velocity over time. The current state has 2 variables and the desired state has 2 variables. How can I program a steering-controller for a multi-spline?

  • $\begingroup$ It is dificult to me to understand which is the question $\endgroup$ – pasaba por aqui Feb 10 '18 at 15:09

From the sounds of it, your problem can be approached in 2 generic ways.

1) You define one reward function as a weighted sum of the different reward functions you care about. For instance, if you want to maximize the reward functions of EatingHealthy, EH(state, action), and GoodMoneyManagement, GMM(state, action), you could define your reward function R to be:

R(state, action) = W1*EH(state,action) + W2*GMM(state,action)

Where W1 and W2 are some real values numbers.

In your case, they might as well both be .5 as there may not be a difference of importance between the different splines.

2) Multiple learners In this scenario, you instead implement 2 agents that share the same state input but not the same weights nor outputs. Refer to this answer for further details on how to approach this. This method allows you to add additional learning agents if your problem changes and more splines are needed to be learned.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.