New answers tagged reinforcement-learning
0
In general, NO.
You don't get the "true" state value function. TD-learning approximates the true value function. It can be a very close, or even exact approximation in simple cases, but, in general, it is just an approximation.
Depending on the difficulty of the problem, a non-constant $\alpha$ value can help the policy approximate the true value ...
0
I assume your agent also has to choose which locations to visit next. If so, then there are two rough designs that crossed my mind.
You can use separate agents, one for choosing to inspect or not, and one to choose which adjacent cell to visit. Sum all of the log likelihoods of both agents' actions for the loss.
One particular benefit of this design is that ...
2
Is it possible to tell the Reinforcement Learning agent some rules directly without any constraints?
You could just tweak your reward function to include this restrictions.
In the most simple case, you could reward your agent -1 if $x(t) > 0$ and $y(t) \neq 0$.
The scale of your negative reward depends on your general reward scaling of course.
0
Value Iteration Vs Policy Iteration
Below is the list of differences & similarities between value iteration and policy iteration
Differences
Value Iteration
Policy Iteration
Execution starts with a
random value function
random policy
Algorithm
simpler
complex
Computation costs
more expensive
cheaper
Execution time
Slower
Faster
No of Iterations to ...
0
On creating custom environments:
... always normalize your observation space when you can, i.e., when you know the boundaries
(From stable-baselines)
You could normalize them as part of the environment's state space or before passing them as input to the policy. Depending on the the agent's algorithm implementation, what works for you may vary.
(See this ...
0
The accuracy level is already very high, but how to motivate the agent to find the target as quickly as possible?
You already are, in two different ways:
A penalty (negative reward) for each time step taken.
A positive reward for completing a task, plus discounting.
Both of these choices are sufficient that action values will be maximised by taking the ...
1
The reward is given only at the end of the episode (or when there is timeout there is no reward)
This is a common case. E.g. winning a board game, or reaching a goal state.
How could we learn the value function?
All RL algorithms are designed to cope with this scenario. Actor-Critic is not an exception. Value-based algorithms (including the critic in ...
0
It's an important feature, and you drop it at the risk of the agent failing to learn successfully.
The difference between the TD target without the terminal flag
$$G_t = R_{t+1} + \gamma \text{max}_{a'} Q(S_{t+1}, a')$$
and with the terminal flag applied to $S_{t+1} = S_T$
$$G_t = R_{t+1}$$
is important whenever $Q(S_{T}, a')$ might be evaluated as non-zero. ...
0
You might want to start on googling the "Irregular Cutting Stock Problem". I think your problem formulation is similar to Irregular Cutting Stock Problem. Some cool papers are up in the results such as this heuristic method which is tested on real-world based problem instances.
By browsing the existing heuristic/metaheuristic methods, you may get ...
2
They have a few similarities, but they are quite different. Let me first give you a general description of both approaches/algorithms, so that you start to get a sense of their differences and similarities.
Description
Gradient descent (GD) can be applied to solving any optimization problem where your loss (aka cost or objective) function is differentiable ...
1
Should the constraints reward also be normalized for all 10 constraints?
You should choose a "natural" balance between rewards where possible.
If you have many separate goals to take account of, ideally you should convert them all into some comparable metric that is meangful to the success of the agent. Such as a financial gain/loss, or energy ...
1
I guess it depends on what the goal is. If the goal is a general reward function, this formulation as an MPOMDP could make sense. One way to think about this, is as a way of modeling a general (centralized) POMDP with factored actions and observation spaces.
However, it seems that what you are describing might be an active perception problem, where the goal ...
1
There is this paper talking specifically about the safety of ML/DL
Algorithm but in industrial application. Since the algorithm is purely learning function you have to define the safety in terms of the application you are using it for example diagnosis or surgery.
1
I think you should first start with definition of software safety in health domain. For example, you should start with Therac-25 accident.
Then look at the current scientific articles and standards about software safety in medical domain. Then think about how your algorithm will be tested.
You are thinking Deep RL algorithms as a blackbox but they are ...
4
Ultimately, a policy must be such that is is possible for an agent to execute it.
If the policy depends on the state, the implicit assumption is that the agent has knowledge of the state and can therefore choose its actions accordingly. This is the common case of a MDP as an underlying framework for RL.
If the state is not known to the agent, it may instead ...
Top 50 recent answers are included