How can I formulate a usecase with an additional constraint as a reinforment learning problem?

Question

I am new to the field of reinforcement learning, and I feel a recent use case of mine is highly relevant, but I don't know how to forumate it as a typical reinforcement learning problem.

Let's say I have an actor model. The input to the actor is a state obtained from environment, and the output of the actor is a discrete choice of 3 actions (each action is associated with an action cost). My target is to main a certain of score from enviroment but try our best to achieve the smallest accumulated action cost.

For example, the actions can be A={"use gun": 1000}, B={"use knife": 100}, C={"escape": 10} which basically means if you use gun to fight an enemy, it is supposed to be effective but the cost is 1000, however, if you choose action escape, the cost is low, but it won't harm the enemy.

What I hope to achieve is to : (1) At least win the game (i.e. open the front door of the castle), and (2) target the lowest total action cost. For example, if I can do "use_knife" twice and "escape" twice to open the door, I should not do "use_gun" to open the door.

I am a beginner of RL and I feel normal RL exemplary topic is about to select actions to achieve the highest score, very straightforward. So I was wondering if my above description can be achieved by some standard RL algorithms. Can someone provide some guidance on this? Thanks in advance. I will really appreciate it if you can point me some exemplary repo or papers : )

Answer 1

You could use the cost of each action as a penalty on the reward at each step. So for instance if your agent uses the gun, you could add a reward of -0.1 and using the knife say a reward of -0.01. The positive reward obtained could be 1.0 if the agent is successful at a given step. You will have to play around with these values to see which works best. This is a type of reward shaping.