I am trying to build an RL agent to price paid-for-seating on commercial flights. I should reiterate here - I am not talking about the price of the ticket - rather, I am talking about the pricing you see if you click on the seat map to choose where on the plane you sit (exits rows, window seats, etc). The general set up is:
- After choosing their flights (for a booking of n people), a customer will view a web page with the available seat types and their prices visible.
- They select between zero and n seats from a seat map with a variety of different prices for different seats, to be added to their booking.
- The revenue from step 2 is observed as the reward.
Each 'episode' is the selling cycle of one flight. Whether the customer buys a chosen seat or not, the inventory goes down as they still have a ticket for the flight so will get a seat at departure. I would like to change prices on the fly, rather than fix a set of optimal prices throughout the selling cycle.
I have not decided on a general architecture yet. I want to take various booking, flight, and inventory information into account, so I know I will be using function approximation (most likely a neural net) to generalise over the state space.
However, I am less clear on how to set up my action space. I imagine an action would amount to a vector with a price for each different seat type (window seat, exit row, etc). If I have, for example, 8 different seat types, and 10 different price points for each, this gives me a total of 10^8 different actions, many of which will be very similar. In a sense, each action is comprised of a combination of sub-actions - the action of pricing each seat type.
Additionally, each sub-action (pricing one seat type) is somewhat dependent on the others, in the sense that the price of one seat type will likely affect the demand (and hence reward contribution) for another. For example, if you set window seats to a very cheap price, people will be less likely to spend a normal amount for the other seat types. Hence, I doubt the problem can be decomposed into a set of sub-problems.
I'm interested if there has been any research into dealing with a problem like this. Clearly any agent I build needs some way to generalise across actions to some degree, since collecting real data on millions of actions is not possible, even just for one state.
As I see it, this comes down to three questions:
- Is it possible to get an agent that can deal with a set of actions (prices) as a single decision?
- Is it possible to get this agent to understand actions in relative terms? Say for example, one set of potential prices is [10, 12, 20], for middle seats, aisle seats, and window seats. Can I get my agent to realise that there is a natural ordering there, and that the first two pricing actions are more similar to each other than to the third possible action?
- Further to this, is it possible to generalise from this set of actions - could an agent be set up to understand that the set of prices [10, 13, 20] is very similar to the first set?
I haven't been able to find any literature on this, especially relating to the second question - any help would be much appreciated!