# What does a joint probability density function have to do with Stochastic Optimal Control and Reinforcement Learning?

I stumbled upon a job offer from a company that was looking for someone who was good with Reinforcement Learning (applied to finance) and something in their offer caught my eye. It goes something like this:

We want you to be able to study the price dynamic (of a stock I suppose) and its evolution in order to extract a Joint PDF that will be used in the Optimal Stochastic Control of a Loss Function (or gain)

The thing is I understand what each of these things mean and how they are used separately (from my background in Control theory & dynamical systems) and I worked with fitting Joint PDFs and Copulas before, but I don't understand how a Joint PDF would help with the "Optimal Stochastic Control of a Loss Function" ? Thanks.

• Are you familiar with Reinforcement Learning? – user9947 Mar 27 '20 at 14:37
• I mean are you aware in RL the agent is trying to learn a probablity distribution? – user9947 Mar 27 '20 at 14:51
• If i give my 2 cents (not very reliable), what they mean by joint PDF is the joint distribution of rewards, states and actions. Once you know this you can take an optimal action to maximize rewards or minimize loss in this case. Compared to a control system approach (the only one I know is Kalman Filter) you continuously need readings to make a good prediction about the internal state, there is no probability (Except the noise I guess). If i am not wrong in KF they provide a matrix where there are multiple controlling variables, which is missing in RL. Thus in RL its all about the probability – user9947 Mar 28 '20 at 12:34
• given a prior state i.e unlike KF You cannot deterministically determine the value of control matrix, and hence use of probability. (Sorry if this sounds gibberish, I just have overview of both the methods). Anyways you can join our main chatroom and I'll tag some relevant experts in this site here is the link: chat.stackexchange.com/rooms/43371/the-singularity – user9947 Mar 28 '20 at 12:37
• Also amazon.com/… this is the book you might want to refer to. Although, I'll say this it is quite daunting especially for RL beginners. The author has multiple books on this topic (a more mathematical formulation of RL), so you can check it out. – user9947 Mar 28 '20 at 12:41

## 1 Answer

Extracting a joint PDF just means that you create a model that models the behavior of several variables combined instead of in isolation.

If these variables aren't independent and your loss functions is influenced by all of them, you obviously have to learn this joint PDF to minimize your loss.

So I don't see this statement as particularly mysterious.

• The question is not "how can the joint distribution be useful in general", but "how a Joint PDF would help with the "Optimal Stochastic Control of a Loss Function"", although this answer may also answer the original question, if you are familiar with optimal stochastic control, etc. – nbro Mar 27 '20 at 16:07
• @nbro yes that's exactly it. I'm wondering about the procedure by which modeling a joint PDF separately would help with the Optimal Stochastic Control of a Loss Function. I'm just learning RL so I'm sure that's where the confusion comes from. – Metrician Mar 28 '20 at 12:05
• From where do you get the "separately"? – BlindKungFuMaster Mar 30 '20 at 13:16