3
$\begingroup$

I stumbled upon a job offer from a company that was looking for someone who was good with Reinforcement Learning (applied to finance) and something in their offer caught my eye. It goes something like this:

We want you to be able to study the price dynamic (of a stock I suppose) and its evolution in order to extract a Joint PDF that will be used in the Optimal Stochastic Control of a Loss Function (or gain)

The thing is I understand what each of these things mean and how they are used separately (from my background in Control theory & dynamical systems) and I worked with fitting Joint PDFs and Copulas before, but I don't understand how a Joint PDF would help with the "Optimal Stochastic Control of a Loss Function" ? Thanks.

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ Are you familiar with Reinforcement Learning? $\endgroup$
    – user9947
    Mar 27, 2020 at 14:37
  • 1
    $\begingroup$ I mean are you aware in RL the agent is trying to learn a probablity distribution? $\endgroup$
    – user9947
    Mar 27, 2020 at 14:51
  • 1
    $\begingroup$ If i give my 2 cents (not very reliable), what they mean by joint PDF is the joint distribution of rewards, states and actions. Once you know this you can take an optimal action to maximize rewards or minimize loss in this case. Compared to a control system approach (the only one I know is Kalman Filter) you continuously need readings to make a good prediction about the internal state, there is no probability (Except the noise I guess). If i am not wrong in KF they provide a matrix where there are multiple controlling variables, which is missing in RL. Thus in RL its all about the probability $\endgroup$
    – user9947
    Mar 28, 2020 at 12:34
  • 1
    $\begingroup$ given a prior state i.e unlike KF You cannot deterministically determine the value of control matrix, and hence use of probability. (Sorry if this sounds gibberish, I just have overview of both the methods). Anyways you can join our main chatroom and I'll tag some relevant experts in this site here is the link: chat.stackexchange.com/rooms/43371/the-singularity $\endgroup$
    – user9947
    Mar 28, 2020 at 12:37
  • 1
    $\begingroup$ Also amazon.com/… this is the book you might want to refer to. Although, I'll say this it is quite daunting especially for RL beginners. The author has multiple books on this topic (a more mathematical formulation of RL), so you can check it out. $\endgroup$
    – user9947
    Mar 28, 2020 at 12:41

1 Answer 1

Reset to default
1
$\begingroup$

Extracting a joint PDF just means that you create a model that models the behavior of several variables combined instead of in isolation.

If these variables aren't independent and your loss functions is influenced by all of them, you obviously have to learn this joint PDF to minimize your loss.

So I don't see this statement as particularly mysterious.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ The question is not "how can the joint distribution be useful in general", but "how a Joint PDF would help with the "Optimal Stochastic Control of a Loss Function"", although this answer may also answer the original question, if you are familiar with optimal stochastic control, etc. $\endgroup$
    – nbro
    Mar 27, 2020 at 16:07
  • $\begingroup$ @nbro yes that's exactly it. I'm wondering about the procedure by which modeling a joint PDF separately would help with the Optimal Stochastic Control of a Loss Function. I'm just learning RL so I'm sure that's where the confusion comes from. $\endgroup$
    – Metrician
    Mar 28, 2020 at 12:05
  • $\begingroup$ From where do you get the "separately"? $\endgroup$
    – BlindKungFuMaster
    Mar 30, 2020 at 13:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .