Let's say we have a neural network that was trained with a dataset $D$ to solve some task. Would it be possible to "reverse-engineer" this neural network and get a vague idea of the dataset $D$ it was trained on?
$\begingroup$
$\endgroup$
0
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
You can already do this with some neural networks, such as GANs and VAEs, which are generative models that learn a probability distribution over the inputs, so they learn how to produce e.g. images that are similar to the images they were trained with.
Now, if you're interested in whether there is a black-box method, i.e. a method that, for every possible neural network, would tell you the dataset a neural network was trained with, that seems to be a harder task and definitely an ill-posed problem, but I suspect that people working on adversarial machine learning have already attempted or will attempt to do something similar.