0
$\begingroup$

Say I want to train a NN that generates outputs of some sort (say, even numbers). Note that the network does not classify outputs, but, rather GENERATES the output.

I want let it run forward and generate some number, then either give it a positive reward of 1 for an even number, and a reward of -1 for an odd number, to make i output only even numbers over time.

What would be an architecture for such a NN?

I am getting caught in the part where here is actually no input, and I can't really start with a hidden layer, can I?

I am quite confused and would appreciate guidance

$\endgroup$
2
$\begingroup$

Whether a neural network has learned anything or not, it is a function that maps some input to an output. Training is the process of tweaking the weights so that the output is something that we want. Thus there is always in input of some sorts.

The problem you have presented, of generating even numbers, is much like a Generative Adversarial Network (GAN). In a GAN there are 2 networks: a Generator that tries to generate a sample from a target distribution and a Discriminator that tries to tell real samples from fake samples. The classic analogy being a criminal making counterfeit money and a copy trying to tell what is real money or not.

The generator input is usually a random number (or a matrix of random numbers). The generator then learns to transform a particular random input to a particular point in the target space.

So to answer your question, no there can't be a neural network with 0 inputs as there must always be an input of some kind. Even if the network was to generate a sequence instead of one instance, it would still need something to start with.

For your example, there would have to be some input for the network to start with. A really simple NN that could solve your problem might look like:

_input = [RandomInteger()]
neuralNetworkWeights = [2]
result = _input * neuralNetworkWeights
result is always even
$\endgroup$
  • $\begingroup$ Thanks for the quick answer! Some questions: 1. The NN you suggested relies on prior knoledge. I wanted to train the weight(s), not set it. 2. Why do i need a discriminator? I have real data (just check if the (rounded) output is even. I don't understands why GANs are needed here $\endgroup$ – Gulzar Jan 27 at 20:02
  • $\begingroup$ 1. The NN i suggested does rely on prior knowledge but its a simple example of one that works. If the network had only 1 neuron, it would eventually learn an even number, not necessarily 2 (because multiplying any integer with an even number makes an even number). If it had 10 neurons, after it was trained it would learn some set of numbers such that after multiplying with a number it would produce an even number. The example was to highlight that an input into the NN is necessary. $\endgroup$ – Jaden Travnik Jan 27 at 20:10
  • $\begingroup$ 2. Gans were added as a reference to networks that generate information. You would need to use a discriminator if you wanted to generate something from a distribution. In your "even number" example, a single network would be trained with supervised learning if you already had the real data. The difference between them is a broad subject which I think is out of scope for this comment. :p $\endgroup$ – Jaden Travnik Jan 27 at 20:16
  • $\begingroup$ In the extreme case of this question, I suppose it would even be sufficient to just have a single input node that always has a value of $1$, rather than having a random input value. It would make for a very boring generator though, which would likely learn to deterministically output the same even number every time, and no other even numbers. $\endgroup$ – Dennis Soemers Jan 27 at 20:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.