3
In the past, I have used TensorFlow (1 and 2), Keras and PyTorch, so I will give an answer based on my experience. Currently, I use TF 2 and Keras (the version shipped with TF 2).
In my (but not only) opinion, TF 1 is really ugly and painful, given that it involves sessions, placeholders and, in general, you need to define the computational graph before ...
3
TensorFlow was developed by Google and is based on Theano (Python library), while Facebook developed PyTorch using the Torch library. Both frames are useful and have a great community behind them. Both provide machine learning libraries to accomplish various tasks and do the job. TensorFlow is a powerful and deep learning tool with active visualization and ...
2
Your dataset class probably have a lot of preprocessing code. You should use a dataloader. It will prefetch data from the dataset when the GPU is processing. Also, you can process all the data beforehand and save to a file. Multiple GPU cannot scale as the GPU have to get all data to one GPU to calculate the loss. The performance of 4 GPU is around 3.5x. A ...
2
I understand your question as: "How did the author select the number of neurons in their hidden layer?"
The number of neurons in the hidden layer is how you can control the complexity of the function you are trying to generate to map the inputs to an output. The more neurons in the hidden layer the more complex the function thus you can capture more ...
1
Neural networks learn badly with large input ranges. Scale your inputs to a smaller range e.g. -2 to 2, and convert to/from this range to represent your function interval consistently.
1
I thought about my input-layer. I had the 500 states one hot encoded. So 499 of every input node would be 0. And 0 is very bad in an neural network. I tried the same code with the "CardPole-v0" and it worked.
So think about your input guys
1
You cannot do this:
$\mathop{\mathbb{E}_\pi }[r(\tau )\bigtriangledown log \pi (\tau )] \\= \mathop{\mathbb{E}_\pi }[r(\tau )] \,\, \mathop{\mathbb{E}_\pi }[\bigtriangledown log \pi (\tau )]$
That is because $r(\tau )$ and $\bigtriangledown log \pi (\tau )$ are correlated by their dependence on $\tau$. In a simpler concrete example, if your expectation ...
1
The higher (or smaller) the learning rate, the higher (or, respectively, smaller) the contribution of the gradient of the objective function, with respect to the parameters of the model, to the new parameters of the model. Therefore, if you progressively increase (or decrease) the learning rate, then you will accelerate (or, respectively, slow down) the ...
1
BCELoss ( Binary Cross Entropy Loss) is used for binary classifier, which is a neural network that have a binary output, 0 or 1. It is not used for multi-output neural network like your case. For that kind of networks, you can use MSELoss or CrossEntropyLoss as your loss for the network.
For the calculation of BCE, it is shown on pytorch documentation.
...
1
Your forward function is not using the previous hidden state.
observe: you pass hidden but never use it.
def forward(self, input, hidden):
layer = self.hiddenWx1(input)
layer = self.hiddenWx2(layer)
a_next = self.tanh(layer)
z = self.z1(a_next)
z = self.z2(z)
y_next = self.softmax(z)
return y_next,a_next
1
If your interest is positional information, encode it!
This could include learning an embedding for each position and leveraging that in your model. You could also use an approach to hard-encode rather than learn it (kinda like adding sinusoids in the transformer paper Attention is All You Need
an example of a paper that encodes the 2D positional info: ...
1
There are a few things you could do to improve this NN, but are probably worth covering in different questions.
Your main problem though is that you forgot to reset the gradient after each training batch. You need to call optim.zero_grad() in order to do this, at the start of each training loop. Otherwise, using PyTorch, the gradient values keep ...
1
I tried to play with your code and found changing loss function to the cross_entropy alternate of negative log-likelihood makes the difference between 2000th epoch's loss and 9000th epoch's loss is greater about 0.2 alternate of 0.09
I also tried to change optimizer and learning rate but no loss didn't improve.
you can explore the modified code may help ...
1
PyTorch now has a C++ frontend. I haven't tried it, but I'm sure you could use that.
Another option, which is more production-tested, is using a message passing framework such as ZeroMQ to communicate requests and results between Python and C++ executables.
Only top voted, non community-wiki answers of a minimum length are eligible
