New answers tagged


Can't see that this has been mentioned yet - there are ways to generate text non-sequentially using a non-autoregressive transformer, where you produce the entire response to the context at once. This typically produces worse accuracy scores because there are interdependencies within the text being produced - a model translating "thank you" could ...


Yes, RNN can work on the functions you have mentioned. In fact, neural networks can approximate anything (Universal Approximation Theorem). This question also reminds me of Neural Turing Machine. But, it would be a complete waste to use RNNs or NNs for such a task.


I cannot answer your question but I am stuck in a similar rabbit hole so hopefully these references can help you. The loss function you are describing would be 0-1 loss. However, 0 would be the if our output matches and 1 would be if it does not. This function is not smooth and not convex. Thus we often replace it with a surrogate loss function such as log ...


One LSTM layer should be enough unless you have lots of data. The same thing goes for the number of nodes in the layer. Start small first so 5 to 10 nodes and increment it until the performance is reasonable. Once you have a model working you can apply regularization if you think it will improve performance by reducing overfitting of the training data. You ...

Top 50 recent answers are included