Questions tagged [sequence-modelling]

The tag has no usage guidance.

31 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5
votes
2answers
680 views

What evaluation metric are used for sequence-to-sequence prediction problems?

I am solving many sequence-to-sequence prediction problems using RNN/LSTM. What type of evaluation metrics can be used for sequence prediction problems? One metric is the mean squared error (MSE) ...
3
votes
0answers
40 views

Can sequence-to-sequence models be used to convert source code from one programming language to another?

Sequence-to-sequence models have achieved good performance in natural language translation. Could these models also be applied to convert source code written in one programming language to source code ...
2
votes
0answers
64 views

NLP Bible verse division problem: Whats the best model/method?

I'm working on a project compiling various versions of the Bible into a dataset. For the most part versions separate verses discreetly. In some versions, however, verses are combined. Instead of verse ...
2
votes
0answers
207 views

Why is the transformer for time series forecasting faster than RNN?

I've been reading different papers which implements the Transformer for time series forecasting. Most of the them are claiming that the training time is significantly faster then using a normal RNN. ...
2
votes
0answers
34 views

Best approach for online Machine Translation with few hundred of samples?

I want to implement a model that improves itself with the passage of time. My main task is to build a machine translator (from English to Urdu).. The problem I am facing is that I have very little ...
1
vote
1answer
42 views

Continuous sequence data with Transformer model

What is the right way to input continuous, temporal(time series) data into Transformer network. Assume we're using the basic TransformerBlock here. Since data is continuous with no tokens, Token ...
1
vote
0answers
29 views

Why and when transformers are better than CNN's in sequence modeling tasks?

Transformers have made a revolution in the domain of NLP and gave rise to a rapid boost of neural networks in a variety of language modelling problems, TTS and, recently, achieved competitive accuracy ...
1
vote
0answers
16 views

Seq2Seq Models not used for NLP - input to the Decoder?

I am looking into Seq2Seq models but using it to make multi-step predictions of factory data and I am getting a little confused with the inputs to the Decoder model. Correct me if I am wrong, but the ...
1
vote
0answers
20 views

Are there any successful applications of transformers of small size (<10k weights)?

In the problems of NLP and sequence modeling Transformer architectures based on self-attention mechanism https://arxiv.org/abs/1706.03762 have achieved impressive results and now are the first choices ...
1
vote
0answers
22 views

Building a resume recommendation for a job post?

There are few challenges I am facing when building a resume recommendation for a particular job positing. Let's say we convert the resume into a vector on n-dimensions and job description also as an n-...
1
vote
0answers
22 views

What's the difference between RNNs and Feed Forward Neural Networks if a fixed size vector can preserve sequential information?

I was watching a Youtube video in which the problem of trying to predict the last word in a sentence was posed. The sentence was "I took my cat for a" and the last word was "walk"....
1
vote
0answers
27 views

length independent sequence classification methods

I am looking to do sequence classification using deep learning. The length of my sequences can vary from a few hundred to several tens of thousands of characters. I was wondering what is a good ...
1
vote
0answers
47 views

Model for supervised sequence classification task

The Problem I am currently working on a sequence classification problem I try to solve with machine learning. The target variable is the current state of a system. This target variable is following a ...
1
vote
0answers
23 views

What's the best method to predict/generate signal from one sensor (source) to signal from another another (target)?

I was wondering what is the best method out there to find relationship between two 1D signals so that I can predict/generate one (source) from the other (target). For example, let's say that in ...
1
vote
0answers
37 views

How to pad sequences during training for an encoder decoder model

I've got an encoder-decoder model for character level English language spelling correction, it is pretty basic stuff with a two LSTM encoder and another LSTM decoder. However, up until now, I have ...
1
vote
0answers
24 views

Generation of realistic real-valued sequences using Wasserstein GAN fails

My goal is to generate artificial sequences of real-valued data (e.g. time series) with GANs. Starting simple I tried to generate realistic sine-waves using a Wasserstein GAN. But even on this simple ...
1
vote
0answers
37 views

Sequence-to-Sequence models without specifying the start and end of sentences

Is there a seq-to-seq model which does not require to know the start and end of a sentence? I need to model a system which gets a long sequence of words and creates a long sequence of tokens as long ...
1
vote
0answers
18 views

Make an LSTM model for each class separately

I have a dataset of some activities. The dataset contains the status of different sensors and the label of activity. T trained a model in Keras with the following architecture which models the ...
1
vote
0answers
24 views

Literature on Sequence Regresssion

I have some rated time-sequential data and I would like to test if an ANN can learn a correlation between my measurements and ratings. I suspect I could just try a CNN where 1 Dimension is time or an ...
1
vote
0answers
22 views

Language Model from missing data

I want to learn how a set of operations (my vocabulary) are composed in a dataset of algorithms (corpus). The algorithms are a sequence of higher level operations which have varying low-level ...
1
vote
1answer
88 views

Can HMM, MRF, or CRF be used to classify the state of a single observation, not the entire observation sequence?

I learn that the Viterbi algorithm used for Hidden Markov Model (HMM) can classify a sequence of hidden states from the corresponding observations; Markov Random Field (MRF) and Conditional Random ...
1
vote
1answer
56 views

Why feeding the correct output as input during training of seq2seq models?

So, I've read about seq2seq for time-series and it seemed really promising, but when I went to implement it, all the tutorial I've found use the correct output as input to the decoder phase during ...
1
vote
2answers
18 views

Is seq2seq the best model when input/output sequences have fixed length?

I understand that seq2seq models are perfectly suitable when the input and/or the output have variable lengths. However, if we know exactly the input/output sequence lengths of the neural network. Is ...
0
votes
0answers
16 views

Attention vs Global Depthwise convolution

The main advantages of the self-attention mechanism are: Ability to capture long-range dependencies Ease to parallelize on GPU or TPU However, I wonder, why the same goals cannot be achieved by ...
0
votes
0answers
14 views

Are there any well-known ways to fuzzy-cluster (variable length) sequences of trajectories?

I have this issue where I need to create 'soft' clusters for different trajectories. The data is sequences of integers where each integer means a specific point; so I have sequences like $s=(1,47,9)$ ...
0
votes
0answers
23 views

What is the difference between model setup, model configuration, and model customization?

In the context of research papers related to deep learning models, the authors usually mention these terms in the experiment section when they are talking about the model: configuration, setup. For ...
0
votes
0answers
12 views

MBART and better Domain Specific Translations Using Masks?

I'm implementing https://huggingface.co/facebook/mbart-large-50-many-to-many-mmt for translations as it has shown promising results but I wanted to see if there was a way to translate specific parts ...
0
votes
1answer
26 views

Normalization of possibly not fully representative data

I am trying to train a classification RNN model on a sequence of table medical data, but I stuck with the normalization problem. I realized that I cannot simply use MinMaxScaler, because of 3 problems:...
0
votes
0answers
27 views

Representing variable-length sequences

I want to train a model over a variable-length sequential data (e.g. the temperature at different times of day) where the output depends on what the temperature is at time ...
0
votes
0answers
99 views

How are temporal links made between following sequences in RNN?

Say I use an RNN, whatever is the cell's type, to perform time series classification. It can thus be seen as sequence classification. The time series is split into random, equal size, overlapping ...
0
votes
0answers
34 views

Network design to learn multiple sequences of multiple categories

For learning a single sequence, LSTM only should suffice. However, my situation is different here. I have a list of sequences to learn: The sale volumes of 12 months, these are the sequences And ...