4

In language theory, there are generally several admitted levels that can be studied in relation with one another or independently. The semantic level is the one dealing with the meaning of the text ("semantic" comes from the greek and means "to signify"). The semantic level is therefore generally independent from the syntax and even the language used to ...


4

So if I understood correctly: You have data from 2 sensors in time: Ar flow and BackGas Flow (SCCM, what is that?) You have that data for multiple products. 1 - Since it is relatively low dimensional, you may try using raw data with K-Means or Self Organizing Maps. 2 - If you searching for anomalies in time, you might try using feature engineering with ...


3

Usually, you need to ensure that your convolutions are causal, meaning that there is no information leakage from the future into the past. You could start by looking at this paper, which compares Temporal Convolutional Networks (TCN) with vanilla RNNs models.


3

ANNs & RNNs can be used to create some great models in many different domains, including time-series forecasting. However, across all of these domains, they suffer from the problem of hyper-parameter optimization. Because neural networks are so flexible, it is not clear, at the outset, which arrangement of neurons will be most effective to solve a given ...


2

Given the (usual) higher architectural complexity of ML models compared to more classical forecasting models, ML models might also require more data, otherwise they might just overfit the training dataset. Furthermore, online learning (or training) of a neural network using stochastic gradient descent (that is, one example at a time) might also be ...


2

The second implementation looks more correct and inline with how Bidirectional is defined. Specifically, bidirectionality doen't change the forward/backward logic of either direction, and just merges (concat/sum/...) the outputs of forward/backward at a matching timestep t. You can check how Keras implements it here. There are distinct self.forward_layer ...


2

Things like this a really hot topic in research right now, and it's very difficult to get high accuracy on a chaotic system like the stock market. That being said, I would probably recommend preprocessing your data rather than having your primary neural network decide what to accept and what not to. For example, in your specific case, you could model a ...


2

Yes, definitely. In the simplest example, predicting an output value for a time series is classification. You take in the previous time steps and classify what is the most likely next value. You could do this with a RNN (Recurrent Neural Network) for example. If the activation functions are all linear, the nerual network is just a glorified linear regression....


2

You are right CNN based models can outperform RNN. You can take a look at this paper where they compared different RNN models with TCN (temporal convolutional networks) on different sequence modeling tasks. Even though there are no big differences in terms of results there are some nice properties that CNN based models offers such as: parallelism, stable ...


2

there's a lot to un-pack in this question. Why do they only pick 500 rows? my guess: in order to keep the example running quickly. tsfresh usually takes a while to calculate its features. note that when they evaluated their model, they took the last 500 samples. What's the point of re-arranging the rows/columns? answer: the data frame format that ...


2

This problem has been formally termed as Delayed MDP (Katsikopoulos & Engelbrecht, 2003)[1] - the actions generated are not instantly applied to the environment and/or the captured observations are not immediately seen by the agent, as expected in an MDP. The delay can either be: a) A constant delay - Constant Delayed MDP (CDMDP) (Learning and planning ...


2

You can use CNN for time-series data. The Convolutional Recurrent Neural Network (RCNN) is one of the examples. Convolutional layers basically extract features from images. It is not related to time-series data processing. Some CNNs (such as in ResNet, Highway Networks, and DenseNet) use some recurrent concepts to improve their prediction, but they all are ...


2

Edit I just noticed that the model you are referring to is built very differently than the transformer from Attention is All You Need since it only uses one half of the architecture. Thus my answer below is not be complete. I thus have to add the following: (The final two paragraphs still apply as they are, though) The Keras model is quite weird, and while ...


2

As you can see in Fig. 2 of the WaveNet paper the receptive field is 5, but the input size is larger (16). The receptive field defines what a single output neuron can see (see arrows in Fig. 2). The receptive field could also be greater than the input, e.g. if you want to use or you only have the last 12 time steps and use the following structure (WaveNet ...


1

If you have sold only once or very few items you will need some prior input (domain knowledge). One term for search is intermittent time series. Here is a stored search. When you have many time series, related, and interest in both totals and single series, that is called hierarchical forecasting. One expert is here (the author of that blog was the founder ...


1

Most RL algorithms assume a discretization of time (although RL can also be applied to continuous-time problems [1]), i.e., in theory, it doesn't really matter what the actual time between consecutive time steps is, but, in practice, you may have delays in the rewards or observations, so you cannot perform e.g. the TD updates immediately. One natural ...


1

There could be multiple possible ways to extract the features. One would be to use RNNs for a temporal relationship as the input data is time-series.


1

I'm currently working with Temporal Convolution Networks (TCNs) for making predictions with time series data (link to article here: https://medium.com/@raushan2807/temporal-convolutional-networks-bfea16e6d7d2). These types of networks, like other types of convolutional networks for time series, use a dilated convolution operation, which, unlike the standard ...


1

If I understand your problem correctly, you can test on just about any environment, and just omit parts of the observations to ensure your RNN is learning. For example, you can test on cartpole, ignoring the velocity and angular velocity states. This way the MDP isn't actually Markovian and you'll need the RNN to learn.


1

Your idea is a good one. Another idea is to upsample or aggregate your data. For example, average by week if you generally have a couple of missing days in every week. A similar question on Stack Exchange: https://stats.stackexchange.com/questions/374935/how-to-deal-with-really-sparse-time-series-data-for-a-binary-classification-task


1

An interesting model I encountered in a course is Facebook Prophet. Prophet takes into account trends, seasonality, and holidays for its predictions. As you can probably guess, this is a model that fits Facebook's needs very well. I'll give a brief introduction then provide a link where you can read more. Prophet fits a couple of functions of time ...


1

Yes, LSTM are ideal for this. For even stronger representational capacity, make your LSTM's multi-layered. Using 1-dimensional convolutions in a CNN is a common way to exctract information from time series too, so there's no harm in trying. Typically, you'll test many models out and take the one that has best validation performance.


1

Just for clarification: your description (1 sample per minute) does not match the example data (far fewer data points which is understandable, but also two data points in one minute which contradicts the initial assertion.) If your actual measurements are like that you should first work on the sampling process to get reliable data. For creating predictions, ...


1

Yes, you can fit any time series (with or without external variables) using HMM, but there are some constraints: It should follow the Markov property. There is some variance that other models are not able to capture (in other words, the system is partially observable). Adding to point 1, for HMM, it should hold true, but the way Baum Welch algorithm ...


1

Problem is in the output layer and you are using categorical_crossentropy for a loss function. Quoting Keras documentation: Note: when using the categorical_crossentropy loss, your targets should be in categorical format (e.g. if you have 10 classes, the target for each sample should be a 10-dimensional vector that is all-zeros except for a 1 at the ...


1

You need to have access to the 696th hour (or successive hours), otherwise, you cannot test your model. An alternative would be, for example, to train your model on the first 693 hours, validate it on the 694th hour, and test it on the 695th hour.


1

Definitely! but at that point it would be training a transformer-encoder (gpt2's architecture) and not GPT2 because GPT2 is defined by the weights / training procedure / data it was trained and not the architecture, and I don't think it would transfer properly to time series.


1

I have come across the same issue but in language. Where each input was a sentence, hence of different lengths. The easier solution is to just find the longest sequence, extract its length, and 0 pad all other values to be able to get all of them to the same size, and then use any recurrent neural network architecture (Since you're dealing with a time ...


1

(this response should be a comment but I don't have yet the reputation to comment). If I'm understanding your problem correctly you have a variable number of input which have an order and only one output ? It look like the kind of task where you could use recurrent neural network (the most common ones are the LSTM and GRU). If you use a recurrent neural ...


Only top voted, non community-wiki answers of a minimum length are eligible