# Tag Info

4

In language theory, there are generally several admitted levels that can be studied in relation with one another or independently. The semantic level is the one dealing with the meaning of the text ("semantic" comes from the greek and means "to signify"). The semantic level is therefore generally independent from the syntax and even the language used to ...

4

So if I understood correctly: You have data from 2 sensors in time: Ar flow and BackGas Flow (SCCM, what is that?) You have that data for multiple products. 1 - Since it is relatively low dimensional, you may try using raw data with K-Means or Self Organizing Maps. 2 - If you searching for anomalies in time, you might try using feature engineering with ...

3

Usually, you need to ensure that your convolutions are causal, meaning that there is no information leakage from the future into the past. You could start by looking at this paper, which compares Temporal Convolutional Networks (TCN) with vanilla RNNs models.

3

ANNs & RNNs can be used to create some great models in many different domains, including time-series forecasting. However, across all of these domains, they suffer from the problem of hyper-parameter optimization. Because neural networks are so flexible, it is not clear, at the outset, which arrangement of neurons will be most effective to solve a given ...

2

Given the (usual) higher architectural complexity of ML models compared to more classical forecasting models, ML models might also require more data, otherwise they might just overfit the training dataset. Furthermore, online learning (or training) of a neural network using stochastic gradient descent (that is, one example at a time) might also be ...

2

The second implementation looks more correct and inline with how Bidirectional is defined. Specifically, bidirectionality doen't change the forward/backward logic of either direction, and just merges (concat/sum/...) the outputs of forward/backward at a matching timestep t. You can check how Keras implements it here. There are distinct self.forward_layer ...

2

I do not know how you will apply your data to the techniques I'll give you some brief overview of techniques used in time series prediction: Extended Kalman Filtering: This is a kind of control system approach and is generally used to control trajectory of missiles. Here is a question (based on an EKF paper) in our stack on this topic. You can check the ...

2

Things like this a really hot topic in research right now, and it's very difficult to get high accuracy on a chaotic system like the stock market. That being said, I would probably recommend preprocessing your data rather than having your primary neural network decide what to accept and what not to. For example, in your specific case, you could model a ...

2

You can use CNN for time-series data. The Convolutional Recurrent Neural Network (RCNN) is one of the examples. Convolutional layers basically extract features from images. It is not related to time-series data processing. Some CNNs (such as in ResNet, Highway Networks, and DenseNet) use some recurrent concepts to improve their prediction, but they all are ...

2

Yes, definitely. In the simplest example, predicting an output value for a time series is classification. You take in the previous time steps and classify what is the most likely next value. You could do this with a RNN (Recurrent Neural Network) for example. If the activation functions are all linear, the nerual network is just a glorified linear regression....

2

You are right CNN based models can outperform RNN. You can take a look at this paper where they compared different RNN models with TCN (temporal convolutional networks) on different sequence modeling tasks. Even though there are no big differences in terms of results there are some nice properties that CNN based models offers such as: parallelism, stable ...

2

there's a lot to un-pack in this question. Why do they only pick 500 rows? my guess: in order to keep the example running quickly. tsfresh usually takes a while to calculate its features. note that when they evaluated their model, they took the last 500 samples. What's the point of re-arranging the rows/columns? answer: the data frame format that ...

1

How about a Temporal Convolutional Network? It feels like for such a long sequences having the recurrent/memory based approach is not too feasible. But, intuitively, the 1D convolutions should be able to pick out those rare features from your extremely long sequences. There are also claims that TCNs are comparable to RNNs in performance on common tasks, so ...

1

In general, multi output models is not that different. I.e. As Raghu mentioned in commentary, you could train separate model for each output. There is even helper module in sklearn for that (MultiOutputRegressor) DecisionTreeRegressor from sklearn allows multiple outputs out-of-the box Any neural network framework allows any number of outputs In your ...

1

If you have sold only once or very few items you will need some prior input (domain knowledge). One term for search is intermittent time series. Here is a stored search. When you have many time series, related, and interest in both totals and single series, that is called hierarchical forecasting. One expert is here (the author of that blog was the founder ...

1

There could be multiple possible ways to extract the features. One would be to use RNNs for a temporal relationship as the input data is time-series.

1

I'm currently working with Temporal Convolution Networks (TCNs) for making predictions with time series data (link to article here: https://medium.com/@raushan2807/temporal-convolutional-networks-bfea16e6d7d2). These types of networks, like other types of convolutional networks for time series, use a dilated convolution operation, which, unlike the standard ...

1

If I understand your problem correctly, you can test on just about any environment, and just omit parts of the observations to ensure your RNN is learning. For example, you can test on cartpole, ignoring the velocity and angular velocity states. This way the MDP isn't actually Markovian and you'll need the RNN to learn.

1

Your idea is a good one. Another idea is to upsample or aggregate your data. For example, average by week if you generally have a couple of missing days in every week. A similar question on Stack Exchange: https://stats.stackexchange.com/questions/374935/how-to-deal-with-really-sparse-time-series-data-for-a-binary-classification-task

1

An interesting model I encountered in a course is Facebook Prophet. Prophet takes into account trends, seasonality, and holidays for its predictions. As you can probably guess, this is a model that fits Facebook's needs very well. I'll give a brief introduction then provide a link where you can read more. Prophet fits a couple of functions of time ...

1

Yes, LSTM are ideal for this. For even stronger representational capacity, make your LSTM's multi-layered. Using 1-dimensional convolutions in a CNN is a common way to exctract information from time series too, so there's no harm in trying. Typically, you'll test many models out and take the one that has best validation performance.

1

Just for clarification: your description (1 sample per minute) does not match the example data (far fewer data points which is understandable, but also two data points in one minute which contradicts the initial assertion.) If your actual measurements are like that you should first work on the sampling process to get reliable data. For creating predictions, ...

1

Yes, you can fit any time series (with or without external variables) using HMM, but there are some constraints: It should follow the Markov property. There is some variance that other models are not able to capture (in other words, the system is partially observable). Adding to point 1, for HMM, it should hold true, but the way Baum Welch algorithm ...

1

Problem is in the output layer and you are using categorical_crossentropy for a loss function. Quoting Keras documentation: Note: when using the categorical_crossentropy loss, your targets should be in categorical format (e.g. if you have 10 classes, the target for each sample should be a 10-dimensional vector that is all-zeros except for a 1 at the ...

1

You need to have access to the 696th hour (or successive hours), otherwise, you cannot test your model. An alternative would be, for example, to train your model on the first 693 hours, validate it on the 694th hour, and test it on the 695th hour.

1

Definitely! but at that point it would be training a transformer-encoder (gpt2's architecture) and not GPT2 because GPT2 is defined by the weights / training procedure / data it was trained and not the architecture, and I don't think it would transfer properly to time series.

1

I have come across the same issue but in language. Where each input was a sentence, hence of different lengths. The easier solution is to just find the longest sequence, extract its length, and 0 pad all other values to be able to get all of them to the same size, and then use any recurrent neural network architecture (Since you're dealing with a time ...

1

(this response should be a comment but I don't have yet the reputation to comment). If I'm understanding your problem correctly you have a variable number of input which have an order and only one output ? It look like the kind of task where you could use recurrent neural network (the most common ones are the LSTM and GRU). If you use a recurrent neural ...

1

I have not come across a model like this yet. BUT If you have not tried smaller models I'd recommend trying that first. Justification: this lets you use learning curves to diagnose what to do next. Also, you might try starting with a GRU (the LSTM overhead may not be needed). One Idea for a starting model Observe that turnout does not affect weather. ...

1

In an RNN, the output of the previous state is passed as an input to the current state. Intuitively, there is a temporal (time-based) relationship in the way in which input is processed in an RNN. It can understand how the current state was achieved on the basis of the previous values, i.e value at time-step $t$ is a result of value at time-steps $t-1, t-2$, ...

Only top voted, non community-wiki answers of a minimum length are eligible