4

In language theory, there are generally several admitted levels that can be studied in relation with one another or independently. The semantic level is the one dealing with the meaning of the text ("semantic" comes from the greek and means "to signify"). The semantic level is therefore generally independent from the syntax and even the language used to ...


3

Usually you need to ensure that your convolutions are causal, meaning that there is no information leakage from the future into the past. You could start by looking at this paper which compares Temporal Convolutional Networks (TCN) with vanilla RNNs models.


2

You can use CNN in timeseries data. Convolutional Recurrent Neural Network(RCNN) is one of the examples. Convolutional layers basically extract feature from image, It is not related to time series data passing, Neither of them you mention on the question. CNN therefore use some recurrent concept to improve their prediction such as in ResNet, Highway Networks,...


2

You are right CNN based models can outperform RNN. You can take a look at this paper where they compared different RNN models with TCN (temporal convolutional networks) on different sequence modeling tasks. Even though there are no big differences in terms of results there are some nice properties that CNN based models offers such as: parallelism, stable ...


2

there's a lot to un-pack in this question. Why do they only pick 500 rows? my guess: in order to keep the example running quickly. tsfresh usually takes a while to calculate its features. note that when they evaluated their model, they took the last 500 samples. What's the point of re-arranging the rows/columns? answer: the data frame format that ...


2

Given the (usual) higher architectural complexity of ML models compared to more classical forecasting models, ML models might also require more data, otherwise they might just overfit the training dataset. Furthermore, online learning (or training) of a neural network using stochastic gradient descent (that is, one example at a time) might also be ...


2

ANNs & RNNs can be used to create some great models in many different domains, including time-series forecasting. However, across all of these domains, they suffer from the problem of hyper-parameter optimization. Because neural networks are so flexible, it is not clear, at the outset, which arrangement of neurons will be most effective to solve a given ...


2

The second implementation looks more correct and inline with how Bidirectional is defined. Specifically, bidirectionality doen't change the forward/backward logic of either direction, and just merges (concat/sum/...) the outputs of forward/backward at a matching timestep t. You can check how Keras implements it here. There are distinct self.forward_layer ...


1

I do not know how you will apply your data to the techniques I'll give you some brief overview of techniques used in time series prediction: Extended Kalman Filtering: This is a kind of control system approach and is generally used to control trajectory of missiles. Here is a question (based on an EKF paper) in our stack on this topic. You can check the ...


1

Things like this a really hot topic in research right now, and it's very difficult to get high accuracy on a chaotic system like the stock market. That being said, I would probably recommend preprocessing your data rather than having your primary neural network decide what to accept and what not to. For example, in your specific case, you could model a ...


1

Yes, LSTM are ideal for this. For even stronger representational capacity, make your LSTM's multi-layered. Using 1-dimensional convolutions in a CNN is a common way to exctract information from time series too, so there's no harm in trying. Typically, you'll test many models out and take the one that has best validation performance.


1

Just for clarification: your description (1 sample per minute) does not match the example data (far fewer data points which is understandable, but also two data points in one minute which contradicts the initial assertion.) If your actual measurements are like that you should first work on the sampling process to get reliable data. For creating predictions, ...


1

Yes, you can fit any time series (with or without external variables) using HMM, but there are some constraints: It should follow the Markov property. There is some variance that other models are not able to capture (in other words, the system is partially observable). Adding to point 1, for HMM, it should hold true, but the way Baum Welch algorithm ...


1

Problem is in the output layer and you are using categorical_crossentropy for a loss function. Quoting Keras documentation: Note: when using the categorical_crossentropy loss, your targets should be in categorical format (e.g. if you have 10 classes, the target for each sample should be a 10-dimensional vector that is all-zeros except for a 1 at the ...


1

You need to have access to the 696th hour (or successive hours), otherwise, you cannot test your model. An alternative would be, for example, to train your model on the first 693 hours, validate it on the 694th hour, and test it on the 695th hour.


1

Definitely! but at that point it would be training a transformer-encoder (gpt2's architecture) and not GPT2 because GPT2 is defined by the weights / training procedure / data it was trained and not the architecture, and I don't think it would transfer properly to time series.


1

I have come across the same issue but in language. Where each input was a sentence, hence of different lengths. The easier solution is to just find the longest sequence, extract its length, and 0 pad all other values to be able to get all of them to the same size, and then use any recurrent neural network architecture (Since you're dealing with a time ...


1

(this response should be a comment but I don't have yet the reputation to comment). If I'm understanding your problem correctly you have a variable number of input which have an order and only one output ? It look like the kind of task where you could use recurrent neural network (the most common ones are the LSTM and GRU). If you use a recurrent neural ...


1

Under the assumption, that the papers at Google Scholar are expressing the current state of technology it's easy to take a closer look into the list to determine in which conference proceedings they were published in the past. The first information is, that a dedicated conference on human activity recognition isn't available, but the topic is spread over ...


1

Couple reccomendations: 1) I dont think your overfitting, your test loss is not ever increasing and is staying reasonbly proportional to train loss -- This may indicate that whatever loss your using is not a good indicator of the metric of interest (in this case, it seems you want that to be accuracy, but data is imbalnced so maybe look at avg precision?) 2) ...


Only top voted, non community-wiki answers of a minimum length are eligible