2 votes
Accepted

Is data useless for a neural network if some inputs are derivatives of other inputs?

No it is not useless. The relationship may not be obvious, and having the data will allow the network to learn this 𝑓 relationship. Further, even if 𝑓 is obvious, networks are so sample inefficient ...
chessprogrammer's user avatar
2 votes

Would maximizing (instead of minimizing) error of an LLM/HMM lead to complex behavior?

You cannot really invert the loss, because that's undefined most likely. Take linear regression with OLS, then we know that the loss function is quadratic wrt the parameters (assuming to have 1 ...
Alberto's user avatar
  • 1,358
2 votes
Accepted

What are the differences between loss surfaces that "derive"from different observations?

Let's take in consideration linear regression. You have a dataset composed by $x,y$ pairs, and you assume they are linearly related, thus you model this problem with LR: $$ y = wx+b $$ Now, you want ...
Alberto's user avatar
  • 1,358
1 vote

Would maximizing (instead of minimizing) error of an LLM/HMM lead to complex behavior?

In HMM, a simple mechanism to reduce overfitting and therefore generating variety in the system output is tweaking the transition matrix A and/or the symbol emission probability B. Allowing for some ...
Jaume Oliver Lafont's user avatar
1 vote
Accepted

Creating a Dataset from Time Series Data

What you are trying to accomplish is called sequence prediction. It takes in a sequence of data, and spits out a score (target variable). Time-series data is usually formatted as 3D arrays with shape <...
Robin van Hoorn's user avatar
1 vote
Accepted

What are possible reasons for the validation loss increasing with more data?

As you pointed out, duplicates can be a possible reason for such behavior. There are a few more possibilities: Class Imbalance - data is skewed towards a particular class(if you are solving a ...
Nityanand Mathur's user avatar
1 vote

How can imitation learning data be collected?

Imitation learning data usually means data gathered from an expert, that is data from an agent proficient in the task. The agent may be: A human operator: have the operator complete the task and ...
Rexcirus's user avatar
  • 1,141
1 vote
Accepted

Generating synthetic time series data with limited data

Synthetic data generation is a hot and new topic at the moment. Im writing my MSc thesis on time-series synthetic data generation using TimeGAN. First, overfitting is a general problem, which can also ...
Robin van Hoorn's user avatar
1 vote

How to perform binary classification when one class is more predominant than the other?

From your case, it seems like you want your algorithm to classify both 1s and 0s with high accuracy. To increase the number of 1s and get it to a comparable level as 0s, you could generate new ...
Aditya Goyal's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible