2
votes
Accepted
Is data useless for a neural network if some inputs are derivatives of other inputs?
No it is not useless.
The relationship may not be obvious, and having the data will allow the network to learn this 𝑓 relationship.
Further, even if 𝑓 is obvious, networks are so sample inefficient ...
- 2,355
2
votes
Would maximizing (instead of minimizing) error of an LLM/HMM lead to complex behavior?
You cannot really invert the loss, because that's undefined most likely.
Take linear regression with OLS, then we know that the loss function is quadratic wrt the parameters (assuming to have 1 ...
- 1,358
2
votes
Accepted
What are the differences between loss surfaces that "derive"from different observations?
Let's take in consideration linear regression. You have a dataset composed by $x,y$ pairs, and you assume they are linearly related, thus you model this problem with LR:
$$
y = wx+b
$$
Now, you want ...
- 1,358
1
vote
Would maximizing (instead of minimizing) error of an LLM/HMM lead to complex behavior?
In HMM, a simple mechanism to reduce overfitting and therefore generating variety in the system output is tweaking the transition matrix A and/or the symbol emission probability B. Allowing for some ...
1
vote
Accepted
Creating a Dataset from Time Series Data
What you are trying to accomplish is called sequence prediction. It takes in a sequence of data, and spits out a score (target variable).
Time-series data is usually formatted as 3D arrays with shape <...
- 2,103
1
vote
Accepted
What are possible reasons for the validation loss increasing with more data?
As you pointed out, duplicates can be a possible reason for such behavior.
There are a few more possibilities:
Class Imbalance - data is skewed towards a particular class(if you are solving a ...
1
vote
How can imitation learning data be collected?
Imitation learning data usually means data gathered from an expert, that is data from an agent proficient in the task.
The agent may be:
A human operator: have the operator complete the task and ...
- 1,141
1
vote
Accepted
Generating synthetic time series data with limited data
Synthetic data generation is a hot and new topic at the moment. Im writing my MSc thesis on time-series synthetic data generation using TimeGAN.
First, overfitting is a general problem, which can also ...
- 2,103
1
vote
How to perform binary classification when one class is more predominant than the other?
From your case, it seems like you want your algorithm to classify both 1s and 0s with high accuracy. To increase the number of 1s and get it to a comparable level as 0s, you could generate new ...
- 11
Only top scored, non community-wiki answers of a minimum length are eligible