Questions tagged [regression]
For questions related to regression (both linear and non-linear) in the context of machine learning and AI.
121
questions
0
votes
0
answers
9
views
bad prediction when having noise on the data: LSTM time-series regression
I want to predict the force plate using a smart insole using the LSTM model for time series prediction. the data on the force plate has positive and negative values (I think the resulting positive ...
0
votes
1
answer
37
views
Why does GridSearchCV model give worse results despite same parameters used with base model
I am trying to make prediction using random forest regression and then utilize GridSearchCV to tune hyperparameters(just 'n_estimators') however results of GridSearchCV are worse than base model. ...
0
votes
1
answer
43
views
Multi-layer network only predicts linear trends
I have made a neural network from scratch (in java), which is refusing to switch out of linear regression. I have pushed up the layer sizes (it now has 2 hidden layers, both with 5 neurons), and yet ...
- 13
1
vote
0
answers
63
views
Out of distribution detection (OOD) in the context of regression problems
I'm working in a regression setting to predict a scalar value $y$ from an input $\textbf{x} \in \mathbb{R}^D$ and I'm interested in understanding whenever my model is fed with something that it is ...
- 257
0
votes
0
answers
7
views
Expected Revenue Using Gradient Boost for Regression
I have trained a ML algorithm (gradient boost) to do regression on banana prices, such that I can guess the selling price of any given banana. Using scikit's regression boost algorithm, I am able to ...
- 37
0
votes
0
answers
22
views
Why does a quantile regression estimator underestimate the variance when using the quantile huber loss?
I have a question to quantile regression which is related to distributional Reinforcement Learning. Let the quantile loss (QL) be defined as
\begin{align*}
\mathcal{L}^{\tau}_{\text{QR}}(\...
- 35
0
votes
0
answers
19
views
Approaching time series classification with feature dependend label
In one of my projects it's all about modelling wether a continuos condition $Y_1$ become greater or less than a threshold. For this purpose, I have a huge Collection of time series data of ...
- 69
0
votes
0
answers
10
views
Tree boosting additive loss
In the XGBoost documentation, they specify that the additive training is done given an objective $obj^{(t)}$ defined as
$obj^{(t)} = \sum\limits_{i=1}^n \ell(y_i, \hat{y}_i^{(t-1)}+f_t(x_i)) + \sum\...
- 281
1
vote
1
answer
35
views
How do I interpret this loss function?
In this AI note from https://deeplearning.ai, the loss function below is used for a regression problem. However, I don't know how to interpret this loss function.
First, does the author take the ...
- 113
0
votes
0
answers
47
views
Basic question about gradient for nominal regression
Say that we want to binary-classify images using a sigmoid function with the entropy-loss function. This algorithm is quite slow.
The sigmoid function is:
I find that this could be traced to the $L(y,...
- 1
1
vote
0
answers
70
views
Alternatives to Bayesian optimization
I am given a dataset $\mathcal{D} = \{\mathbf{x}_i\}_{i=1}^n$ and I need to find the point (in my case a material) $\mathbf{x}^*$ that maximizes a property $y$ (which can be obtained from a black-box ...
- 121
1
vote
0
answers
24
views
Active Learning regression with Random Forest
I have a dataset of about 8k points and I am trying to employ active learning with the random forest regressor. I have split the dataset to train and ...
- 121
1
vote
0
answers
19
views
How can evaluate the success of my algoritm?
A little bit of context. I have a classification algorithm based on mathematical discriminator and I am not applying any machine learning or AI technique, just moving window and several relative ...
- 11
0
votes
1
answer
13
views
Dealing with incomplete file sets for a CNN for medical imaging regression problem
I'm trying to solve a medical imaging regression problem using a CNN. Each of the samples in my data set consists of one, two, or three of the following file types:
flair.nii.gz
mprage.nii.gz
swi....
- 115
0
votes
1
answer
73
views
Entirely linear neural network learning non-linear function
I have a neural network that's trained on a sine wave. It uses a lookback of 20 to see what the last 20 predictions were and predict the next value. This network has only a single Linear layer (input ...
- 1,301
0
votes
0
answers
26
views
Is it possible to use a Support Vector Regression for continuous variables?
I'm working with a small dataset to predict the temperature (30 data points), as I don't have enough information, Im using RFE method so I use only the necessary features. However, I'm still searching ...
- 3
0
votes
1
answer
26
views
How do I know if my Random Forest Regressor Model is overfitted?
Im creating a Random Forest Regressor Model with a small dataset (30 data points). I tried with other models but RF was the best one, however, after applying GridSearchCv I got that the training set ...
- 3
0
votes
1
answer
195
views
Is there any way to train a regression model with negative values that is more stable?
I have a regression model where my target values contain roughly 60% negative values and 40% positive values. My model architecture includes a robert-large, 1 linear layer. I trained it after 1 epoch, ...
0
votes
1
answer
135
views
YearPrediction dataset for a regression task: is it possible to evaluate a fair comparison between standard loss and a quadratic one?
We are trying to evaluate a loss function on the Year Prediction (Million Songs) data set.
The problem is that we don't know how to configure an experiment in order to test if one loss (the standard ...
0
votes
0
answers
58
views
Predicting using time-series data and static data?
I have recently been working on predicting the final value of articles on Steemit.com using downloaded data. I have a large variety of features which divide into two types. Features which change over ...
0
votes
0
answers
16
views
Which existing model could be used for wind speed and direction prediction?
I am trying to predict the wind speed and wind direction in a graph network for a geographical area. The dataset includes the start and end nodes, the distance between them, and wind speed and ...
- 1
1
vote
1
answer
33
views
Prediction of continuous variable based on threshold
The independent variables are date, count, atmp, and ...
- 111
0
votes
0
answers
15
views
Confidence intervals for regression
I have submitted a paper to arxiv and their moderator want me to write confidence interval in paper.
The response i got is that
...
- 123
0
votes
0
answers
15
views
Given a dataset with columns $x_1, x_2, \dots, x_n, y$, which algorithm can tell me which $x$ values had the biggest impact on $y$?
I have a set of data that is generated from a monolithic Monte Carlo-based program. I've set up a way to monitor the calculations of this program and output values correlating to those calculations, ...
0
votes
0
answers
33
views
Transformers for regression on permutation of fixed size sequence?
Transformers have shown remarkable performance operating on sequences, but are equivariant to the order in the input sequence. Positional Encoding alleviates that problem, but how good is it?
In my ...
- 246
0
votes
0
answers
23
views
Neural network have difficulty on capturing weak characteristics
I want use neural network to approximate a non-linear function. The function is,
$$
F(X1,X2,X3) = A \times X1^{K1} \times exp((X1-X2) \times K2) \times exp(X3 \times K3)
$$
where X1/X2/X3 are input ...
0
votes
0
answers
14
views
Travel time between locations, two features with locations or one feature with segment
I'm working on a project where we are using supervised machine learning to predict the travel time between locations. The output value is the travel time. I'm debating whether to have two features or ...
0
votes
0
answers
17
views
What is the canocial way to handling differing input and output dimensions for the transformer model?
I have an essential regression task, where the input is of dimension $d$ and the output is a scalar. I think the transformer model is a good fit for this problem. In the vanilla multi-head-attention ...
- 51
0
votes
0
answers
86
views
What can be the reasons for validation MSE < training MSE at beginning of training and network failing to generalize afterwards?
I am using a Convolutional Neural Network for regressing time series data. The objective is to predict an obfuscated metric. The training metrics and losses are as follows.
The val_loss is lower than ...
- 33
0
votes
1
answer
205
views
How to make a proper approximation of Sine function with Neural Networks?
TL;DR;
How to build a neural network that properly approximates the sine function with different ranges?
Context and Question:
From this question I decided to use the Sergey's answer, however I used a ...
- 111
3
votes
1
answer
74
views
Why does my regression-NN completely fail to predict some points?
I would like to train a NN in order to approximate an unknown function $y = f(x_1,x_2)$. I have a lot of measurements $y = [y_1,\dots,y_K]$ (with K that could be in the range of 10-100 thousands) ...
- 31
0
votes
1
answer
520
views
What's the best model to use for CNN(deep learning) regression task for small image dataset?
What are the best Deep learning models(with how many layers) to use in a regression task for a custom dataset containing around 100 images of only one object per image which is more or less ...
- 1
1
vote
1
answer
53
views
Does a second-order fully-connected layer have any uses?
I was thinking about implementing second-order regression via a fully-connected layer, and I came up with this:
$X$ is the input data, shaped $(features, batch\_number)$.
$w0$ is the bias, shaped $(...
- 113
-1
votes
1
answer
58
views
Is my dataset a time series dataset? and should I use an LSTM?
I have a dataset where I am recording temperature after every 4milliseconds till 500 and another feature "conductivity value". The length of the dataset is around a 1000 rows. I need to find ...
- 51
0
votes
0
answers
11
views
What to predict in a limited transaction dataset?
I have been given a task with a real transaction dataset. The task is to predict something using either logistic regression or simple binary classification.
The columns are as follow:
Transaction ID
...
- 31
1
vote
0
answers
58
views
Is the VC dimension of a MLP regressor a valid upper bound on how many points it can exactly fit?
I want to calculate an upper bound on how many training points an MLP regressor can fit with ~0 error. I don't care about the test error, I want to overfit as much as possible the (few) training ...
- 11
0
votes
1
answer
79
views
Do I need to tune the hyper-parameters or more data if SVR model performs poorly?
I am using non-linear data to SVR and have tried tuning the hyperparameters and still have a poor model performance. Do I need more data or format the data for more suitable results?
I get similar ...
1
vote
0
answers
13
views
What is the best way to train a text-based regressor model?
I want to build a deep learning model that can predict a continuous value (LogP in this case) given text inputs (SMILES notations in this case), the dataset is as illustrated below.
SMILES notations
...
- 121
1
vote
1
answer
43
views
Best way to measure regression accuracy?
I'm asking because classification problems have very concrete metrics like accuracy that are totally transparent to understand.
Whereas regression models seem to have a very large number of possible ...
- 296
0
votes
2
answers
151
views
How to pass multiple vectors and numeric features as input to the neural network? [closed]
I need help in a regression scenario. I have 12 input features. 4 of which are coordinates (each is a vector) in XYZ plane ...
0
votes
0
answers
27
views
Regress values inside the bounding boxes to predict a value in Object Detection
I am currently working on an object detection task. I have a dataset of Grayscale and Depth Images. The annotation format is x1, y1, x2, y2, class, depth. I have calculated this depth (of each object/...
- 1
0
votes
1
answer
81
views
Which loss function could I use to solve a regression problem as a classification problem (where we discretize the labels into buckets)?
I am considering a rather typical regression problem, but, for practice, I am trying to implement this as a classification problem.
The setup is as follows. I have $\mathbb{R}$-valued labels $y_i \in [...
- 3
0
votes
0
answers
28
views
Regression for a discrete variable
I'm building a model (neural net) that would predict a quality score for images.
Ground truth is given by a 4-level discrete variable (0%, 33%, 67%, 100%), and I would like to build a model that would ...
- 101
0
votes
1
answer
49
views
How to explain that a same DNN model have radically different behaviours with each new initialization and training?
I'm trying to predict the continuous values of a variable $y$ using a Fully Connected Neural Network while providing it with data from a $(3300, 13)$ matrix $X$ where $X[i, :]=[0,...,1,...,0,x_{i}]$. ...
- 509
1
vote
0
answers
76
views
Is it possible to use RGB image with decimal values when feeding training data to CNN?
I am working with four grayscale images of float32 data type to perform regression using Keras. Three images are stacked using np.dstack to form a RGB data-set. The ...
- 11
3
votes
2
answers
181
views
Can predictions of a neural network using ReLU activation be non-linear (i.e. follow the pattern) outside of the scope of trained data?
Training on a quadratic function
x = np.linspace(-10, 10, num=1000)
np.random.shuffle(x)
y = x**2
Will predict an expected quadratic curve between ...
2
votes
1
answer
319
views
How to get more accuracy of the logistic regression model?
I am working on a Baby Crying Detection model using logistic regression.
Out of $581$ audios, $222$ are of a baby crying. Each audio is of $5$ seconds.
what I have done is convert each audio into ...
0
votes
0
answers
165
views
Predicting the probability of a periodically happening event occurring at a given time
I have encountered this problem on how to predict the probability of a periodically happening event occurring at a given time.
For example, we have an event called being_an_undergrad. There are many ...
- 1
0
votes
1
answer
268
views
How to forecast multiple target attributes in Python?
I need to forecast two non-correlated time-series (non-stationary). A sample is presented below:
...
0
votes
0
answers
47
views
Multi-target regression using scikit-learn without ytrain
I would like to use the multi-target regression with scikit-learn. However, the examples I've seen use Xtrain and ytrain?
What is ytrain in regression?
I know y it is used for classes in ...
- 1