Questions tagged [regression]
For questions related to regression (both linear and non-linear) in the context of machine learning and AI.
132
questions
0
votes
1
answer
34
views
What are the consequences when we multiply, instead of add, a penalty term?
The typical objective function in regression problems like Lasso or Ridge includes a Residual Sum of Squares (RSS) term added to a penalty term based on a norm of the coefficients.
What are the ...
- 113
3
votes
1
answer
51
views
Regression loss conditioned by the ground-truth values
I'm working on a regression problem with a CNN in which the input is a single image, and the output is an angle in degrees (which determines a specific measure related to the image).
Sometimes, the ...
- 43
1
vote
1
answer
40
views
What number classes makes a classification problem continuous
I am working on a classification problem, where I have sequences of images and I want to train a model to predict the index of the image with some wanted property.
The target classes would obviously ...
- 21
0
votes
0
answers
25
views
How to make a RandomForestRegressor learn to differentiatie similar inputs with different outputs?
I'm working on a regression task with Sklearn RandomForestRegressor and I'm having some trouble distinguishing between two similar data with very different expected outputs. For example, each pair of ...
0
votes
0
answers
65
views
Does the accuracy of a regression learner depend on the way we feed data?
Consider a plot of points as such:
As one notices, this looks like an alternating sequence. Further, it can be divided into two subsequences as $a_{\text{odd}}$ and $a_{\text{even}}$ as they seem to ...
- 101
0
votes
0
answers
25
views
Pixel-wise regression only focus on edge
I am trying to use unet to learn pixel-wise regression from one image to one groundtruth with the same image size. The network seems to focus too much on the edge of the image, and it does not learn ...
- 1
0
votes
0
answers
16
views
Sparse linear discriminant analysis for regression problem?
So far, Linear Discriminant Analysis has beed used for classification problems http://proceedings.mlr.press/v38/wu15.pdf . I wonder if there are any ways to adapt it to regression problems?
- 1
1
vote
1
answer
34
views
CNN-Regression insensitive to input data
I'm currently training a CNN + multiple target regression model that does the following
input: $ \dim x = (L, 2), \text{where} \ x_i \in (-0.1, 0.1) $
output: $\dim y = (M), \text{where} \ y_i \geq ...
- 11
0
votes
1
answer
40
views
How can a Regression based Neural Network learn class thresholds?
I understand that to solve multilabel classification problems, we can use the softmax activation function in the output layer of the neural network. The softmax function outputs probabilities of each ...
- 13
0
votes
1
answer
198
views
How to do backpropagation with argmax?
I am attempting to utilize two networks: a classifier and a linear network. Based on the output class of the first network, my goal is to retrieve the corresponding value from the linear network using ...
0
votes
0
answers
11
views
Label transformation vs Methods in Imblanced Regression for Imbalanced Regression tasks
I've seen some papers discussing the imbalanced regression recently and was wondering what's the benefit of this line of approaches compared to conventional data transformations (e.g., Square-root, ...
- 101
2
votes
1
answer
1k
views
Is it possible to use LLMs for regression tasks?
I want to use LLMs to predict edge weights in a graph based on attributes between two nodes. Is this even possible? If not, what would you recommend?
I tried to look up uses of LLM in regression tasks,...
0
votes
0
answers
15
views
Regression Model overestimates in train-mode
I have a Deep Learning Regression model to predict some values. The results are fine when I use the model in Evaluation Mode, but when I turn Training Mode on the model tends to overestimate the ...
- 1
0
votes
0
answers
19
views
Confused about interaction terms in polynomial regression
I am trying to code multivariate polynomial regression from scratch and I got confused about how interaction terms work. I saw that a polynomial regression with 2 inputs and with interaction terms ...
0
votes
1
answer
24
views
Regression Model diverging after adding a new feature with higher variance and magnitude
In a time series regression problem I'm predicting "change" rather than the actual intended value i.e
Instead of:
...
3
votes
0
answers
76
views
How can i tinker my neural network to learn stronger on rare events?
I am training a neural network on a regression problem.
Most of the time the actual y (label) has the same value (say ~0.2) and only in rare cases the actual y is very different (say 2.0 or -2.0)
...
- 31
0
votes
0
answers
34
views
Using MSE or RMSE instead of CrossEntropy in Question Answering NLP problems. What are the problems if we used?
When you predict Start Index, end Index in Question Answering NLP task (SQUAD Data), you use CrossEntropy as a loss function. ...
- 253
0
votes
1
answer
30
views
why by adding additional information as number of sequence on dataset can avoid overfitting
I am developing a regression model to analyze walking styles. The dataset I am using to build the model is from 2 different sources, let's call them dataset A and dataset B.
Dataset A has a shape of <...
- 11
0
votes
0
answers
15
views
Regressing parameters that map between two curves machine learning
I am wondering if anyone has experience in regressing out parameters that map one curve to another. For example, I have two curves that look like this. I used some non-linear equation to map orange to ...
0
votes
0
answers
19
views
What happens if one uses non-valid kernels in regression?
When introducing kernelization in regression, it is emphasized that a kernel function $k(x_i, x_j)$ has to represent a scalar product in some high dimensional feature space, $k(x_i, x_j) = \phi(x_i)^T ...
- 101
0
votes
0
answers
55
views
How to create a global forecasting model using deep learning?
I am aiming to build a global/general forecasting model (don't know what's the proper terminology) using deep learning. The idea behind this is to create a model trained on several time series that ...
- 23
0
votes
0
answers
33
views
How to output a function given a time series data as an input using supervised learning?
I have a spreadsheet with time series data collected from two sensors, one measuring temperature and the other measuring humidity. And I also collected data from an experiment that I conducted, the ...
- 1
2
votes
2
answers
142
views
Why is a simple regression problem so hard for an MLP to learn?
Consider a very simple problem, which is to find the maximum value out of a list of 5 numbers between 0 and 1. This is obviously trivial, but serves as a good example for a real-world problem I'm ...
- 191
0
votes
0
answers
34
views
Multi-output regression problems
I am training CNNs on 3D image data (dimensions [500, 512, 512]) to locate 7 3D points inside the image. I have thought of two different ways to solve this problem, ...
2
votes
1
answer
58
views
Is there any interpretation method suitable for CNNs which do regression tasks?
I mainly tackle regression problems by CNNs, and want to find a reliable method to calculate the heatmaps for NN's results. However, I find almost all interpretation methods including CAM is used for ...
- 21
0
votes
0
answers
19
views
What's a good regression algorithm for handling tabular data that have categorical data, "list of words"
Problem statement: I want to predict future prices of trips based on historical pricing data.
I'm looking for an algorithm that has the following features:
Unsupervised algorithm
Limit the amount of ...
- 1
1
vote
0
answers
28
views
bad prediction when having noise on the data: LSTM time-series regression
I want to predict the force plate using a smart insole using the LSTM model for time series prediction. the data on the force plate has positive and negative values (I think the resulting positive ...
- 11
0
votes
1
answer
443
views
Why does GridSearchCV model give worse results despite same parameters used with base model
I am trying to make prediction using random forest regression and then utilize GridSearchCV to tune hyperparameters(just 'n_estimators'). However results of GridSearchCV are worse than base model. ...
0
votes
1
answer
48
views
Multi-layer network only predicts linear trends
I have made a neural network from scratch (in java), which is refusing to switch out of linear regression. I have pushed up the layer sizes (it now has 2 hidden layers, both with 5 neurons), and yet ...
- 13
1
vote
0
answers
336
views
Out of distribution detection (OOD) in the context of regression problems
I'm working in a regression setting to predict a scalar value $y$ from an input $\textbf{x} \in \mathbb{R}^D$ and I'm interested in understanding whenever my model is fed with something that it is ...
- 307
1
vote
1
answer
42
views
How do I interpret this loss function?
In this AI note from https://deeplearning.ai, the loss function below is used for a regression problem. However, I don't know how to interpret this loss function.
First, does the author take the ...
- 113
0
votes
0
answers
52
views
Basic question about gradient for nominal regression
Say that we want to binary-classify images using a sigmoid function with the entropy-loss function. This algorithm is quite slow.
The sigmoid function is:
I find that this could be traced to the $L(y,...
- 79
1
vote
0
answers
241
views
Alternatives to Bayesian optimization
I am given a dataset $\mathcal{D} = \{\mathbf{x}_i\}_{i=1}^n$ and I need to find the point (in my case a material) $\mathbf{x}^*$ that maximizes a property $y$ (which can be obtained from a black-box ...
- 150
1
vote
0
answers
165
views
Active Learning regression with Random Forest
I have a dataset of about 8k points and I am trying to employ active learning with the random forest regressor. I have split the dataset to train and ...
- 150
1
vote
0
answers
22
views
How can evaluate the success of my algoritm?
A little bit of context. I have a classification algorithm based on mathematical discriminator and I am not applying any machine learning or AI technique, just moving window and several relative ...
- 123
1
vote
2
answers
38
views
Dealing with incomplete file sets for a CNN for medical imaging regression problem
I'm trying to solve a medical imaging regression problem using a CNN. Each of the samples in my data set consists of one, two, or three of the following file types:
flair.nii.gz
mprage.nii.gz
swi....
- 125
0
votes
1
answer
130
views
Entirely linear neural network learning non-linear function
I have a neural network that's trained on a sine wave. It uses a lookback of 20 to see what the last 20 predictions were and predict the next value. This network has only a single Linear layer (input ...
- 1,346
0
votes
1
answer
47
views
How do I know if my Random Forest Regressor Model is overfitted?
Im creating a Random Forest Regressor Model with a small dataset (30 data points). I tried with other models but RF was the best one, however, after applying GridSearchCv I got that the training set ...
- 3
0
votes
1
answer
670
views
Is there any way to train a regression model with negative values that is more stable?
I have a regression model where my target values contain roughly 60% negative values and 40% positive values. My model architecture includes a robert-large, 1 linear layer. I trained it after 1 epoch, ...
0
votes
0
answers
19
views
Which existing model could be used for wind speed and direction prediction?
I am trying to predict the wind speed and wind direction in a graph network for a geographical area. The dataset includes the start and end nodes, the distance between them, and wind speed and ...
- 1
1
vote
1
answer
80
views
Prediction of continuous variable based on threshold
The independent variables are date, count, atmp, and ...
- 111
0
votes
1
answer
353
views
How to make a proper approximation of Sine function with Neural Networks?
TL;DR;
How to build a neural network that properly approximates the sine function with different ranges?
Context and Question:
From this question I decided to use the Sergey's answer, however I used a ...
- 111
3
votes
1
answer
99
views
Why does my regression-NN completely fail to predict some points?
I would like to train a NN in order to approximate an unknown function $y = f(x_1,x_2)$. I have a lot of measurements $y = [y_1,\dots,y_K]$ (with K that could be in the range of 10-100 thousands) ...
- 31
0
votes
1
answer
810
views
What's the best model to use for CNN(deep learning) regression task for small image dataset?
What are the best Deep learning models(with how many layers) to use in a regression task for a custom dataset containing around 100 images of only one object per image which is more or less ...
- 1
1
vote
1
answer
82
views
Does a second-order fully-connected layer have any uses?
I was thinking about implementing second-order regression via a fully-connected layer, and I came up with this:
$X$ is the input data, shaped $(features, batch\_number)$.
$w0$ is the bias, shaped $(...
- 113
-1
votes
1
answer
61
views
Is my dataset a time series dataset? and should I use an LSTM?
I have a dataset where I am recording temperature after every 4milliseconds till 500 and another feature "conductivity value". The length of the dataset is around a 1000 rows. I need to find ...
- 51
0
votes
0
answers
15
views
What to predict in a limited transaction dataset?
I have been given a task with a real transaction dataset. The task is to predict something using either logistic regression or simple binary classification.
The columns are as follow:
Transaction ID
...
- 31
1
vote
0
answers
66
views
Is the VC dimension of a MLP regressor a valid upper bound on how many points it can exactly fit?
I want to calculate an upper bound on how many training points an MLP regressor can fit with ~0 error. I don't care about the test error, I want to overfit as much as possible the (few) training ...
- 11
0
votes
1
answer
115
views
Do I need to tune the hyper-parameters or more data if SVR model performs poorly?
I am using non-linear data to SVR and have tried tuning the hyperparameters and still have a poor model performance. Do I need more data or format the data for more suitable results?
I get similar ...
1
vote
0
answers
14
views
What is the best way to train a text-based regressor model?
I want to build a deep learning model that can predict a continuous value (LogP in this case) given text inputs (SMILES notations in this case), the dataset is as illustrated below.
SMILES notations
...
- 121