Questions tagged [regression]

For questions related to regression (both linear and non-linear) in the context of machine learning and AI.

Filter by
Sorted by
Tagged with
-2 votes
0 answers
15 views

Lost when trying to get good time series prediction results (regression problem) even after trying many things

I'm not able to get good results after a long time testing when using TensorFlow to predict time series data (regression problem). I don't know if the problem is with the data (little quantity and/or ...
0 votes
0 answers
18 views

CNN multioutput regression architecture modification

I am working on a regression task where the model has to predict two values at the same time. The idea is that the dataset consists of 16 features, where the first 8 features represent the first value ...
1 vote
1 answer
180 views

How does the regression layer in the localization network of a spatial transformer work?

I am trying to understand the spatial transformer network mentioned in this paper https://papers.nips.cc/paper/5854-spatial-transformer-networks.pdf. I am clear about the last two stages of the ...
2 votes
1 answer
147 views

What work has been done with Poisson-style regression via neural networks with exponential activation functions?

The first neural net I wrote was a classifier. After that, I learned that neural nets can be used for regression tasks, even quantile regression. It has become clear to me that the usual games with ...
0 votes
1 answer
894 views

What's the best model to use for CNN(deep learning) regression task for small image dataset?

What are the best Deep learning models(with how many layers) to use in a regression task for a custom dataset containing around 100 images of only one object per image which is more or less ...
0 votes
0 answers
45 views

Find the relationship between data in this plot

Attached image. How would you find the relationship between independent variable (x) and dependent variable (y)? Is it linear or non-linear? What would the function looks like? P.S. I believe this is ...
0 votes
1 answer
35 views

Circular regression for joystick movements?

I've been playing around with some behavioral cloning of a simple old game that uses a joystick. As with behavioral cloning in general, if I record many games, then for each state there are many ...
1 vote
0 answers
35 views

Achieving low train error for exponential response?

I'm trying to fit an ML model with perfect information to predict an exponentially distributed response without getting exponentially distributed error... Also, this situation is special for a few of ...
1 vote
2 answers
40 views

Dealing with incomplete file sets for a CNN for medical imaging regression problem

I'm trying to solve a medical imaging regression problem using a CNN. Each of the samples in my data set consists of one, two, or three of the following file types: flair.nii.gz mprage.nii.gz swi....
0 votes
1 answer
36 views

chained linear regression models vs feed forward NN

I am trying to understand the difference between feedforward NN and chained linear regression models, if and why they can model nonlinear functions. both are able to model non-linear dependencies ...
1 vote
1 answer
451 views

Handling imbalanced data with multiple targets

I have the model which has 3 outputs (it is a regression task, I have the angle of the steering wheel, brake and acceleration). I can divide my values to some smaller bins and in this way I can change ...
1 vote
1 answer
39 views

CNN-Regression insensitive to input data

I'm currently training a CNN + multiple target regression model that does the following input: $ \dim x = (L, 2), \text{where} \ x_i \in (-0.1, 0.1) $ output: $\dim y = (M), \text{where} \ y_i \geq ...
0 votes
1 answer
600 views

Why does GridSearchCV model give worse results despite same parameters used with base model

I am trying to make prediction using random forest regression and then utilize GridSearchCV to tune hyperparameters(just 'n_estimators'). However results of GridSearchCV are worse than base model. ...
1 vote
2 answers
71 views

Is MSE cost for a linear regression model a convex function with one global optimum?

Here is the thing: MSE cost for a linear regression model is a convex function with one global optimum, and it can be solved efficiently using gradient descent or in closed form (SVD, normal eq. ....)...
0 votes
1 answer
31 views

why by adding additional information as number of sequence on dataset can avoid overfitting

I am developing a regression model to analyze walking styles. The dataset I am using to build the model is from 2 different sources, let's call them dataset A and dataset B. Dataset A has a shape of <...
0 votes
1 answer
37 views

What are the consequences when we multiply, instead of add, a penalty term?

The typical objective function in regression problems like Lasso or Ridge includes a Residual Sum of Squares (RSS) term added to a penalty term based on a norm of the coefficients. What are the ...
3 votes
1 answer
67 views

Regression loss conditioned by the ground-truth values

I'm working on a regression problem with a CNN in which the input is a single image, and the output is an angle in degrees (which determines a specific measure related to the image). Sometimes, the ...
1 vote
1 answer
45 views

What number classes makes a classification problem continuous

I am working on a classification problem, where I have sequences of images and I want to train a model to predict the index of the image with some wanted property. The target classes would obviously ...
0 votes
0 answers
25 views

How to make a RandomForestRegressor learn to differentiatie similar inputs with different outputs?

I'm working on a regression task with Sklearn RandomForestRegressor and I'm having some trouble distinguishing between two similar data with very different expected outputs. For example, each pair of ...
0 votes
0 answers
65 views

Does the accuracy of a regression learner depend on the way we feed data?

Consider a plot of points as such: As one notices, this looks like an alternating sequence. Further, it can be divided into two subsequences as $a_{\text{odd}}$ and $a_{\text{even}}$ as they seem to ...
2 votes
1 answer
60 views

Is there any interpretation method suitable for CNNs which do regression tasks?

I mainly tackle regression problems by CNNs, and want to find a reliable method to calculate the heatmaps for NN's results. However, I find almost all interpretation methods including CAM is used for ...
0 votes
0 answers
33 views

Pixel-wise regression only focus on edge

I am trying to use unet to learn pixel-wise regression from one image to one groundtruth with the same image size. The network seems to focus too much on the edge of the image, and it does not learn ...
0 votes
0 answers
16 views

Sparse linear discriminant analysis for regression problem?

So far, Linear Discriminant Analysis has beed used for classification problems http://proceedings.mlr.press/v38/wu15.pdf . I wonder if there are any ways to adapt it to regression problems?
0 votes
1 answer
47 views

How can a Regression based Neural Network learn class thresholds?

I understand that to solve multilabel classification problems, we can use the softmax activation function in the output layer of the neural network. The softmax function outputs probabilities of each ...
0 votes
1 answer
315 views

How to do backpropagation with argmax?

I am attempting to utilize two networks: a classifier and a linear network. Based on the output class of the first network, my goal is to retrieve the corresponding value from the linear network using ...
0 votes
0 answers
13 views

Label transformation vs Methods in Imblanced Regression for Imbalanced Regression tasks

I've seen some papers discussing the imbalanced regression recently and was wondering what's the benefit of this line of approaches compared to conventional data transformations (e.g., Square-root, ...
2 votes
1 answer
2k views

Is it possible to use LLMs for regression tasks?

I want to use LLMs to predict edge weights in a graph based on attributes between two nodes. Is this even possible? If not, what would you recommend? I tried to look up uses of LLM in regression tasks,...
0 votes
0 answers
18 views

Regression Model overestimates in train-mode

I have a Deep Learning Regression model to predict some values. The results are fine when I use the model in Evaluation Mode, but when I turn Training Mode on the model tends to overestimate the ...
0 votes
0 answers
24 views

Confused about interaction terms in polynomial regression

I am trying to code multivariate polynomial regression from scratch and I got confused about how interaction terms work. I saw that a polynomial regression with 2 inputs and with interaction terms ...
0 votes
1 answer
25 views

Regression Model diverging after adding a new feature with higher variance and magnitude

In a time series regression problem I'm predicting "change" rather than the actual intended value i.e Instead of: ...
3 votes
0 answers
100 views

How can i tinker my neural network to learn stronger on rare events?

I am training a neural network on a regression problem. Most of the time the actual y (label) has the same value (say ~0.2) and only in rare cases the actual y is very different (say 2.0 or -2.0) ...
0 votes
3 answers
152 views

What ML algorithm should I use that suits this data?

What if I have some data, let's say I'm trying to answer if education level and IQ affect earnings, and I want to analyze this data and put in a regression model to predict earnings based on the IQ ...
2 votes
2 answers
148 views

Why is a simple regression problem so hard for an MLP to learn?

Consider a very simple problem, which is to find the maximum value out of a list of 5 numbers between 0 and 1. This is obviously trivial, but serves as a good example for a real-world problem I'm ...
0 votes
0 answers
52 views

Basic question about gradient for nominal regression

Say that we want to binary-classify images using a sigmoid function with the entropy-loss function. This algorithm is quite slow. The sigmoid function is: I find that this could be traced to the $L(y,...
0 votes
2 answers
519 views

Possible model to use to find pixel locations of objects

I want to make a model that outputs the centre pixel of objects appearing in an image. My current method involves using a CNN with L2 loss to output an image of equivalent size to the input where ...
1 vote
0 answers
30 views

bad prediction when having noise on the data: LSTM time-series regression

I want to predict the force plate using a smart insole using the LSTM model for time series prediction. the data on the force plate has positive and negative values (I think the resulting positive ...
0 votes
1 answer
48 views

Multi-layer network only predicts linear trends

I have made a neural network from scratch (in java), which is refusing to switch out of linear regression. I have pushed up the layer sizes (it now has 2 hidden layers, both with 5 neurons), and yet ...
1 vote
0 answers
379 views

Out of distribution detection (OOD) in the context of regression problems

I'm working in a regression setting to predict a scalar value $y$ from an input $\textbf{x} \in \mathbb{R}^D$ and I'm interested in understanding whenever my model is fed with something that it is ...
1 vote
1 answer
47 views

How do I interpret this loss function?

In this AI note from https://deeplearning.ai, the loss function below is used for a regression problem. However, I don't know how to interpret this loss function. First, does the author take the ...
1 vote
0 answers
275 views

Alternatives to Bayesian optimization

I am given a dataset $\mathcal{D} = \{\mathbf{x}_i\}_{i=1}^n$ and I need to find the point (in my case a material) $\mathbf{x}^*$ that maximizes a property $y$ (which can be obtained from a black-box ...
1 vote
0 answers
190 views

Active Learning regression with Random Forest

I have a dataset of about 8k points and I am trying to employ active learning with the random forest regressor. I have split the dataset to train and ...
1 vote
0 answers
23 views

How can evaluate the success of my algoritm?

A little bit of context. I have a classification algorithm based on mathematical discriminator and I am not applying any machine learning or AI technique, just moving window and several relative ...
0 votes
1 answer
145 views

Entirely linear neural network learning non-linear function

I have a neural network that's trained on a sine wave. It uses a lookback of 20 to see what the last 20 predictions were and predict the next value. This network has only a single Linear layer (input ...
0 votes
1 answer
59 views

How do I know if my Random Forest Regressor Model is overfitted?

Im creating a Random Forest Regressor Model with a small dataset (30 data points). I tried with other models but RF was the best one, however, after applying GridSearchCv I got that the training set ...
0 votes
1 answer
815 views

Is there any way to train a regression model with negative values that is more stable?

I have a regression model where my target values contain roughly 60% negative values and 40% positive values. My model architecture includes a robert-large, 1 linear layer. I trained it after 1 epoch, ...
0 votes
0 answers
20 views

Which existing model could be used for wind speed and direction prediction?

I am trying to predict the wind speed and wind direction in a graph network for a geographical area. The dataset includes the start and end nodes, the distance between them, and wind speed and ...
2 votes
1 answer
183 views

Is it a good idea to train a CNN to detect the hydration value (percentage) in skin images and evaluate it with the MSE?

I have a large dataset of skin images, each one associated with a hydration value (percentage). Now I'm looking into predicting the hydration value from an image. My thinking: train a CNN on the ...
1 vote
1 answer
86 views

Prediction of continuous variable based on threshold

The independent variables are date, count, atmp, and ...
0 votes
1 answer
399 views

How to make a proper approximation of Sine function with Neural Networks?

TL;DR; How to build a neural network that properly approximates the sine function with different ranges? Context and Question: From this question I decided to use the Sergey's answer, however I used a ...
3 votes
1 answer
106 views

Why does my regression-NN completely fail to predict some points?

I would like to train a NN in order to approximate an unknown function $y = f(x_1,x_2)$. I have a lot of measurements $y = [y_1,\dots,y_K]$ (with K that could be in the range of 10-100 thousands) ...