Questions tagged [regression]

For questions related to regression (both linear and non-linear) in the context of machine learning and AI.

Filter by
Sorted by
Tagged with
0 votes
1 answer
34 views

What are the consequences when we multiply, instead of add, a penalty term?

The typical objective function in regression problems like Lasso or Ridge includes a Residual Sum of Squares (RSS) term added to a penalty term based on a norm of the coefficients. What are the ...
BigMistake's user avatar
3 votes
1 answer
51 views

Regression loss conditioned by the ground-truth values

I'm working on a regression problem with a CNN in which the input is a single image, and the output is an angle in degrees (which determines a specific measure related to the image). Sometimes, the ...
Cezoz08's user avatar
  • 43
1 vote
1 answer
40 views

What number classes makes a classification problem continuous

I am working on a classification problem, where I have sequences of images and I want to train a model to predict the index of the image with some wanted property. The target classes would obviously ...
mavex857's user avatar
0 votes
0 answers
25 views

How to make a RandomForestRegressor learn to differentiatie similar inputs with different outputs?

I'm working on a regression task with Sklearn RandomForestRegressor and I'm having some trouble distinguishing between two similar data with very different expected outputs. For example, each pair of ...
Luís Henrique Bandória's user avatar
0 votes
0 answers
65 views

Does the accuracy of a regression learner depend on the way we feed data?

Consider a plot of points as such: As one notices, this looks like an alternating sequence. Further, it can be divided into two subsequences as $a_{\text{odd}}$ and $a_{\text{even}}$ as they seem to ...
DatBoi's user avatar
  • 101
0 votes
0 answers
25 views

Pixel-wise regression only focus on edge

I am trying to use unet to learn pixel-wise regression from one image to one groundtruth with the same image size. The network seems to focus too much on the edge of the image, and it does not learn ...
K.Nguyen's user avatar
0 votes
0 answers
16 views

Sparse linear discriminant analysis for regression problem?

So far, Linear Discriminant Analysis has beed used for classification problems http://proceedings.mlr.press/v38/wu15.pdf . I wonder if there are any ways to adapt it to regression problems?
PT_98's user avatar
  • 1
1 vote
1 answer
34 views

CNN-Regression insensitive to input data

I'm currently training a CNN + multiple target regression model that does the following input: $ \dim x = (L, 2), \text{where} \ x_i \in (-0.1, 0.1) $ output: $\dim y = (M), \text{where} \ y_i \geq ...
RLLL's user avatar
  • 11
0 votes
1 answer
40 views

How can a Regression based Neural Network learn class thresholds?

I understand that to solve multilabel classification problems, we can use the softmax activation function in the output layer of the neural network. The softmax function outputs probabilities of each ...
Dawood Ahmad's user avatar
0 votes
1 answer
198 views

How to do backpropagation with argmax?

I am attempting to utilize two networks: a classifier and a linear network. Based on the output class of the first network, my goal is to retrieve the corresponding value from the linear network using ...
Subrat Prasad's user avatar
0 votes
0 answers
11 views

Label transformation vs Methods in Imblanced Regression for Imbalanced Regression tasks

I've seen some papers discussing the imbalanced regression recently and was wondering what's the benefit of this line of approaches compared to conventional data transformations (e.g., Square-root, ...
Rowing0914's user avatar
2 votes
1 answer
1k views

Is it possible to use LLMs for regression tasks?

I want to use LLMs to predict edge weights in a graph based on attributes between two nodes. Is this even possible? If not, what would you recommend? I tried to look up uses of LLM in regression tasks,...
sharkeater123's user avatar
0 votes
0 answers
15 views

Regression Model overestimates in train-mode

I have a Deep Learning Regression model to predict some values. The results are fine when I use the model in Evaluation Mode, but when I turn Training Mode on the model tends to overestimate the ...
nmb's user avatar
  • 1
0 votes
0 answers
19 views

Confused about interaction terms in polynomial regression

I am trying to code multivariate polynomial regression from scratch and I got confused about how interaction terms work. I saw that a polynomial regression with 2 inputs and with interaction terms ...
Vladislav Korecký's user avatar
0 votes
1 answer
24 views

Regression Model diverging after adding a new feature with higher variance and magnitude

In a time series regression problem I'm predicting "change" rather than the actual intended value i.e Instead of: ...
Darren Rahnemoon's user avatar
3 votes
0 answers
76 views

How can i tinker my neural network to learn stronger on rare events?

I am training a neural network on a regression problem. Most of the time the actual y (label) has the same value (say ~0.2) and only in rare cases the actual y is very different (say 2.0 or -2.0) ...
Carl Philip's user avatar
0 votes
0 answers
34 views

Using MSE or RMSE instead of CrossEntropy in Question Answering NLP problems. What are the problems if we used?

When you predict Start Index, end Index in Question Answering NLP task (SQUAD Data), you use CrossEntropy as a loss function. ...
Deshwal's user avatar
  • 253
0 votes
1 answer
30 views

why by adding additional information as number of sequence on dataset can avoid overfitting

I am developing a regression model to analyze walking styles. The dataset I am using to build the model is from 2 different sources, let's call them dataset A and dataset B. Dataset A has a shape of <...
stack offer's user avatar
0 votes
0 answers
15 views

Regressing parameters that map between two curves machine learning

I am wondering if anyone has experience in regressing out parameters that map one curve to another. For example, I have two curves that look like this. I used some non-linear equation to map orange to ...
user2551700's user avatar
0 votes
0 answers
19 views

What happens if one uses non-valid kernels in regression?

When introducing kernelization in regression, it is emphasized that a kernel function $k(x_i, x_j)$ has to represent a scalar product in some high dimensional feature space, $k(x_i, x_j) = \phi(x_i)^T ...
Botond's user avatar
  • 101
0 votes
0 answers
55 views

How to create a global forecasting model using deep learning?

I am aiming to build a global/general forecasting model (don't know what's the proper terminology) using deep learning. The idea behind this is to create a model trained on several time series that ...
David Díaz's user avatar
0 votes
0 answers
33 views

How to output a function given a time series data as an input using supervised learning?

I have a spreadsheet with time series data collected from two sensors, one measuring temperature and the other measuring humidity. And I also collected data from an experiment that I conducted, the ...
Kasiopea's user avatar
2 votes
2 answers
142 views

Why is a simple regression problem so hard for an MLP to learn?

Consider a very simple problem, which is to find the maximum value out of a list of 5 numbers between 0 and 1. This is obviously trivial, but serves as a good example for a real-world problem I'm ...
Daniel's user avatar
  • 191
0 votes
0 answers
34 views

Multi-output regression problems

I am training CNNs on 3D image data (dimensions [500, 512, 512]) to locate 7 3D points inside the image. I have thought of two different ways to solve this problem, ...
andytaylor823's user avatar
2 votes
1 answer
58 views

Is there any interpretation method suitable for CNNs which do regression tasks?

I mainly tackle regression problems by CNNs, and want to find a reliable method to calculate the heatmaps for NN's results. However, I find almost all interpretation methods including CAM is used for ...
minghuisvn's user avatar
0 votes
0 answers
19 views

What's a good regression algorithm for handling tabular data that have categorical data, "list of words"

Problem statement: I want to predict future prices of trips based on historical pricing data. I'm looking for an algorithm that has the following features: Unsupervised algorithm Limit the amount of ...
zzzz8888's user avatar
1 vote
0 answers
28 views

bad prediction when having noise on the data: LSTM time-series regression

I want to predict the force plate using a smart insole using the LSTM model for time series prediction. the data on the force plate has positive and negative values (I think the resulting positive ...
stack offer's user avatar
0 votes
1 answer
443 views

Why does GridSearchCV model give worse results despite same parameters used with base model

I am trying to make prediction using random forest regression and then utilize GridSearchCV to tune hyperparameters(just 'n_estimators'). However results of GridSearchCV are worse than base model. ...
dancineer's user avatar
0 votes
1 answer
48 views

Multi-layer network only predicts linear trends

I have made a neural network from scratch (in java), which is refusing to switch out of linear regression. I have pushed up the layer sizes (it now has 2 hidden layers, both with 5 neurons), and yet ...
Gamaray's user avatar
  • 13
1 vote
0 answers
336 views

Out of distribution detection (OOD) in the context of regression problems

I'm working in a regression setting to predict a scalar value $y$ from an input $\textbf{x} \in \mathbb{R}^D$ and I'm interested in understanding whenever my model is fed with something that it is ...
James Arten's user avatar
1 vote
1 answer
42 views

How do I interpret this loss function?

In this AI note from https://deeplearning.ai, the loss function below is used for a regression problem. However, I don't know how to interpret this loss function. First, does the author take the ...
Zarif's user avatar
  • 113
0 votes
0 answers
52 views

Basic question about gradient for nominal regression

Say that we want to binary-classify images using a sigmoid function with the entropy-loss function. This algorithm is quite slow. The sigmoid function is: I find that this could be traced to the $L(y,...
Mah Neh's user avatar
  • 79
1 vote
0 answers
241 views

Alternatives to Bayesian optimization

I am given a dataset $\mathcal{D} = \{\mathbf{x}_i\}_{i=1}^n$ and I need to find the point (in my case a material) $\mathbf{x}^*$ that maximizes a property $y$ (which can be obtained from a black-box ...
ado sar's user avatar
  • 150
1 vote
0 answers
165 views

Active Learning regression with Random Forest

I have a dataset of about 8k points and I am trying to employ active learning with the random forest regressor. I have split the dataset to train and ...
ado sar's user avatar
  • 150
1 vote
0 answers
22 views

How can evaluate the success of my algoritm?

A little bit of context. I have a classification algorithm based on mathematical discriminator and I am not applying any machine learning or AI technique, just moving window and several relative ...
GGChe's user avatar
  • 123
1 vote
2 answers
38 views

Dealing with incomplete file sets for a CNN for medical imaging regression problem

I'm trying to solve a medical imaging regression problem using a CNN. Each of the samples in my data set consists of one, two, or three of the following file types: flair.nii.gz mprage.nii.gz swi....
Paul Reiners's user avatar
0 votes
1 answer
130 views

Entirely linear neural network learning non-linear function

I have a neural network that's trained on a sine wave. It uses a lookback of 20 to see what the last 20 predictions were and predict the next value. This network has only a single Linear layer (input ...
Recessive's user avatar
  • 1,346
0 votes
1 answer
47 views

How do I know if my Random Forest Regressor Model is overfitted?

Im creating a Random Forest Regressor Model with a small dataset (30 data points). I tried with other models but RF was the best one, however, after applying GridSearchCv I got that the training set ...
Gaby's user avatar
  • 3
0 votes
1 answer
670 views

Is there any way to train a regression model with negative values that is more stable?

I have a regression model where my target values contain roughly 60% negative values and 40% positive values. My model architecture includes a robert-large, 1 linear layer. I trained it after 1 epoch, ...
Việt Nguyễn's user avatar
0 votes
0 answers
19 views

Which existing model could be used for wind speed and direction prediction?

I am trying to predict the wind speed and wind direction in a graph network for a geographical area. The dataset includes the start and end nodes, the distance between them, and wind speed and ...
bsha's user avatar
  • 1
1 vote
1 answer
80 views

Prediction of continuous variable based on threshold

The independent variables are date, count, atmp, and ...
There's user avatar
  • 111
0 votes
1 answer
353 views

How to make a proper approximation of Sine function with Neural Networks?

TL;DR; How to build a neural network that properly approximates the sine function with different ranges? Context and Question: From this question I decided to use the Sergey's answer, however I used a ...
Hans's user avatar
  • 111
3 votes
1 answer
99 views

Why does my regression-NN completely fail to predict some points?

I would like to train a NN in order to approximate an unknown function $y = f(x_1,x_2)$. I have a lot of measurements $y = [y_1,\dots,y_K]$ (with K that could be in the range of 10-100 thousands) ...
MttRch's user avatar
  • 31
0 votes
1 answer
810 views

What's the best model to use for CNN(deep learning) regression task for small image dataset?

What are the best Deep learning models(with how many layers) to use in a regression task for a custom dataset containing around 100 images of only one object per image which is more or less ...
Sevcenko's user avatar
1 vote
1 answer
82 views

Does a second-order fully-connected layer have any uses?

I was thinking about implementing second-order regression via a fully-connected layer, and I came up with this: $X$ is the input data, shaped $(features, batch\_number)$. $w0$ is the bias, shaped $(...
HappyFace's user avatar
  • 113
-1 votes
1 answer
61 views

Is my dataset a time series dataset? and should I use an LSTM?

I have a dataset where I am recording temperature after every 4milliseconds till 500 and another feature "conductivity value". The length of the dataset is around a 1000 rows. I need to find ...
Araib karim's user avatar
0 votes
0 answers
15 views

What to predict in a limited transaction dataset?

I have been given a task with a real transaction dataset. The task is to predict something using either logistic regression or simple binary classification. The columns are as follow: Transaction ID ...
Rami Hoteit's user avatar
1 vote
0 answers
66 views

Is the VC dimension of a MLP regressor a valid upper bound on how many points it can exactly fit?

I want to calculate an upper bound on how many training points an MLP regressor can fit with ~0 error. I don't care about the test error, I want to overfit as much as possible the (few) training ...
Daniele 's user avatar
0 votes
1 answer
115 views

Do I need to tune the hyper-parameters or more data if SVR model performs poorly?

I am using non-linear data to SVR and have tried tuning the hyperparameters and still have a poor model performance. Do I need more data or format the data for more suitable results? I get similar ...
Taqi Ahmed's user avatar
1 vote
0 answers
14 views

What is the best way to train a text-based regressor model?

I want to build a deep learning model that can predict a continuous value (LogP in this case) given text inputs (SMILES notations in this case), the dataset is as illustrated below. SMILES notations ...
mac179's user avatar
  • 121