Questions tagged [regression]
For questions related to regression (both linear and non-linear) in the context of machine learning and AI.
134
questions
0
votes
0
answers
8
views
What is the best architecture for multi-target text regression?
I'm building an AI model using Google's 'Civil-Comments' dataset. It has 7 different labels, each a float than can be anywhere from 0 to 1. Embedding Bags, which I have read about. do not perform well....
0
votes
0
answers
33
views
Predict more elements than the input
I can use any machine learning algorithms (but neural networks are better for me) to resolve this issue: use few elements as input (numerical) to predict more elements as output. In normal regression ...
- 101
0
votes
0
answers
19
views
What to do after cross-validation?
After using cross-validation to see how a custom predictive function performs on unseen data, I applied to function to the original dataset, and the performance (based on coefficient of determination) ...
- 1
0
votes
1
answer
26
views
Regression model training improvement
I am fairly new to TensorFlow and ML in general and am currently working on a regression neural network while learning about different parts and concepts of it. My goal is to try & achieve a model ...
- 101
0
votes
0
answers
13
views
Apply weight to one feature based on another one for training a regression model
I have 1000 items that have a numerical feature y, the ground truth that I want to predict.
Each item has another feature c that ...
- 101
0
votes
1
answer
50
views
Is it possible to predict the next number in a stream that comes from a known PDF?
Lets say we have an arbitrarily large stream of numbers, numbers ranging from 1 to 100. You know these numbers follow a known distribution, e.g exponential distribution.
Is it possible predict the ...
- 11
0
votes
0
answers
58
views
Graph-Level Regression Task
I'm currently working on a system that predicts energy consumption of a set of buildings using graph convolutionals networks (GCN), which is a Graph-Level regression task (1 prediction for every ...
- 1
0
votes
1
answer
37
views
CNN multioutput regression architecture modification
I am working on a regression task where the model has to predict two values at the same time. The idea is that the dataset consists of 16 features, where the first 8 features represent the first value ...
0
votes
0
answers
46
views
Find the relationship between data in this plot
Attached image. How would you find the relationship between independent variable (x) and dependent variable (y)? Is it linear or non-linear? What would the function looks like?
P.S. I believe this is ...
0
votes
1
answer
36
views
Circular regression for joystick movements?
I've been playing around with some behavioral cloning of a simple old game that uses a joystick. As with behavioral cloning in general, if I record many games, then for each state there are many ...
- 121
1
vote
0
answers
35
views
Achieving low train error for exponential response?
I'm trying to fit an ML model with perfect information to predict an exponentially distributed response without getting exponentially distributed error...
Also, this situation is special for a few of ...
- 434
0
votes
1
answer
46
views
chained linear regression models vs feed forward NN
I am trying to understand the difference between feedforward NN and chained linear regression models, if and why they can model nonlinear functions.
both are able to model non-linear dependencies
...
1
vote
2
answers
78
views
Is MSE cost for a linear regression model a convex function with one global optimum?
Here is the thing: MSE cost for a linear regression model is a convex function with one global optimum, and it can be solved efficiently using gradient descent or in closed form (SVD, normal eq. ....)...
- 89
0
votes
1
answer
46
views
What are the consequences when we multiply, instead of add, a penalty term?
The typical objective function in regression problems like Lasso or Ridge includes a Residual Sum of Squares (RSS) term added to a penalty term based on a norm of the coefficients.
What are the ...
- 113
3
votes
1
answer
83
views
Regression loss conditioned by the ground-truth values
I'm working on a regression problem with a CNN in which the input is a single image, and the output is an angle in degrees (which determines a specific measure related to the image).
Sometimes, the ...
- 53
1
vote
1
answer
47
views
What number classes makes a classification problem continuous
I am working on a classification problem, where I have sequences of images and I want to train a model to predict the index of the image with some wanted property.
The target classes would obviously ...
- 43
0
votes
0
answers
27
views
How to make a RandomForestRegressor learn to differentiatie similar inputs with different outputs?
I'm working on a regression task with Sklearn RandomForestRegressor and I'm having some trouble distinguishing between two similar data with very different expected outputs. For example, each pair of ...
0
votes
0
answers
65
views
Does the accuracy of a regression learner depend on the way we feed data?
Consider a plot of points as such:
As one notices, this looks like an alternating sequence. Further, it can be divided into two subsequences as $a_{\text{odd}}$ and $a_{\text{even}}$ as they seem to ...
- 101
0
votes
0
answers
39
views
Pixel-wise regression only focus on edge
I am trying to use unet to learn pixel-wise regression from one image to one groundtruth with the same image size. The network seems to focus too much on the edge of the image, and it does not learn ...
- 1
1
vote
1
answer
46
views
CNN-Regression insensitive to input data
I'm currently training a CNN + multiple target regression model that does the following
input: $ \dim x = (L, 2), \text{where} \ x_i \in (-0.1, 0.1) $
output: $\dim y = (M), \text{where} \ y_i \geq ...
- 11
0
votes
1
answer
47
views
How can a Regression based Neural Network learn class thresholds?
I understand that to solve multilabel classification problems, we can use the softmax activation function in the output layer of the neural network. The softmax function outputs probabilities of each ...
- 43
0
votes
1
answer
473
views
How to do backpropagation with argmax?
I am attempting to utilize two networks: a classifier and a linear network. Based on the output class of the first network, my goal is to retrieve the corresponding value from the linear network using ...
3
votes
1
answer
4k
views
Is it possible to use LLMs for regression tasks?
I want to use LLMs to predict edge weights in a graph based on attributes between two nodes. Is this even possible? If not, what would you recommend?
I tried to look up uses of LLM in regression tasks,...
0
votes
1
answer
26
views
Regression Model diverging after adding a new feature with higher variance and magnitude
In a time series regression problem I'm predicting "change" rather than the actual intended value i.e
Instead of:
...
3
votes
0
answers
134
views
How can i tinker my neural network to learn stronger on rare events?
I am training a neural network on a regression problem.
Most of the time the actual y (label) has the same value (say ~0.2) and only in rare cases the actual y is very different (say 2.0 or -2.0)
...
- 31
0
votes
1
answer
39
views
why by adding additional information as number of sequence on dataset can avoid overfitting
I am developing a regression model to analyze walking styles. The dataset I am using to build the model is from 2 different sources, let's call them dataset A and dataset B.
Dataset A has a shape of <...
- 11
2
votes
2
answers
159
views
Why is a simple regression problem so hard for an MLP to learn?
Consider a very simple problem, which is to find the maximum value out of a list of 5 numbers between 0 and 1. This is obviously trivial, but serves as a good example for a real-world problem I'm ...
- 201
2
votes
1
answer
61
views
Is there any interpretation method suitable for CNNs which do regression tasks?
I mainly tackle regression problems by CNNs, and want to find a reliable method to calculate the heatmaps for NN's results. However, I find almost all interpretation methods including CAM is used for ...
- 21
1
vote
0
answers
35
views
bad prediction when having noise on the data: LSTM time-series regression
I want to predict the force plate using a smart insole using the LSTM model for time series prediction. the data on the force plate has positive and negative values (I think the resulting positive ...
- 11
0
votes
1
answer
828
views
Why does GridSearchCV model give worse results despite same parameters used with base model
I am trying to make prediction using random forest regression and then utilize GridSearchCV to tune hyperparameters(just 'n_estimators'). However results of GridSearchCV are worse than base model. ...
0
votes
1
answer
48
views
Multi-layer network only predicts linear trends
I have made a neural network from scratch (in java), which is refusing to switch out of linear regression. I have pushed up the layer sizes (it now has 2 hidden layers, both with 5 neurons), and yet ...
- 113
1
vote
0
answers
473
views
Out of distribution detection (OOD) in the context of regression problems
I'm working in a regression setting to predict a scalar value $y$ from an input $\textbf{x} \in \mathbb{R}^D$ and I'm interested in understanding whenever my model is fed with something that it is ...
- 307
1
vote
1
answer
51
views
How do I interpret this loss function?
In this AI note from https://deeplearning.ai, the loss function below is used for a regression problem. However, I don't know how to interpret this loss function.
First, does the author take the ...
- 113
0
votes
0
answers
53
views
Basic question about gradient for nominal regression
Say that we want to binary-classify images using a sigmoid function with the entropy-loss function. This algorithm is quite slow.
The sigmoid function is:
I find that this could be traced to the $L(y,...
user57690
1
vote
0
answers
332
views
Alternatives to Bayesian optimization
I am given a dataset $\mathcal{D} = \{\mathbf{x}_i\}_{i=1}^n$ and I need to find the point (in my case a material) $\mathbf{x}^*$ that maximizes a property $y$ (which can be obtained from a black-box ...
- 221
1
vote
0
answers
211
views
Active Learning regression with Random Forest
I have a dataset of about 8k points and I am trying to employ active learning with the random forest regressor. I have split the dataset to train and ...
- 221
1
vote
0
answers
23
views
How can evaluate the success of my algoritm?
A little bit of context. I have a classification algorithm based on mathematical discriminator and I am not applying any machine learning or AI technique, just moving window and several relative ...
- 123
1
vote
2
answers
51
views
Dealing with incomplete file sets for a CNN for medical imaging regression problem
I'm trying to solve a medical imaging regression problem using a CNN. Each of the samples in my data set consists of one, two, or three of the following file types:
flair.nii.gz
mprage.nii.gz
swi....
- 125
0
votes
1
answer
160
views
Entirely linear neural network learning non-linear function
I have a neural network that's trained on a sine wave. It uses a lookback of 20 to see what the last 20 predictions were and predict the next value. This network has only a single Linear layer (input ...
- 1,406
0
votes
1
answer
102
views
How do I know if my Random Forest Regressor Model is overfitted?
Im creating a Random Forest Regressor Model with a small dataset (30 data points). I tried with other models but RF was the best one, however, after applying GridSearchCv I got that the training set ...
- 3
0
votes
1
answer
1k
views
Is there any way to train a regression model with negative values that is more stable?
I have a regression model where my target values contain roughly 60% negative values and 40% positive values. My model architecture includes a robert-large, 1 linear layer. I trained it after 1 epoch, ...
0
votes
0
answers
20
views
Which existing model could be used for wind speed and direction prediction?
I am trying to predict the wind speed and wind direction in a graph network for a geographical area. The dataset includes the start and end nodes, the distance between them, and wind speed and ...
- 1
1
vote
1
answer
101
views
Prediction of continuous variable based on threshold
The independent variables are date, count, atmp, and ...
- 111
0
votes
1
answer
454
views
How to make a proper approximation of Sine function with Neural Networks?
TL;DR;
How to build a neural network that properly approximates the sine function with different ranges?
Context and Question:
From this question I decided to use the Sergey's answer, however I used a ...
- 111
3
votes
1
answer
139
views
Why does my regression-NN completely fail to predict some points?
I would like to train a NN in order to approximate an unknown function $y = f(x_1,x_2)$. I have a lot of measurements $y = [y_1,\dots,y_K]$ (with K that could be in the range of 10-100 thousands) ...
- 31
0
votes
1
answer
1k
views
What's the best model to use for CNN(deep learning) regression task for small image dataset?
What are the best Deep learning models(with how many layers) to use in a regression task for a custom dataset containing around 100 images of only one object per image which is more or less ...
- 1
1
vote
1
answer
89
views
Does a second-order fully-connected layer have any uses?
I was thinking about implementing second-order regression via a fully-connected layer, and I came up with this:
$X$ is the input data, shaped $(features, batch\_number)$.
$w0$ is the bias, shaped $(...
- 113
-1
votes
1
answer
63
views
Is my dataset a time series dataset? and should I use an LSTM?
I have a dataset where I am recording temperature after every 4milliseconds till 500 and another feature "conductivity value". The length of the dataset is around a 1000 rows. I need to find ...
- 51
0
votes
0
answers
15
views
What to predict in a limited transaction dataset?
I have been given a task with a real transaction dataset. The task is to predict something using either logistic regression or simple binary classification.
The columns are as follow:
Transaction ID
...
- 31
1
vote
0
answers
77
views
Is the VC dimension of a MLP regressor a valid upper bound on how many points it can exactly fit?
I want to calculate an upper bound on how many training points an MLP regressor can fit with ~0 error. I don't care about the test error, I want to overfit as much as possible the (few) training ...
- 11