Questions tagged [normalisation]
The normalisation tag has no usage guidance.
27
questions
0
votes
0
answers
8
views
Why is avoiding normalized models a practical solution for reducing the complexity in NNLM?
In the paper Efficient Estimation of Word Representations in Vector Space, the authors say that "avoiding normalized models completely by using models that are not normalized during training"...
- 1
0
votes
0
answers
7
views
Drastic Change in MSE when Scaling Factor Changes
I am training an autoencoder meant to detect anomalies. Initially I scaled my data using a min-max scaler. I realized that this scalar isn't the best because anomalies can cause bias in the scalar. ...
- 113
0
votes
1
answer
38
views
What does linear regime of nonlinearity mean in normalisation?
In section 3 paragraph 2 of Batch Normalization: Accelerating Deep Network Training b y Reducing Internal Covariate Shift paper (https://arxiv.org/abs/1502.03167) they say that normalizing a layer's ...
- 21
3
votes
2
answers
57
views
What if each sample was normalized on its own before sending them to the neural network?
The standard method is to normalize the entire dataset (the training part) then send it to the model to train on. However I’ve noticed that in this manner the model doesn’t really work well when ...
- 31
0
votes
1
answer
60
views
During batch normalization is the mini-batch gone through twice, one to calculate the mean and variance and then to normalize them?
I am asking this question because while designing my own model, I had repeated gradient explosion issues, so I wanted to try batch normalization. I really want to understand the details and math ...
0
votes
0
answers
41
views
CNN: Input Normalization for Time Series Data (Grad-CAM)
I trained a CNN model on univariate time series data. As I often read that it is advisable to normalize the data, e.g., using z-normalization, I started with training on z-normalized versions of the ...
0
votes
0
answers
24
views
What is the status of "local Response Normalization" in deep learning community?
In the benchmark papers of deep learning, authors generally contribute multiple novel techniques. Some techniques withstand time and some other techniques might fade out gradually although they seem ...
- 3,481
2
votes
3
answers
604
views
Do I need to normalize all state-space variables? If so, how?
I am playing around with a DRL agent in a stock-trading environment.
I have normalized all the external input data (the features that my agent will use). However, what about characteristics that don't ...
- 312
1
vote
0
answers
56
views
Normalisation of reward function
Problem
Currently, I have some problems defining a reward function for my RL project and mainly with how to normalise the score such that the highest possible score for all instances of the ...
- 31
0
votes
1
answer
57
views
Which generalization of standard deviation to use for multidimensional input normalization
For machine learning tasks, it's common to normalize input data by subtracting the mean $\mu$ and dividing by the standard deviation $\sigma$ of the dataset:
$$\hat{x_i} = \frac{x_i - \mu}{\sigma}$$
—...
- 103
2
votes
2
answers
3k
views
Should I apply normalization to the observations in deep reinforcement learning?
I am new to DRL and trying to implement my custom environment. I want to know if normalization and regularization techniques are as important in RL as in Deep Learning.
In my custom environment, the ...
- 21
1
vote
0
answers
27
views
How to choose proper normalization strategy for the activations?
I am reading a survey on various normalization techniques adopted in neural network architectures.
The purpose of introducing normalization is understandable - to stabilize the training and avoid ...
- 2,174
1
vote
0
answers
44
views
Does it make sense to apply batch normalization to a batch size of 1?
I am interested in your opinion on the topic if you think that it makes sense to use batch normalization layer in a network that is trained with a batch size of 1. This is a special case as part of an ...
- 273
0
votes
1
answer
34
views
Normalization of possibly not fully representative data
I am trying to train a classification RNN model on a sequence of table medical data, but I stuck with the normalization problem. I realized that I cannot simply use MinMaxScaler, because of 3 problems:...
- 111
1
vote
1
answer
454
views
How to scale all positive continuous reward?
My RL project has all positive continuous rewards for every step and the goal is to have the maximum cumulative reward (episodic reward). The problem is that the rewards are too close and all between ...
1
vote
0
answers
34
views
Why do you calculate the mean and standard deviation over the complete dataset before training rather than for every batch?
In most implementations of neural networks the features are scaled to make the optimization of the loss function as stable as possible.
Mostly a min-max scaler is used. Alternatively, there is also a ...
- 11
1
vote
0
answers
34
views
For image preprocessing, is it better to use normalization or standartization?
For a neural network model that classifies images, is it better to use normalization (dividing by 255.0) or using standardization (subtract mean and divide by STD)?
When I started learning ...
- 133
1
vote
1
answer
103
views
How to define a "don't care" class in time series classification in Pytorch?
This is a theoretical question.
Setup
I have a time series classification task in which I should output a classification of 3 classes for every time stamp t.
All ...
- 689
1
vote
1
answer
243
views
How to normalize images before training?
I have seen people normalize images by just dividing 255. But why? Why not use mean normalization or Z-score Normalization? I also came across this StackOverflow topic while searching but the answers ...
2
votes
1
answer
135
views
Is data leakage relevant when scaling across samples?
I have a question about data leakage when pre-processing data for a neural network and whether data leakage actually applies in my instance.
I have variance stabilising transformed genomic data. ...
- 151
2
votes
1
answer
188
views
Why does batch norm standardize with sample mean/variance, when it also learns parameters to scale the mean/variance?
Batch norm is a normalizing layer that is shown to help deep networks learn faster and with higher generalization accuracy. It normalizes the activations of the previous layer to a mean $\beta$ and ...
- 1,400
0
votes
1
answer
338
views
Flatten image using Neural network and matrix transpose
I have read a lecture note of Prof. Andrew Ng. There was something about data normalization like how can we flatten an image of (64x64x3) into a (64x64x3)*x1 vector. After that there is pictorial ...
- 125
0
votes
2
answers
114
views
What would be a typical pre-processing and data normalization pipeline for time series data (for non-linear models such as neural networks)?
I've started to work on time series. I was wondering what would be the best data normalizing and pre-processing technique for non-linear models, specifically, neural networks.
One I can think of is ...
- 240
-1
votes
1
answer
157
views
Is it necessary to standardise the expected output
Normalisation transform data into a range:
$$X_i = \dfrac{X_i - Min}{Max-Min}$$
Practically, I found out that the model doesn't generalise well when using normalisation of input data, instead of ...
- 1,283
2
votes
1
answer
144
views
Do I need to denormalise results in linear regression?
I have learned so far how to linear regression with one or multiple features. So far, so good, everything seems to work fine, at least for my first simple examples.
However, I now need to normalise my ...
- 173
7
votes
0
answers
99
views
Normalizing Normal Distributions in Thompson Sampling for online Reinforcement Learning
In my implementation of Thompson Sampling (TS) for online Reinforcement Learning, my distribution for selecting $a$ is $\mathcal{N}(Q(s, a), \frac{1}{C(s,a)+1})$, where $C(s,a)$ is the number of times ...
- 71
2
votes
1
answer
527
views
Could the normalisation of the inputs make the neural network insensitive to changes in the inputs?
When using neural networks (NNs), we often normalized the inputs. I think this is done to equally capture the changes in any input feature, that is, if any feature takes huge values and other features ...
- 191