Questions tagged [normalisation]
The normalisation tag has no usage guidance.
35
questions
0
votes
1
answer
34
views
What are the consequences when we multiply, instead of add, a penalty term?
The typical objective function in regression problems like Lasso or Ridge includes a Residual Sum of Squares (RSS) term added to a penalty term based on a norm of the coefficients.
What are the ...
- 113
-2
votes
0
answers
16
views
Normalization in 1NF, 2NF and 3NF
I have a table and I want to create db and tables for normalization in 1NF, 2NF and 3NF
based on below table :
"
librarian_fname librarian_surname shift day assignment
Jesse ...
- 1
0
votes
0
answers
26
views
In multilayer perceptron neural networks, is normalization required? or is optional? Why the network only seem to work when I use normalization?
I followed the steps in the following article (which teaches how to program a multilayer perceptron neural network from scratch in Python): https://machinelearningmastery.com/implement-backpropagation-...
- 233
0
votes
2
answers
33
views
Must I "prime" my normalizer with the same data I trained it with in order to use it?
I trained a Keras Network. During training, I would first initialize a normalizer from the values in the entire dataset, then partition into train, test and validation datasets. After partitioning, I ...
- 149
0
votes
0
answers
16
views
FastGAN implementation has redundant SpectralNorm followed by BatchNorm?
I am implementing a version of FastGAN, and it seems like (see the official repo) they use a Spectral norm directly followed by a Batch norm. Doesn't the spectral norm get fully cancelled by the ...
- 21
1
vote
1
answer
35
views
What's $\mathbb{V}[\gamma]$ in Return-based Scaling: Yet Another Normalisation Trick for Deep RL?
In the paper Return-based Scaling: Yet Another Normalisation Trick for Deep RL,
$\sigma$, the scaling factor to normalize the TD error, is computed as:
$\sigma^2 = \mathbb{V}[R] + \mathbb{V}[\gamma]\...
- 165
0
votes
1
answer
24
views
Should I apply a min-max scale (range 0 to 1) before applying the normalisation or should I apply the z-score normalisation directly?
I want to implement a neural network in Pytorch for medical image segmentation. I should normalise my data.
Should I apply a min-max scale (range 0 to 1) before applying the normalisation or should I ...
- 101
0
votes
0
answers
16
views
Normalisation in feature extraction using pre-trained model
I have a dataset with medical images. I want to implement a network for super-resolution using GANs. One of the criteria of the optimisation is a perceptual loss. For that I will use a pretrained vgg ...
- 101
0
votes
1
answer
50
views
Can we model the global statistics of the dataset features using LayerNorm?
When applying layer normalization in a neural network, we consider the mean and variance of each input data point to the layer separately without considering the global statistics in the entire data-...
0
votes
1
answer
313
views
In the attention mechanism, why don't we normalize after multiplying values?
As this question says:
In scaled dot product attention, we scale our outputs by dividing the
dot product by the square root of the dimensionality of the matrix:
The reason why is stated that this ...
- 554
0
votes
0
answers
9
views
Why is avoiding normalized models a practical solution for reducing the complexity in NNLM?
In the paper Efficient Estimation of Word Representations in Vector Space, the authors say that "avoiding normalized models completely by using models that are not normalized during training"...
- 1
0
votes
0
answers
13
views
Drastic Change in MSE when Scaling Factor Changes
I am training an autoencoder meant to detect anomalies. Initially I scaled my data using a min-max scaler. I realized that this scalar isn't the best because anomalies can cause bias in the scalar. ...
- 113
0
votes
1
answer
109
views
What does linear regime of nonlinearity mean in normalisation?
In section 3 paragraph 2 of Batch Normalization: Accelerating Deep Network Training b y Reducing Internal Covariate Shift paper (https://arxiv.org/abs/1502.03167) they say that normalizing a layer's ...
- 21
3
votes
2
answers
87
views
What if each sample was normalized on its own before sending them to the neural network?
The standard method is to normalize the entire dataset (the training part) then send it to the model to train on. However I’ve noticed that in this manner the model doesn’t really work well when ...
- 31
0
votes
1
answer
84
views
During batch normalization is the mini-batch gone through twice, one to calculate the mean and variance and then to normalize them?
I am asking this question because while designing my own model, I had repeated gradient explosion issues, so I wanted to try batch normalization. I really want to understand the details and math ...
2
votes
3
answers
1k
views
Do I need to normalize all state-space variables? If so, how?
I am playing around with a DRL agent in a stock-trading environment.
I have normalized all the external input data (the features that my agent will use). However, what about characteristics that don't ...
- 342
1
vote
0
answers
95
views
Normalisation of reward function
Problem
Currently, I have some problems defining a reward function for my RL project and mainly with how to normalise the score such that the highest possible score for all instances of the ...
- 31
0
votes
1
answer
68
views
Which generalization of standard deviation to use for multidimensional input normalization
For machine learning tasks, it's common to normalize input data by subtracting the mean $\mu$ and dividing by the standard deviation $\sigma$ of the dataset:
$$\hat{x_i} = \frac{x_i - \mu}{\sigma}$$
—...
- 103
3
votes
2
answers
5k
views
Should I apply normalization to the observations in deep reinforcement learning?
I am new to DRL and trying to implement my custom environment. I want to know if normalization and regularization techniques are as important in RL as in Deep Learning.
In my custom environment, the ...
- 31
1
vote
0
answers
29
views
How to choose proper normalization strategy for the activations?
I am reading a survey on various normalization techniques adopted in neural network architectures.
The purpose of introducing normalization is understandable - to stabilize the training and avoid ...
- 2,564
1
vote
1
answer
58
views
Does it make sense to apply batch normalization to a batch size of 1?
I am interested in your opinion on the topic if you think that it makes sense to use batch normalization layer in a network that is trained with a batch size of 1. This is a special case as part of an ...
- 273
0
votes
1
answer
35
views
Normalization of possibly not fully representative data
I am trying to train a classification RNN model on a sequence of table medical data, but I stuck with the normalization problem. I realized that I cannot simply use MinMaxScaler, because of 3 problems:...
- 111
1
vote
1
answer
590
views
How to scale all positive continuous reward?
My RL project has all positive continuous rewards for every step and the goal is to have the maximum cumulative reward (episodic reward). The problem is that the rewards are too close and all between ...
1
vote
0
answers
39
views
Why do you calculate the mean and standard deviation over the complete dataset before training rather than for every batch?
In most implementations of neural networks the features are scaled to make the optimization of the loss function as stable as possible.
Mostly a min-max scaler is used. Alternatively, there is also a ...
- 11
1
vote
0
answers
64
views
For image preprocessing, is it better to use normalization or standartization?
For a neural network model that classifies images, is it better to use normalization (dividing by 255.0) or using standardization (subtract mean and divide by STD)?
When I started learning ...
- 153
1
vote
1
answer
143
views
How to define a "don't care" class in time series classification in Pytorch?
This is a theoretical question.
Setup
I have a time series classification task in which I should output a classification of 3 classes for every time stamp t.
All ...
- 749
1
vote
1
answer
453
views
How to normalize images before training?
I have seen people normalize images by just dividing 255. But why? Why not use mean normalization or Z-score Normalization? I also came across this StackOverflow topic while searching but the answers ...
2
votes
1
answer
218
views
Is data leakage relevant when scaling across samples?
I have a question about data leakage when pre-processing data for a neural network and whether data leakage actually applies in my instance.
I have variance stabilising transformed genomic data. ...
- 161
2
votes
1
answer
355
views
Why does batch norm standardize with sample mean/variance, when it also learns parameters to scale the mean/variance?
Batch norm is a normalizing layer that is shown to help deep networks learn faster and with higher generalization accuracy. It normalizes the activations of the previous layer to a mean $\beta$ and ...
- 1,510
0
votes
1
answer
447
views
Flatten image using Neural network and matrix transpose
I have read a lecture note of Prof. Andrew Ng. There was something about data normalization like how can we flatten an image of (64x64x3) into a (64x64x3)*x1 vector. After that there is pictorial ...
- 125
1
vote
2
answers
133
views
What would be a typical pre-processing and data normalization pipeline for time series data (for non-linear models such as neural networks)?
I've started to work on time series. I was wondering what would be the best data normalizing and pre-processing technique for non-linear models, specifically, neural networks.
One I can think of is ...
- 290
-1
votes
1
answer
261
views
Is it necessary to standardise the expected output
Normalisation transform data into a range:
$$X_i = \dfrac{X_i - Min}{Max-Min}$$
Practically, I found out that the model doesn't generalise well when using normalisation of input data, instead of ...
- 1,283
2
votes
1
answer
171
views
Do I need to denormalise results in linear regression?
I have learned so far how to linear regression with one or multiple features. So far, so good, everything seems to work fine, at least for my first simple examples.
However, I now need to normalise my ...
- 173
8
votes
0
answers
118
views
Normalizing Normal Distributions in Thompson Sampling for online Reinforcement Learning
In my implementation of Thompson Sampling (TS) for online Reinforcement Learning, my distribution for selecting $a$ is $\mathcal{N}(Q(s, a), \frac{1}{C(s,a)+1})$, where $C(s,a)$ is the number of times ...
- 81
2
votes
1
answer
579
views
Could the normalisation of the inputs make the neural network insensitive to changes in the inputs?
When using neural networks (NNs), we often normalized the inputs. I think this is done to equally capture the changes in any input feature, that is, if any feature takes huge values and other features ...
- 191