Questions tagged [normalisation]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer
34 views

What are the consequences when we multiply, instead of add, a penalty term?

The typical objective function in regression problems like Lasso or Ridge includes a Residual Sum of Squares (RSS) term added to a penalty term based on a norm of the coefficients. What are the ...
BigMistake's user avatar
-2 votes
0 answers
16 views

Normalization in 1NF, 2NF and 3NF

I have a table and I want to create db and tables for normalization in 1NF, 2NF and 3NF based on below table : " librarian_fname librarian_surname shift day assignment Jesse ...
erik's user avatar
  • 1
0 votes
0 answers
26 views

In multilayer perceptron neural networks, is normalization required? or is optional? Why the network only seem to work when I use normalization?

I followed the steps in the following article (which teaches how to program a multilayer perceptron neural network from scratch in Python): https://machinelearningmastery.com/implement-backpropagation-...
will The J's user avatar
0 votes
2 answers
33 views

Must I "prime" my normalizer with the same data I trained it with in order to use it?

I trained a Keras Network. During training, I would first initialize a normalizer from the values in the entire dataset, then partition into train, test and validation datasets. After partitioning, I ...
Pittsburgh DBA's user avatar
0 votes
0 answers
16 views

FastGAN implementation has redundant SpectralNorm followed by BatchNorm?

I am implementing a version of FastGAN, and it seems like (see the official repo) they use a Spectral norm directly followed by a Batch norm. Doesn't the spectral norm get fully cancelled by the ...
Ronald's user avatar
  • 21
1 vote
1 answer
35 views

What's $\mathbb{V}[\gamma]$ in Return-based Scaling: Yet Another Normalisation Trick for Deep RL?

In the paper Return-based Scaling: Yet Another Normalisation Trick for Deep RL, $\sigma$, the scaling factor to normalize the TD error, is computed as: $\sigma^2 = \mathbb{V}[R] + \mathbb{V}[\gamma]\...
Sanyou's user avatar
  • 165
0 votes
1 answer
24 views

Should I apply a min-max scale (range 0 to 1) before applying the normalisation or should I apply the z-score normalisation directly?

I want to implement a neural network in Pytorch for medical image segmentation. I should normalise my data. Should I apply a min-max scale (range 0 to 1) before applying the normalisation or should I ...
Janikas's user avatar
  • 101
0 votes
0 answers
16 views

Normalisation in feature extraction using pre-trained model

I have a dataset with medical images. I want to implement a network for super-resolution using GANs. One of the criteria of the optimisation is a perceptual loss. For that I will use a pretrained vgg ...
Janikas's user avatar
  • 101
0 votes
1 answer
50 views

Can we model the global statistics of the dataset features using LayerNorm?

When applying layer normalization in a neural network, we consider the mean and variance of each input data point to the layer separately without considering the global statistics in the entire data-...
Meisam Ashraf's user avatar
0 votes
1 answer
313 views

In the attention mechanism, why don't we normalize after multiplying values?

As this question says: In scaled dot product attention, we scale our outputs by dividing the dot product by the square root of the dimensionality of the matrix: The reason why is stated that this ...
Peyman's user avatar
  • 554
0 votes
0 answers
9 views

Why is avoiding normalized models a practical solution for reducing the complexity in NNLM?

In the paper Efficient Estimation of Word Representations in Vector Space, the authors say that "avoiding normalized models completely by using models that are not normalized during training"...
Propr's user avatar
  • 1
0 votes
0 answers
13 views

Drastic Change in MSE when Scaling Factor Changes

I am training an autoencoder meant to detect anomalies. Initially I scaled my data using a min-max scaler. I realized that this scalar isn't the best because anomalies can cause bias in the scalar. ...
theastronomist's user avatar
0 votes
1 answer
109 views

What does linear regime of nonlinearity mean in normalisation?

In section 3 paragraph 2 of Batch Normalization: Accelerating Deep Network Training b y Reducing Internal Covariate Shift paper (https://arxiv.org/abs/1502.03167) they say that normalizing a layer's ...
rkuang25's user avatar
3 votes
2 answers
87 views

What if each sample was normalized on its own before sending them to the neural network?

The standard method is to normalize the entire dataset (the training part) then send it to the model to train on. However I’ve noticed that in this manner the model doesn’t really work well when ...
Maks's user avatar
  • 31
0 votes
1 answer
84 views

During batch normalization is the mini-batch gone through twice, one to calculate the mean and variance and then to normalize them?

I am asking this question because while designing my own model, I had repeated gradient explosion issues, so I wanted to try batch normalization. I really want to understand the details and math ...
liyu zerihun's user avatar
2 votes
3 answers
1k views

Do I need to normalize all state-space variables? If so, how?

I am playing around with a DRL agent in a stock-trading environment. I have normalized all the external input data (the features that my agent will use). However, what about characteristics that don't ...
Vladimir Belik's user avatar
1 vote
0 answers
95 views

Normalisation of reward function

Problem Currently, I have some problems defining a reward function for my RL project and mainly with how to normalise the score such that the highest possible score for all instances of the ...
Jesse's user avatar
  • 31
0 votes
1 answer
68 views

Which generalization of standard deviation to use for multidimensional input normalization

For machine learning tasks, it's common to normalize input data by subtracting the mean $\mu$ and dividing by the standard deviation $\sigma$ of the dataset: $$\hat{x_i} = \frac{x_i - \mu}{\sigma}$$ —...
yuri kilochek's user avatar
3 votes
2 answers
5k views

Should I apply normalization to the observations in deep reinforcement learning?

I am new to DRL and trying to implement my custom environment. I want to know if normalization and regularization techniques are as important in RL as in Deep Learning. In my custom environment, the ...
moyukh's user avatar
  • 31
1 vote
0 answers
29 views

How to choose proper normalization strategy for the activations?

I am reading a survey on various normalization techniques adopted in neural network architectures. The purpose of introducing normalization is understandable - to stabilize the training and avoid ...
spiridon_the_sun_rotator's user avatar
1 vote
1 answer
58 views

Does it make sense to apply batch normalization to a batch size of 1?

I am interested in your opinion on the topic if you think that it makes sense to use batch normalization layer in a network that is trained with a batch size of 1. This is a special case as part of an ...
user3352632's user avatar
0 votes
1 answer
35 views

Normalization of possibly not fully representative data

I am trying to train a classification RNN model on a sequence of table medical data, but I stuck with the normalization problem. I realized that I cannot simply use MinMaxScaler, because of 3 problems:...
banderlog013's user avatar
1 vote
1 answer
590 views

How to scale all positive continuous reward?

My RL project has all positive continuous rewards for every step and the goal is to have the maximum cumulative reward (episodic reward). The problem is that the rewards are too close and all between ...
fardis nadimi's user avatar
1 vote
0 answers
39 views

Why do you calculate the mean and standard deviation over the complete dataset before training rather than for every batch?

In most implementations of neural networks the features are scaled to make the optimization of the loss function as stable as possible. Mostly a min-max scaler is used. Alternatively, there is also a ...
flomax's user avatar
  • 11
1 vote
0 answers
64 views

For image preprocessing, is it better to use normalization or standartization?

For a neural network model that classifies images, is it better to use normalization (dividing by 255.0) or using standardization (subtract mean and divide by STD)? When I started learning ...
artas2357's user avatar
  • 153
1 vote
1 answer
143 views

How to define a "don't care" class in time series classification in Pytorch?

This is a theoretical question. Setup I have a time series classification task in which I should output a classification of 3 classes for every time stamp t. All ...
Gulzar's user avatar
  • 749
1 vote
1 answer
453 views

How to normalize images before training?

I have seen people normalize images by just dividing 255. But why? Why not use mean normalization or Z-score Normalization? I also came across this StackOverflow topic while searching but the answers ...
Baran Aldemir's user avatar
2 votes
1 answer
218 views

Is data leakage relevant when scaling across samples?

I have a question about data leakage when pre-processing data for a neural network and whether data leakage actually applies in my instance. I have variance stabilising transformed genomic data. ...
user9317212's user avatar
2 votes
1 answer
355 views

Why does batch norm standardize with sample mean/variance, when it also learns parameters to scale the mean/variance?

Batch norm is a normalizing layer that is shown to help deep networks learn faster and with higher generalization accuracy. It normalizes the activations of the previous layer to a mean $\beta$ and ...
user3667125's user avatar
  • 1,510
0 votes
1 answer
447 views

Flatten image using Neural network and matrix transpose

I have read a lecture note of Prof. Andrew Ng. There was something about data normalization like how can we flatten an image of (64x64x3) into a (64x64x3)*x1 vector. After that there is pictorial ...
Encipher's user avatar
  • 125
1 vote
2 answers
133 views

What would be a typical pre-processing and data normalization pipeline for time series data (for non-linear models such as neural networks)?

I've started to work on time series. I was wondering what would be the best data normalizing and pre-processing technique for non-linear models, specifically, neural networks. One I can think of is ...
Leo's user avatar
  • 290
-1 votes
1 answer
261 views

Is it necessary to standardise the expected output

Normalisation transform data into a range: $$X_i = \dfrac{X_i - Min}{Max-Min}$$ Practically, I found out that the model doesn't generalise well when using normalisation of input data, instead of ...
Dan D.'s user avatar
  • 1,283
2 votes
1 answer
171 views

Do I need to denormalise results in linear regression?

I have learned so far how to linear regression with one or multiple features. So far, so good, everything seems to work fine, at least for my first simple examples. However, I now need to normalise my ...
Golo Roden's user avatar
8 votes
0 answers
118 views

Normalizing Normal Distributions in Thompson Sampling for online Reinforcement Learning

In my implementation of Thompson Sampling (TS) for online Reinforcement Learning, my distribution for selecting $a$ is $\mathcal{N}(Q(s, a), \frac{1}{C(s,a)+1})$, where $C(s,a)$ is the number of times ...
Kevin's user avatar
  • 81
2 votes
1 answer
579 views

Could the normalisation of the inputs make the neural network insensitive to changes in the inputs?

When using neural networks (NNs), we often normalized the inputs. I think this is done to equally capture the changes in any input feature, that is, if any feature takes huge values and other features ...
pranav's user avatar
  • 191