Questions tagged [normalisation]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Why is avoiding normalized models a practical solution for reducing the complexity in NNLM?

In the paper Efficient Estimation of Word Representations in Vector Space, the authors say that "avoiding normalized models completely by using models that are not normalized during training"...
  • 1
0 votes
0 answers
7 views

Drastic Change in MSE when Scaling Factor Changes

I am training an autoencoder meant to detect anomalies. Initially I scaled my data using a min-max scaler. I realized that this scalar isn't the best because anomalies can cause bias in the scalar. ...
0 votes
1 answer
38 views

What does linear regime of nonlinearity mean in normalisation?

In section 3 paragraph 2 of Batch Normalization: Accelerating Deep Network Training b y Reducing Internal Covariate Shift paper (https://arxiv.org/abs/1502.03167) they say that normalizing a layer's ...
3 votes
2 answers
57 views

What if each sample was normalized on its own before sending them to the neural network?

The standard method is to normalize the entire dataset (the training part) then send it to the model to train on. However I’ve noticed that in this manner the model doesn’t really work well when ...
  • 31
0 votes
1 answer
60 views

During batch normalization is the mini-batch gone through twice, one to calculate the mean and variance and then to normalize them?

I am asking this question because while designing my own model, I had repeated gradient explosion issues, so I wanted to try batch normalization. I really want to understand the details and math ...
0 votes
0 answers
41 views

CNN: Input Normalization for Time Series Data (Grad-CAM)

I trained a CNN model on univariate time series data. As I often read that it is advisable to normalize the data, e.g., using z-normalization, I started with training on z-normalized versions of the ...
0 votes
0 answers
24 views

What is the status of "local Response Normalization" in deep learning community?

In the benchmark papers of deep learning, authors generally contribute multiple novel techniques. Some techniques withstand time and some other techniques might fade out gradually although they seem ...
  • 3,481
2 votes
3 answers
604 views

Do I need to normalize all state-space variables? If so, how?

I am playing around with a DRL agent in a stock-trading environment. I have normalized all the external input data (the features that my agent will use). However, what about characteristics that don't ...
1 vote
0 answers
56 views

Normalisation of reward function

Problem Currently, I have some problems defining a reward function for my RL project and mainly with how to normalise the score such that the highest possible score for all instances of the ...
  • 31
0 votes
1 answer
57 views

Which generalization of standard deviation to use for multidimensional input normalization

For machine learning tasks, it's common to normalize input data by subtracting the mean $\mu$ and dividing by the standard deviation $\sigma$ of the dataset: $$\hat{x_i} = \frac{x_i - \mu}{\sigma}$$ —...
2 votes
2 answers
3k views

Should I apply normalization to the observations in deep reinforcement learning?

I am new to DRL and trying to implement my custom environment. I want to know if normalization and regularization techniques are as important in RL as in Deep Learning. In my custom environment, the ...
  • 21
1 vote
0 answers
27 views

How to choose proper normalization strategy for the activations?

I am reading a survey on various normalization techniques adopted in neural network architectures. The purpose of introducing normalization is understandable - to stabilize the training and avoid ...
1 vote
0 answers
44 views

Does it make sense to apply batch normalization to a batch size of 1?

I am interested in your opinion on the topic if you think that it makes sense to use batch normalization layer in a network that is trained with a batch size of 1. This is a special case as part of an ...
0 votes
1 answer
34 views

Normalization of possibly not fully representative data

I am trying to train a classification RNN model on a sequence of table medical data, but I stuck with the normalization problem. I realized that I cannot simply use MinMaxScaler, because of 3 problems:...
1 vote
1 answer
454 views

How to scale all positive continuous reward?

My RL project has all positive continuous rewards for every step and the goal is to have the maximum cumulative reward (episodic reward). The problem is that the rewards are too close and all between ...
1 vote
0 answers
34 views

Why do you calculate the mean and standard deviation over the complete dataset before training rather than for every batch?

In most implementations of neural networks the features are scaled to make the optimization of the loss function as stable as possible. Mostly a min-max scaler is used. Alternatively, there is also a ...
  • 11
1 vote
0 answers
34 views

For image preprocessing, is it better to use normalization or standartization?

For a neural network model that classifies images, is it better to use normalization (dividing by 255.0) or using standardization (subtract mean and divide by STD)? When I started learning ...
  • 133
1 vote
1 answer
103 views

How to define a "don't care" class in time series classification in Pytorch?

This is a theoretical question. Setup I have a time series classification task in which I should output a classification of 3 classes for every time stamp t. All ...
  • 689
1 vote
1 answer
243 views

How to normalize images before training?

I have seen people normalize images by just dividing 255. But why? Why not use mean normalization or Z-score Normalization? I also came across this StackOverflow topic while searching but the answers ...
2 votes
1 answer
135 views

Is data leakage relevant when scaling across samples?

I have a question about data leakage when pre-processing data for a neural network and whether data leakage actually applies in my instance. I have variance stabilising transformed genomic data. ...
2 votes
1 answer
188 views

Why does batch norm standardize with sample mean/variance, when it also learns parameters to scale the mean/variance?

Batch norm is a normalizing layer that is shown to help deep networks learn faster and with higher generalization accuracy. It normalizes the activations of the previous layer to a mean $\beta$ and ...
  • 1,400
0 votes
1 answer
338 views

Flatten image using Neural network and matrix transpose

I have read a lecture note of Prof. Andrew Ng. There was something about data normalization like how can we flatten an image of (64x64x3) into a (64x64x3)*x1 vector. After that there is pictorial ...
  • 125
0 votes
2 answers
114 views

What would be a typical pre-processing and data normalization pipeline for time series data (for non-linear models such as neural networks)?

I've started to work on time series. I was wondering what would be the best data normalizing and pre-processing technique for non-linear models, specifically, neural networks. One I can think of is ...
  • 240
-1 votes
1 answer
157 views

Is it necessary to standardise the expected output

Normalisation transform data into a range: $$X_i = \dfrac{X_i - Min}{Max-Min}$$ Practically, I found out that the model doesn't generalise well when using normalisation of input data, instead of ...
  • 1,283
2 votes
1 answer
144 views

Do I need to denormalise results in linear regression?

I have learned so far how to linear regression with one or multiple features. So far, so good, everything seems to work fine, at least for my first simple examples. However, I now need to normalise my ...
7 votes
0 answers
99 views

Normalizing Normal Distributions in Thompson Sampling for online Reinforcement Learning

In my implementation of Thompson Sampling (TS) for online Reinforcement Learning, my distribution for selecting $a$ is $\mathcal{N}(Q(s, a), \frac{1}{C(s,a)+1})$, where $C(s,a)$ is the number of times ...
  • 71
2 votes
1 answer
527 views

Could the normalisation of the inputs make the neural network insensitive to changes in the inputs?

When using neural networks (NNs), we often normalized the inputs. I think this is done to equally capture the changes in any input feature, that is, if any feature takes huge values and other features ...
  • 191