4
$\begingroup$

I am training a Semi-Supervised GAN, using multivariate time-series with window of shape (180*80) with the generator and discriminator architecture below. My data is scaled using Robust Scaler, so I kept linear activation for the generator output.

During the training I get noise in the generated signals and I can't understand the reason why whereas the original data is smooth. What can be the reason for this noise?

A generated signal.Another generated signal.

def make_generator_model(noise):
    w_init = tf.random_normal_initializer(stddev=0.02)
    gamma_init = tf.random_normal_initializer(1., 0.02)
    
    def residual_layer(layer_input):
        
        res_block = Conv1D(128, 3, strides=1, padding='same')(layer_input)
        res_block = BatchNormalization(gamma_initializer=gamma_init)(res_block)
        res_block = LeakyReLU()(res_block)
        res_block = Conv1D(128, 3, strides=1, padding='same')(res_block)
        res_block = BatchNormalization(gamma_initializer=gamma_init)(res_block)
        res_block = LeakyReLU()(res_block)
        res_add = Add()([res_block, layer_input])
        
        return res_add
    
    in_noise = Input(shape=(100,))
    

    gen = Dense(180*65, kernel_initializer=w_init, use_bias=None)(in_noise)
    gen = BatchNormalization(gamma_initializer=gamma_init)(gen)
    gen = LeakyReLU()(gen)

    gen = Reshape((180, 65))(gen)
    #assert model.output_shape == (None, 45, 256) # Note: None is the batch size

    gen = Conv1D(64, 7, strides=1, padding='same', kernel_initializer=w_init, use_bias=None)(gen)
    #assert model.output_shape == (None, 45, 128)
    gen = BatchNormalization(gamma_initializer=gamma_init)(gen)
    gen = LeakyReLU()(gen)
        
    gen = Conv1D(64, 4, strides=2, padding='same', kernel_initializer=w_init, use_bias=None)(gen)
    #assert model.output_shape == (None, 45, 128)
    gen = BatchNormalization(gamma_initializer=gamma_init)(gen)
    gen = LeakyReLU()(gen)
    
    gen = Conv1D(128, 4, strides=2, padding='same', kernel_initializer=w_init, use_bias=None)(gen)
    #assert model.output_shape == (None, 45, 128)
    gen = BatchNormalization(gamma_initializer=gamma_init)(gen)
    gen = LeakyReLU()(gen)
    
    for i in range(6):
        gen = residual_layer(gen)

    gen = Conv1DTranspose(128, 4, strides=2, padding='same', kernel_initializer=w_init, use_bias=None)(gen)
    #assert model.output_shape == (None, 90, 64)
    gen = BatchNormalization(gamma_initializer=gamma_init)(gen)
    gen = LeakyReLU()(gen)
    
    gen = Conv1DTranspose(128, 4, strides=2, padding='same', kernel_initializer=w_init, use_bias=None)(gen)
    #assert model.output_shape == (None, 90, 64)
    gen = BatchNormalization(gamma_initializer=gamma_init)(gen)
    gen = LeakyReLU()(gen)
    

    out_layer = Conv1D(65, 7, strides=1, padding='same', kernel_initializer=w_init, use_bias=None)(gen)
    #assert model.output_shape == (None, 180, 65)
    
    model = Model(in_noise, out_layer)

    return model

def make_discriminator_model(n_classes=8):
    w_init = tf.random_normal_initializer(stddev=0.02)
    gamma_init = tf.random_normal_initializer(1., 0.02)   
    
    in_window = Input(shape=(180, 65))

    disc = Conv1D(64, 4, strides=1, padding='same', kernel_initializer=w_init)(in_window)
    disc = LeakyReLU()(disc)
    disc = Dropout(0.3)(disc)

    disc = Conv1D(64*2, 4, strides=1, padding='same', kernel_initializer=w_init)(disc)
    disc = LeakyReLU()(disc)
    disc = Dropout(0.3)(disc)
    
    disc = Conv1D(64*4, 4, strides=1, padding='same', kernel_initializer=w_init)(disc)
    disc = LeakyReLU()(disc)
    disc = Dropout(0.3)(disc)
    
    disc = Conv1D(64*8, 4, strides=1, padding='same', kernel_initializer=w_init)(disc)
    disc = LeakyReLU()(disc)
    disc = Dropout(0.3)(disc)
    
    disc = Conv1D(64*16, 4, strides=1, padding='same', kernel_initializer=w_init)(disc)
    disc = LeakyReLU()(disc)
    disc = Dropout(0.3)(disc)
    
    disc = Flatten()(disc)
    
    disc = Dense(128)(disc)
    disc = Dense(128)(disc)
    
    out_layer = Dense(1)(disc)
    
    c_out_layer = Dense(8, activation='softmax')(disc)
    
    model = Model(in_window, out_layer)
    c_model = Model(in_window, c_out_layer)

    return model, c_model
Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ I think at least some of the noise may be because you are using Conv1DTranspose to scale up directly. You might prefer to scale up using simple interpolation then run a normal convolution instead. Hopefully someone can provide a proper answer to this, but intuitively it is related to image GANs' fondness for grid patterns, which is similar high frequency noise. $\endgroup$
    – Neil Slater
    Commented Feb 19, 2021 at 14:17

1 Answer 1

Reset to default
4
$\begingroup$

Sorry cannot directly reply to your comment as I posted without an account, and you were right! I replaced transposed layers with Upscale1D+Conv1D and that solved the issue.

gen = Conv1DTranspose(128, 4, strides=2, padding='same', kernel_initializer=w_init, use_bias=None)(gen)

should become (notice that strides=2 becomes strides=1):

gen = Upscale1D()(gen)
gen = Conv1D(128, 4, strides=1, padding='same', kernel_initializer=w_init, use_bias=None)(gen)
Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ I am glad that worked. If you have time to write a self-answer here with abit more detail, I will definitely upvote it. For instance if you described the working architecture in this answer with the same level of detail as in the question $\endgroup$
    – Neil Slater
    Commented Feb 19, 2021 at 15:41
  • 2
    $\begingroup$ Please, follow the steps here in order to merge your just created account with the unregistered one. $\endgroup$
    – nbro
    Commented Feb 19, 2021 at 15:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .