4
$\begingroup$

I am training a Semi-Supervised GAN, using multivariate time-series with window of shape (180*80) with the generator and discriminator architecture below. My data is scaled using Robust Scaler, so I kept linear activation for the generator output.

During the training I get noise in the generated signals and I can't understand the reason why whereas the original data is smooth. What can be the reason for this noise?

A generated signal.Another generated signal.

def make_generator_model(noise):
    w_init = tf.random_normal_initializer(stddev=0.02)
    gamma_init = tf.random_normal_initializer(1., 0.02)
    
    def residual_layer(layer_input):
        
        res_block = Conv1D(128, 3, strides=1, padding='same')(layer_input)
        res_block = BatchNormalization(gamma_initializer=gamma_init)(res_block)
        res_block = LeakyReLU()(res_block)
        res_block = Conv1D(128, 3, strides=1, padding='same')(res_block)
        res_block = BatchNormalization(gamma_initializer=gamma_init)(res_block)
        res_block = LeakyReLU()(res_block)
        res_add = Add()([res_block, layer_input])
        
        return res_add
    
    in_noise = Input(shape=(100,))
    

    gen = Dense(180*65, kernel_initializer=w_init, use_bias=None)(in_noise)
    gen = BatchNormalization(gamma_initializer=gamma_init)(gen)
    gen = LeakyReLU()(gen)

    gen = Reshape((180, 65))(gen)
    #assert model.output_shape == (None, 45, 256) # Note: None is the batch size

    gen = Conv1D(64, 7, strides=1, padding='same', kernel_initializer=w_init, use_bias=None)(gen)
    #assert model.output_shape == (None, 45, 128)
    gen = BatchNormalization(gamma_initializer=gamma_init)(gen)
    gen = LeakyReLU()(gen)
        
    gen = Conv1D(64, 4, strides=2, padding='same', kernel_initializer=w_init, use_bias=None)(gen)
    #assert model.output_shape == (None, 45, 128)
    gen = BatchNormalization(gamma_initializer=gamma_init)(gen)
    gen = LeakyReLU()(gen)
    
    gen = Conv1D(128, 4, strides=2, padding='same', kernel_initializer=w_init, use_bias=None)(gen)
    #assert model.output_shape == (None, 45, 128)
    gen = BatchNormalization(gamma_initializer=gamma_init)(gen)
    gen = LeakyReLU()(gen)
    
    for i in range(6):
        gen = residual_layer(gen)

    gen = Conv1DTranspose(128, 4, strides=2, padding='same', kernel_initializer=w_init, use_bias=None)(gen)
    #assert model.output_shape == (None, 90, 64)
    gen = BatchNormalization(gamma_initializer=gamma_init)(gen)
    gen = LeakyReLU()(gen)
    
    gen = Conv1DTranspose(128, 4, strides=2, padding='same', kernel_initializer=w_init, use_bias=None)(gen)
    #assert model.output_shape == (None, 90, 64)
    gen = BatchNormalization(gamma_initializer=gamma_init)(gen)
    gen = LeakyReLU()(gen)
    

    out_layer = Conv1D(65, 7, strides=1, padding='same', kernel_initializer=w_init, use_bias=None)(gen)
    #assert model.output_shape == (None, 180, 65)
    
    model = Model(in_noise, out_layer)

    return model

def make_discriminator_model(n_classes=8):
    w_init = tf.random_normal_initializer(stddev=0.02)
    gamma_init = tf.random_normal_initializer(1., 0.02)   
    
    in_window = Input(shape=(180, 65))

    disc = Conv1D(64, 4, strides=1, padding='same', kernel_initializer=w_init)(in_window)
    disc = LeakyReLU()(disc)
    disc = Dropout(0.3)(disc)

    disc = Conv1D(64*2, 4, strides=1, padding='same', kernel_initializer=w_init)(disc)
    disc = LeakyReLU()(disc)
    disc = Dropout(0.3)(disc)
    
    disc = Conv1D(64*4, 4, strides=1, padding='same', kernel_initializer=w_init)(disc)
    disc = LeakyReLU()(disc)
    disc = Dropout(0.3)(disc)
    
    disc = Conv1D(64*8, 4, strides=1, padding='same', kernel_initializer=w_init)(disc)
    disc = LeakyReLU()(disc)
    disc = Dropout(0.3)(disc)
    
    disc = Conv1D(64*16, 4, strides=1, padding='same', kernel_initializer=w_init)(disc)
    disc = LeakyReLU()(disc)
    disc = Dropout(0.3)(disc)
    
    disc = Flatten()(disc)
    
    disc = Dense(128)(disc)
    disc = Dense(128)(disc)
    
    out_layer = Dense(1)(disc)
    
    c_out_layer = Dense(8, activation='softmax')(disc)
    
    model = Model(in_window, out_layer)
    c_model = Model(in_window, c_out_layer)

    return model, c_model
$\endgroup$
1
  • 1
    $\begingroup$ I think at least some of the noise may be because you are using Conv1DTranspose to scale up directly. You might prefer to scale up using simple interpolation then run a normal convolution instead. Hopefully someone can provide a proper answer to this, but intuitively it is related to image GANs' fondness for grid patterns, which is similar high frequency noise. $\endgroup$ Feb 19 at 14:17
4
$\begingroup$

Sorry cannot directly reply to your comment as I posted without an account, and you were right! I replaced transposed layers with Upscale1D+Conv1D and that solved the issue.

gen = Conv1DTranspose(128, 4, strides=2, padding='same', kernel_initializer=w_init, use_bias=None)(gen)

should become (notice that strides=2 becomes strides=1):

gen = Upscale1D()(gen)
gen = Conv1D(128, 4, strides=1, padding='same', kernel_initializer=w_init, use_bias=None)(gen)
$\endgroup$
2
  • 1
    $\begingroup$ I am glad that worked. If you have time to write a self-answer here with abit more detail, I will definitely upvote it. For instance if you described the working architecture in this answer with the same level of detail as in the question $\endgroup$ Feb 19 at 15:41
  • 2
    $\begingroup$ Please, follow the steps here in order to merge your just created account with the unregistered one. $\endgroup$
    – nbro
    Feb 19 at 15:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.