# Do I need to rotate the masks, if I also rotate the images and the masks are generated from the input?

I am training a neural network that takes an input (H, W, 3) and has the output of size (H', W', C). Now, to augment my dataset, since I only have 45k images, I am using the following in my custom data generator

def Generator():
img=tf.keras.preprocessing.image.random_rotation(img,20m,row_axis=0,col_axis=1,channel_axis=2)



Since I am rotating my input images and the output_masks are generated from information about the input (specifically, heat maps around the joint locations) do I also need to rotate the masks as well?

Yes! This is crucial.

If you rotate your input images for segmentation, you need to rotate the output masks as well. Otherwise the loss of your network will not be correctly calculated and your network will not learn how to generalize to rotated input images.

If you use keras, you can use two ImageDataGenerator classes, one for the images and one for the masks, with the same random seed and augmentation parameters. It looks something like this:

data_gen_args = dict(rotation_range=45)

image_datagen = ImageDataGenerator(**data_gen_args)