0
$\begingroup$

I had lidar 3D point cloud data from semantckitti. I want to perform Semantic Segmentation on the data using U-Net. I converted the 3d point cloud data into 2D using spherical conversion and saved the original point cloud data which was in (.bin format) into numpy arrays with dimensions as 64,1024,5 where: 64 = height , 1024 = width and 5 = xyz coordinates, Intensity and Distance from sensor of each point, in that order.

I also projected the semantic information contained in the label files of point cloud(taken from yaml file of semantickitti), on 2D image plane and saved them in .png format with each pixel having depitcing the color of its respective class.

MY QUESTION IS: I have the multichannel numpy arrays with dimensions (64,1024,5) ,label images in .png format with dimensions (64,1024) as the input data for purposes of Training,validation and testing. How can i input this data in U-Net? Can i input the numpy array with (64,1024,5) directly in U-Net? or some processing needs to take place? Also do i need to perform one-hot encoding to my ground truth label images as they donot contain any additional information at the moment.

Improve this question
$\endgroup$
2
  • $\begingroup$ Have you checked the input image dimension of recommended training dataset of the U-Net? $\endgroup$
    – Cloud Cho
    Commented Mar 28 at 19:38
  • $\begingroup$ Have you checked the input image dimension of recommended training dataset of the U-Net? Also it looks like your last dimension is different from the regular image like RGB. I wonder if the neural network could handle the extra information. You may need change the model structure. By the way, you may consider different 3D to 2D conversion like slicing because your height and width doesn't represent 2D surface. Where do you get idea of spherical conversion? $\endgroup$
    – Cloud Cho
    Commented Mar 28 at 19:44

1 Answer 1

Reset to default
0
$\begingroup$

I think you can directly input (64, 1024, 5) shaped data into the model

  1. as long as the model(U-Net)'s architecture assumes the same shape, which you can find from the first layer of the model
  2. and the data type is correct, for example, if your model expect torch.Tensor, but the data are still nd.array, it would throw an error
  3. and as for the pre-processing, actually the above one I mentioned could be one of the parts of pre-processing, but there are steps like normalisation, data augmentation(if you want) that could be involved in the pre-processing stage.

In summary as long as what the model assumes the input's shape and data type the same as what you're presenting to the model, there shouldn't be a technical problem at all.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .