How to make training data for AI

Right now I'm planning to make a DNN for classifying taste of crystals with their molecular structure which include information like no of atoms mass of each atoms atomic no of atoms. How should I make a data set for training testing and validation?

You can cluster all your features in one matrix X, in which each line would be one element of the data set you want to construct, and each column would be a different feature of this element.

You construct then a Y vector containing the different target classes, where the i-th element will be the target class of the i-th X element.

For the following I suppose you use python

Create train and test sets using sklearn wrapper train_test_split on X and Y just built, in which you can shuffle your X and Y (non-avoidable in this case) :

from sklearn.model_selection import train_test_split


BTW, I recommend you to rescale your data with sklearn with for example one of :

from sklearn.preprocessing import MinMaxScaler, StandardScaler


It allows better learning and so better generalization

• Well done on train and test split and scaling. However, @stain also asked about validation set. Some packages (eg most of Keras supervised learning algorithms) can consider a portion (by with absolute or relative size) of training set as the validation set. Otherwise, one can perform train_test_split twice (one on whole data and once on the train set). See datascience.stackexchange.com/a/15136/46505 – Borhan Kazimipour Mar 21 '18 at 2:09
• I can't upvote your comment because of my low reputation but your comment is relevant ! I also thought of mentioning cross validation using k-splits (kfolds) generalizing the concept of training As it is mentioned in your link @O_o answer seems also relevant here – nsaura Mar 21 '18 at 10:46
• I upvoted Borhan's comment for you :) Thanks for contributing! – DukeZhou Mar 23 '18 at 20:33