I am creating a dataset made of many images which are created by preprocessing a long time series. Each image is an array of (128,128) and the there are four classes. I would like to build a dataset similar to the MNIST in scikit-learn.database but I have no idea how to do it.

My aim is to have something that I can call like this:

(x_train, y_train), (x_test, y_test) = my_data()

Should I save them as figures? or as csv? Which is the best way to implement this?


I found this solution and will be happy about any improvements or suggestions:

First, I create a random dataset of images, which are 28x28 pixels, and corresponding random labels (just for sake of clarification, I have another image dataset, this is just for explaining). Then I use a sklearn module for splitting the data:

import numpy as np
from sklearn.model_selection import train_test_split

# create the data and labels
def pixel_dataset(n_data=10, dpi = 28):

    ary = np.zeros((n_data, dpi, dpi))
    label = [] 

    for i in np.arange(n_data):
        labels =['a', 'b', 'c']
        ary[i,::] = np.reshape(np.array(np.random.randint(0, 255, 28*28)),(28,28))
        label.append(np.random.choice(labels, 1)[0])

    return ary, label 

# create the test train split    
def mydata(test_size=0.3):

    X, y = pixel_dataset()       
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = test_size) 
    return (X_train, y_train), (X_test, y_test)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.