3
$\begingroup$

In my daily machine learning / deep learning workflow, I often want to interact with a dataset in my code. Specifically, I would like to be able to load some module / package which can

  1. make sure that I have the dataset in a specific folder
  2. represent the structure of the dataset on a higher, possible object oriented level
  3. is homogeneous across datasets of the same class, e.g. image classification (set of images with labels, possibly bounding boxes)
  4. open source, such that I could contribute a specification to a previously uncovered dataset
  5. that let's me specify a set of commonly used licenses to which I can comply, to filter all compatible datasets

Currently I mainly use Python, but other languages are also in the scope of this question (Matlab, Java). Does such an API exist? If not, which ones come closest to the requirements stated above?

To give you an example of how I would expect it to feel like, see the following python code

import dataset_api as da

in_2012 = da.get('ImageNet', version='2012')
in_2017 = da.get('ImageNet', version='2017')
coco = da.get('COCO')

image_classification = da.merge([in_2012, in_2017, coco])

images = image_classification.images

image = images[0]
image.path  # the absolute file path to the image file on my disk
image.objects  # an array of object that are visible in this image

Note: this question is also posted on datascience.stackexchange.com. I wasn't sure which exchange better fits the question.

$\endgroup$
2
$\begingroup$

scikit-learn has a small data sets API http://scikit-learn.org/stable/datasets/index.html I imagine one can add more data sets locally. Some data sets are for classification, other for regressions.

This is the only one I know about.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.