Questions tagged [datasets]

For questions related to sets of data and their use in AI.

Filter by
Sorted by
Tagged with
3 votes
1 answer
1k views

How to generate labels for self-supervised training?

I've been reading a lot lately about self-supervised learning and I didn't understand very well how to generate the desired label for a given image. Let's say that I have an image classification task, ...
user avatar
3 votes
1 answer
220 views

How to treat (label and process) edge case inputs in machine learning?

In every computer vision project, I struggle with labeling guidelines for border cases. Benchmark datasets don't have this problem, because they are 'cleaned', but in real life unsure cases often ...
user avatar
  • 101
1 vote
1 answer
93 views

Which of the following probability distribution is generating an iid dataset?

Let $X_1, X_2$ be two discrete random variables. Each random variable takes two values: $1, 2$ The probability distribution $p_1$ over $X_1, X_2$ is given by $$p_1(X_1=1, X_2 = 1) = \dfrac{1}{4}$$ $$...
user avatar
  • 3,201
1 vote
2 answers
132 views

How can we "draw i.i.d" from any probability distribution?

Consider the following paragraph from 2 Learning in High Dimensions in from of the paper titled Geometric Deep Learning Grids, Groups, Graphs, Geodesics, and Gauges ...
user avatar
  • 3,201
5 votes
2 answers
265 views

Why do we need both the validation set and test set?

I know that this has been asked a hundred times before, however, I was not able to find a question (and an answer) which actually answered what I wanted to know, respectively, which explained it in a ...
user avatar
4 votes
1 answer
1k views

Dataset containing images of varying dimensions and orientations

I am new to deep learning. I have a dataset of images of varying dimensions of a certain object. A few images of the object are also in varying orientations. The objective is to learn the features ...
user avatar
3 votes
1 answer
2k views

What are "proxy data sets" in machine learning?

The paper Assessment of Deep Generative Models for High-Resolution Synthetic Retinal Image Generation of Age-Related Macular Degeneration uses the term "proxy data sets" in this way To develop DL ...
user avatar
  • 185
3 votes
2 answers
99 views

Are there tools to help labelling images? [closed]

I need to manually classify thousands of pictures into discrete categories, say, where each picture is to be tagged either A, B, or C. Edit: I want to do this work myself, not outsource / crowdsource ...
user avatar
  • 131
1 vote
1 answer
285 views

What is the meaning or implications of the rank of a dataset for machine learning algorithms?

Consider a dataset with $n$ training examples and $d$ features. Let $D_{n \times d}$ be the data matrix and $r$ be the rank of it. In matrices, rank $r$ is generally useful in Knowing the dimension ...
user avatar
  • 3,201
1 vote
2 answers
305 views

Why is the validation performance better than the training performance?

I am training a classifier to identify 24 hand signs of American Sign Language. I created a custom dataset by recording videos in different backgrounds for each of the signs and later converted the ...
user avatar