# Questions tagged [datasets]

For questions related to sets of data and their use in AI.

238 questions
Filter by
Sorted by
Tagged with
135 views

### How to manage large amounts of image data for training?

Right now, I am trying to synthesize training images for a CNN and due to the nature of the application, there is a finite number of sample images to learn from. From other research, I expect to be ...
61 views

### What is the reason for taking tuples as vectors rather than points?

Across the literature of artificial intelligence, especially machine learning, it is normal to treat the tuples of datasets as vectors. Although there is a convention to treat them as data points. ...
16 views

### Why doesn't U-Net work with images different from the dataset?

I have implemented a U-Net, similar to this implementation, but for a different dataset, this one, to segment roads. It works fine using the test folder images, but, for example, when I pick a print ...
61 views

### Why different images of the same person, under some restrictions, are in a 50 dimension manifold?

In this lecture (starting from 1:31:00) the professor says that the set of all images of a person lives in a low dimensional surface (compared the the set of all possible images). And he says that the ...
18 views

### Theoretical limits on correlation between classification algorithm performances

Are there any known theoretical bounds, or at least heuristic approaches, regarding the relation or correlation between the performances of any two different classification algorithms? For example, ...
82 views

### Can we use ML to do anything else other than predicting (in the case of mathematical problems)?

(The math problem here just serves as an example, my question is on this type of problems in general). Given two Schur polynomials, $s_\mu$, $s_\nu$, we know that we can decompose their product into a ...
67 views

440 views

### Should we also shuffle the test dataset when training with SGD?

When training machine learning models (e.g. neural networks) with stochastic gradient descent, it is common practice to (uniformly) shuffle the training data into batches/sets of different samples ...
29 views

### How to find distance between 2 points when dimensions are all of different nature?

I have a dataset with four features: the x coordinate the y coordinate the velocity magnitude angle Now, I want to measure the distance between two points in the dataset, taking into account the ...
3k views

### How to detect LEGO bricks by using a deep learning approach?

In my thesis I dealt with the question how a computer can recognize LEGO bricks. With multiple object detection, I chose a deep learning approach. I also looked at an existing training set of LEGO ...
81 views

I am trying to write custom dataset and dataloader for pascal-voc-2007. It is a multi-label classification problem. There is csv file to hold the name of the images and their corresponding labels. I ...
165 views

### If we want to classify something as either a cat/dog or neither, do we need 2 or 3 classes?

Suppose one trains a CNN to determine if something was either a cat/dog or neither (2 classes), would it be a good idea to assign all cats and dogs to one class and everything else to another? Or ...
266 views

### For each epoch, can I use only on a subset of the full training dataset to train the neural network?

If one has a dataset large enough to learn a highly complex function, say learning chess game-play, and the processing time to run mini-batch gradient descent on this entire dataset is too high, can I ...
81 views

### Which loss function to choose for imbalanced datasets?

For imbalanced datasets (either in the context of computer vision or NLP), from what I learned, it is good to use a weighted log loss. However, in competitions, the people who are in top positions are ...
72 views

### What are some examples of functions that machine learning models compute?

My simple understanding of AI is that it is based on a mathematical model of a problem. If I understood correctly, the model is a polynomial equation and its weights are calculated by training the ...
23 views

### Can a neural network be trained on a dataset containing only values for true output for a classification problem?

I am using a dataset from Google which contains 1,27,000 data points on simulated concentrations of the atmosphere of exoplanets which can sustain life. So, the output label of all these data points ...
45 views

### Could the neural network automatically calculate and get different one-to-many quantities relative to their parent quantity?

Let's say I have a primary dataset that its secondary dataset is hundreds to match and group like an one-to-many relationship. I'm new in this world of the AI but my problem is that many child groups ...
124 views

### Can we use genetic algorithms to evolve datasets?

Genetic algorithms are used to solve many optimization tasks. If I have a dataset, can I evolve it with a genetic algorithm to create an evolved version of the same dataset? We could consider each ...
51 views

### How can I classify houses given a dataset of houses with descriptions?

I have a dataset with a number of houses, for each house, I have a description. For example "The house is luxuriously renovated" or "The house is nicely renovated". My aim is to ...
629 views

### Which evaluation methods can I use for image segmentation?

I implemented an image segmentation pipeline and I trained it on the DICOM dataset. I compared the results of the model with manual segmentation to find the accuracy. Is there other methods for ...
300 views

### TensorFlow 2.0 - Normalizing input to DNN (on structured data) [closed]

I have a structured dataset of around 100 gigs, and I am using DNN for classification in TF 2.0. Because of this huge dataset, I cannot load entire data in memory for training. So, I'll be reading ...
169 views

### Multilabel stratified split for images/object detection

I am working on an object detection model and have thought of looking into stratified splits for the dataset. Now since I am doing object detection I have a variable number of "labels" for ...
2k views

### How many training data required for GAN?

I'm beginning to study and implement GAN to generate more dataset. I'll just try to experiment with state-of-the-art GAN models as described in here https://paperswithcode.com/sota/image-generation-on-...
97 views

### How can I predict the true label for data with incomplete features based on the trained model with data with more features?

Suppose I have a model that was trained with a dataset that contains the features (f1, f2, f3, f4, f5, f6). However, my test dataset does not contain all features ...