Questions tagged [datasets]
For questions related to sets of data and their use in AI.
220
questions
1
vote
0answers
21 views
Identifying rotating and resizing letters with background noise
I'm trying to complete a certain type of captcha for academic purposes. Here is what it looks like:
Between captchas the calligraphy of the letters is the same, but the letters may be resized and ...
0
votes
0answers
17 views
Image regression - estimating sensors from images
I am trying to use images to predict the sensor data of a racing game.
Being a bit of a newcomer I have multiple questions. All help/suggestion is appreciated.
Dataset
The dataset looks something like:...
0
votes
0answers
10 views
Should we create a label mean group of small nearby objects in object detection?
I'm working on object detection models and my dataset sometimes has a lot of small objects (stay far from the scene) (overlapping and nearby) which is really annoying in annotating (it's too small and ...
1
vote
0answers
11 views
Split on dataset with some shared features?
I have a dataset with financial stock data, some of the features are shared, for example daily gold prices, while the stock price for each individual stock is different, the gold price would be the ...
1
vote
0answers
18 views
Pixel values of segmap in multi-class semantic segmentation
I'm preparing a dataset for a multiclass semantic segmentation using U-Net like architecture. To be precise, I've got it ready but a question came to my mind. How does pixel values of a segmentation ...
0
votes
1answer
32 views
Cnn for Combination of both digits and letters(small and capital) [closed]
Hi I am new to machine learning can anyone suggest open dataset consists of both digits and letters(small,capital)
I want images consisisting of both digits and letters to train my cnn model and make ...
0
votes
0answers
21 views
Is it possible to do object detection on an object classification dataset?
I'm new to computer vision, which I find fascinating. I wonder whether it is possible or if there has been any research into going from object recognition data to object detection. In other words, ...
0
votes
0answers
53 views
How does pairwise comparison training work in XGBoost's XGBRanker?
I'm interested in learning to rank with pairwise comparison. While working on this, I found that XGBoost has a model called XGBRanker, which works very well.
I want to find out how the XGBRanker ...
2
votes
4answers
58 views
How do I increase the size of an (almost) balanced dataset?
I am trying to add more data points in my (almost) balanced dataset for training my neural network. I have come across techniques such as SMOTE or Random Over Sampling, but they work best for ...
1
vote
1answer
34 views
How to source training data in ML for information security?
A company entrusts a Data Scientist with the mission of processing and valuing data for the research or treatment of events related to traces of computer attacks. I was wondering how would he get the ...
0
votes
0answers
10 views
Predicting training time using randomly generated datasets
Given a dataset, I need to predict the amount of time it will take to fit a model to it. I plan to do this by creating a csv containing the logs of previously fit models, and passing that data itself ...
1
vote
1answer
29 views
Can residual connections be beneficial when we have a small training dataset?
I have a classification problem, for which an inadequate amount of training data is available. Also, there is no known practical data augmentation approach for this problem (as no unlabelled data is ...
0
votes
0answers
15 views
Why doesn't U-Net work with images different from the dataset?
I have implemented a U-Net, similar to this implementation, but for a different dataset, this one, to segment roads.
It works fine using the test folder images, but, for example, when I pick a print ...
3
votes
1answer
61 views
Why different images of the same person, under some restrictions, are in a 50 dimension manifold?
In this lecture (starting from 1:31:00) the professor says that the set of all images of a person lives in a low dimensional surface (compared the the set of all possible images). And he says that the ...
1
vote
0answers
18 views
Theoretical limits on correlation between classification algorithm performances
Are there any known theoretical bounds, or at least heuristic approaches, regarding the relation or correlation between the performances of any two different classification algorithms?
For example, ...
2
votes
1answer
64 views
Would it be possible to determine the dataset a neural network was trained on?
Let's say we have a neural network that was trained with a dataset $D$ to solve some task. Would it be possible to "reverse-engineer" this neural network and get a vague idea of the dataset $...
3
votes
1answer
60 views
How to treat (label and process) edge case inputs in machine learning?
In every computer vision project, I struggle with labeling guidelines for border cases. Benchmark datasets don't have this problem, because they are 'cleaned', but in real life unsure cases often ...
1
vote
1answer
90 views
What dataset might Elon Musk's Dall-E have used?
Dall-E, it can generate many imaginative images from the description, even some peculiar images, how did they actually create this kind of dataset to train this AI , because there is not much of that ...
2
votes
0answers
49 views
In few-shot classification, should I use my custom dataset as the validation dataset and mini-ImageNet as the training dataset?
I am new to few-shot learning, and I wanted to get a hands-on understanding of it, using Reptile algorithm, applied to my custom dataset.
My custom dataset has 30 categories, with 5 images per ...
4
votes
1answer
123 views
How should I generate datasets for a SARSA agent when the environment is not simple?
I am currently working on my master's thesis and going to apply Deep-SARSA as my DRL algorithm. The problem is that there is no datasets available and I guess that I should generate them somehow. ...
1
vote
0answers
39 views
Is training on single game each time appropriate for an agent to learn to play checkers
I was facing a problem I mentioned in a previous question but after a while, I realize that maybe the problem in the dataset not in the learning rate.
I build the dataset from white positions only i.e ...
0
votes
0answers
19 views
Where can a dataset of relationship between images be used?
I'm making a platform that will collect data about the relationship between different images. For example, if I have three images: a Christmas tree, a gift and Santa... then these will be connected by ...
1
vote
0answers
13 views
Is there a pre-trained network trained on RGB-D (4) channels? [closed]
The most used pre-trained networks for computer vision (e.g. ResNet50) are trained on 3 channels (RGB). At the same time, many cameras used in robotics return RGB-D outputs, that is including depth ...
2
votes
2answers
30 views
Why can't we combine both training and validation data, given that both types of data are used for developing the model?
Sorry if I sound confused. I read that data to be fed to a machine are divided into training, validation and test data. Both training and validation data are used for developing the model. Test data ...
4
votes
1answer
82 views
Can we use ML to do anything else other than predicting (in the case of mathematical problems)?
(The math problem here just serves as an example, my question is on this type of problems in general).
Given two Schur polynomials, $s_\mu$, $s_\nu$, we know that we can decompose their product into a ...
0
votes
0answers
37 views
Is there any rule of thumb to determine the amount of data needed to train a CNN
I am training an AlexNet Convolutional Neural Network to classify images in a dataset. I want to know if there is any general rule for using data augmentation in training a neural network. How can I ...
2
votes
1answer
69 views
How robust are deep networks to class imbalance?
Before deep learning, I worked with machine learning problems where the data had a large class imbalance (30:1 or worse ratios). At that time, all the classifiers struggled, even after under-sampling ...
0
votes
0answers
50 views
What could be a good $\mathcal{R}$ dataset in the article “Old Photo Restoration via Deep Latent Space Translation”?
There are three domains in this article: Old Photo Restoration via Deep Latent Space Translation.
The real old pictures noted by $\mathcal{R}$,
the artificial old pictures noted by $\mathcal{X}$, and
...
1
vote
1answer
72 views
Is it okay to think of any dataset in artificial intelligence as a mathematical set?
A dataset is a collection of data points. It is known that the data points in the dataset can repeat. And the repetition does matter for building AI models.
So, why does the word dataset contain the ...
1
vote
1answer
168 views
Should we also shuffle the test dataset when training with SGD?
When training machine learning models (e.g. neural networks) with stochastic gradient descent, it is common practice to (uniformly) shuffle the training data into batches/sets of different samples ...
1
vote
1answer
59 views
What is the reason for taking tuples as vectors rather than points?
Across the literature of artificial intelligence, especially machine learning, it is normal to treat the tuples of datasets as vectors.
Although there is a convention to treat them as data points. ...
0
votes
0answers
34 views
Transforming neural network target values before training
Consider the scenario in which I am measuring certain $f(a,x)$, which i want to be the target value for some related input $g(a,x)$. In other words, I am trying to map
$$g(a,x)\Rightarrow f(a,x)$$
I ...
0
votes
1answer
61 views
Is it possible to know the distance objects are from camera based on only knowing one object's height?
I am doing a project where I have to know distance a particular object is from camera. In the photo I only know one of the object's height, but I don't know how far away that object is and I don't ...
1
vote
0answers
29 views
How to find distance between 2 points when dimensions are all of different nature?
I have a dataset with four features:
the x coordinate
the y coordinate
the velocity magnitude
angle
Now, I want to measure the distance between two points in the dataset, taking into account the ...
0
votes
0answers
75 views
Multi-label dataloading bottleneck Pytorch
I am trying to write custom dataset and dataloader for pascal-voc-2007. It is a multi-label classification problem. There is csv file to hold the name of the images and their corresponding labels. I ...
1
vote
0answers
69 views
Which loss function to choose for imbalanced datasets?
For imbalanced datasets (either in the context of computer vision or NLP), from what I learned, it is good to use a weighted log loss. However, in competitions, the people who are in top positions are ...
1
vote
1answer
70 views
What are some examples of functions that machine learning models compute?
My simple understanding of AI is that it is based on a mathematical model of a problem. If I understood correctly, the model is a polynomial equation and its weights are calculated by training the ...
0
votes
1answer
23 views
Can a neural network be trained on a dataset containing only values for true output for a classification problem?
I am using a dataset from Google which contains 1,27,000 data points on simulated concentrations of the atmosphere of exoplanets which can sustain life. So, the output label of all these data points ...
1
vote
0answers
42 views
Could the neural network automatically calculate and get different one-to-many quantities relative to their parent quantity?
Let's say I have a primary dataset that its secondary dataset is hundreds to match and group like an one-to-many relationship.
I'm new in this world of the AI but my problem is that many child groups ...
3
votes
2answers
113 views
Can we use genetic algorithms to evolve datasets?
Genetic algorithms are used to solve many optimization tasks.
If I have a dataset, can I evolve it with a genetic algorithm to create an evolved version of the same dataset?
We could consider each ...
1
vote
1answer
49 views
How can I classify houses given a dataset of houses with descriptions?
I have a dataset with a number of houses, for each house, I have a description. For example "The house is luxuriously renovated" or "The house is nicely renovated". My aim is to ...
0
votes
0answers
34 views
Can I use the full dataset for training the CNN feature extractor?
I built a CNN model for extracting features from its dense layer. The extracted features are then used for classification using KNN and Random Forest classifier.
My question is, can I use the whole ...
0
votes
0answers
9 views
Quality of synthetic sample and corresponding label generated by SMOTE or ADASYN?
When SMOTE or ADASYN create synthetic sample and the corresponding label, is the quality of the label trust worthy OR should I should double check the label by myself ?
Particularly in a ...
0
votes
0answers
31 views
How should I label images to get high accuracy with YOLO?
I am new to Object Detection with Yolo and I have questions regarding the labeling (for custom objects): Is there any guideline or tips on how to label images to have high accuracy at the end? ...
0
votes
0answers
19 views
Conditional GAN on harder datasets
I have seen conditional GANs often applied to easier datasets like MNIST and CIFAR-10 to reasonable success, but at the same time these datasets are simple enough that naïve CNNs can fairly easily max ...
0
votes
0answers
35 views
Can I train an object detection model with images with a white background?
To be more specific I have a dataset of 2400 images with unbalanced classes, 1 object per image and sometimes some objects are repeated along the dataset but in a different position and rotation of ...
2
votes
1answer
496 views
How to generate labels for self-supervised training?
I've been reading a lot lately about self-supervised learning and I didn't understand very well how to generate the desired label for a given image.
Let's say that I have an image classification task, ...
1
vote
0answers
137 views
Multilabel stratified split for images/object detection
I am working on an object detection model and have thought of looking into stratified splits for the dataset.
Now since I am doing object detection I have a variable number of "labels" for ...
3
votes
2answers
79 views
How can I predict the true label for data with incomplete features based on the trained model with data with more features?
Suppose I have a model that was trained with a dataset that contains the features (f1, f2, f3, f4, f5, f6). However, my test dataset does not contain all features ...
0
votes
0answers
53 views
Can I train a neural network with multiple datasets (e.g. 25)?
I want to create a neural network that I can train with many datasets (e.g. 20 - 25 datasets). Can I use transfer learning for this? Or is there a better approach than this?
