Questions tagged [datasets]
For questions related to sets of data and their use in AI.
204
questions
-1
votes
0answers
35 views
Given a non-continuous fuzzy set, how can I find these other sets?
I stuck in this question for a few hours now. Can anyone help me?
$\tilde{A}$ is a non-continuous fuzzy set and is defined by:
$$\tilde{A} = \{0.6/x_1 +0.5/x_2 + 1/x_3 + 0.75/x_4 + 0.7/x_5 + 0.8/...
0
votes
1answer
44 views
Is it okay to think of any dataset in artificial intelligence as a mathematical set?
A dataset is a collection of data points. It is known that the data points in the dataset can repeat. And the repetition does matter for building AI models.
So, why does the word dataset contain the ...
0
votes
0answers
12 views
How to extract audio embeddings from Google Audioset dataset using Tensorflow 2.x? [migrated]
I am trying to extract audio embeddings from the TFRecord files from Google Audioset dataset but unable to get it.
So far I am able to only parse tfrecord file as string using ...
1
vote
1answer
44 views
Should we also shuffle the test dataset when training with SGD?
When training machine learning models (e.g. neural networks) with stochastic gradient descent, it is common practice to (uniformly) shuffle the training data into batches/sets of different samples ...
1
vote
1answer
40 views
What is the reason for taking tuples as vectors rather than points?
Across the literature of artificial intelligence, especially machine learning, it is normal to treat the tuples of datasets as vectors.
Although there is a convention to treat them as data points. ...
0
votes
0answers
27 views
Transforming neural network target values before training
Consider the scenario in which I am measuring certain $f(a,x)$, which i want to be the target value for some related input $g(a,x)$. In other words, I am trying to map
$$g(a,x)\Rightarrow f(a,x)$$
I ...
0
votes
1answer
51 views
Is it possible to know the distance objects are from camera based on only knowing one object's height?
I am doing a project where I have to know distance a particular object is from camera. In the photo I only know one of the object's height, but I don't know how far away that object is and I don't ...
1
vote
0answers
27 views
How to find distance between 2 points when dimensions are all of different nature?
I have a dataset with four features:
the x coordinate
the y coordinate
the velocity magnitude
angle
Now, I want to measure the distance between two points in the dataset, taking into account the ...
0
votes
0answers
16 views
Multi-label dataloading bottleneck Pytorch
I am trying to write custom dataset and dataloader for pascal-voc-2007. It is a multi-label classification problem. There is csv file to hold the name of the images and their corresponding labels. I ...
1
vote
0answers
48 views
Which loss function to choose for imbalanced datasets?
For imbalanced datasets (either in the context of computer vision or NLP), from what I learned, it is good to use a weighted log loss. However, in competitions, the people who are in top positions are ...
1
vote
1answer
62 views
What are some examples of functions that machine learning models compute?
My simple understanding of AI is that it is based on a mathematical model of a problem. If I understood correctly, the model is a polynomial equation and its weights are calculated by training the ...
0
votes
1answer
21 views
Can a neural network be trained on a dataset containing only values for true output for a classification problem?
I am using a dataset from Google which contains 1,27,000 data points on simulated concentrations of the atmosphere of exoplanets which can sustain life. So, the output label of all these data points ...
0
votes
0answers
37 views
Could the neural network automatically calculate and get different one-to-many quantities relative to their parent quantity?
Let's say I have a primary dataset that its secondary dataset is hundreds to match and group like an one-to-many relationship.
I'm new in this world of the AI but my problem is that many child groups ...
3
votes
2answers
78 views
Can we use genetic algorithms to evolve datasets?
Genetic algorithms are used to solve many optimization tasks.
If I have a dataset, can I evolve it with a genetic algorithm to create an evolved version of the same dataset?
We could consider each ...
1
vote
1answer
44 views
How can I classify houses given a dataset of houses with descriptions?
I have a dataset with a number of houses, for each house, I have a description. For example "The house is luxuriously renovated" or "The house is nicely renovated". My aim is to ...
0
votes
0answers
29 views
Can I use the full dataset for training the CNN feature extractor?
I built a CNN model for extracting features from its dense layer. The extracted features are then used for classification using KNN and Random Forest classifier.
My question is, can I use the whole ...
0
votes
0answers
8 views
Quality of synthetic sample and corresponding label generated by SMOTE or ADASYN?
When SMOTE or ADASYN create synthetic sample and the corresponding label, is the quality of the label trust worthy OR should I should double check the label by myself ?
Particularly in a ...
0
votes
0answers
25 views
How should I label images to get high accuracy with YOLO?
I am new to Object Detection with Yolo and I have questions regarding the labeling (for custom objects): Is there any guideline or tips on how to label images to have high accuracy at the end? ...
0
votes
0answers
17 views
Conditional GAN on harder datasets
I have seen conditional GANs often applied to easier datasets like MNIST and CIFAR-10 to reasonable success, but at the same time these datasets are simple enough that naïve CNNs can fairly easily max ...
0
votes
0answers
11 views
Can I train an object detection model with images with a white background?
To be more specific I have a dataset of 2400 images with unbalanced classes, 1 object per image and sometimes some objects are repeated along the dataset but in a different position and rotation of ...
1
vote
1answer
179 views
How to generate labels for self-supervised training?
I've been reading a lot lately about self-supervised learning and I didn't understand very well how to generate the desired label for a given image.
Let's say that I have an image classification task, ...
0
votes
0answers
73 views
Multilabel stratified split for images/object detection
I am working on an object detection model and have thought of looking into stratified splits for the dataset.
Now since I am doing object detection I have a variable number of "labels" for ...
3
votes
2answers
73 views
How can I predict the true label for data with incomplete features based on the trained model with data with more features?
Suppose I have a model that was trained with a dataset that contains the features (f1, f2, f3, f4, f5, f6). However, my test dataset does not contain all features ...
0
votes
0answers
52 views
Can I train a neural network with multiple datasets (e.g. 25)?
I want to create a neural network that I can train with many datasets (e.g. 20 - 25 datasets). Can I use transfer learning for this? Or is there a better approach than this?
2
votes
0answers
43 views
How can I formulate a prediction problem (given labeled data) as an RL problem and solve it with Q-learning?
One of my friends sent me a problem he was working on lately, and I couldn't help but I wonder how could it be solved using Q-learning. The statement is as follows:
Given the following datasets, the ...
3
votes
1answer
100 views
Is it possible to classify resistors using ResNet50?
I want to train ResNet50 model using resistor images like below:
I tried it by collecting data from google images and there were quite few. So accuracy was very low (around %10) but I wonder If it is ...
0
votes
0answers
27 views
accessible subset of data impacts performance
I have a problem with a subset of my data which is as follows:
I can train a model (doesn't matter what, xgboost, BERT, etc., it is a text classification problem), on my data and get a decent ...
1
vote
0answers
20 views
How is the performance of a CNN trained with monochrome images on image recognition tasks degraded?
For CNN image recognition tasks, like object recognition/face recognition/object segmentation/posture recognition, are there experiment results about how much will the performance be degraded with ...
0
votes
1answer
28 views
How to split data into training validation and test set when the number of data in classes varies greatly?
I have 5 classes of pictures to classify:
0 -> ~3 200 (~800 initial number before interference and duplication)
1 -> ~9 000 (I reduced from ~90 000)
2 -> ~8 000
3 -> ~3 000
4 -> ~7 200
How to ...
2
votes
0answers
26 views
How does sampling works in case of imbalanced image datasets?
I am solving a problem of image classification of the image dataset for 3 classes. Dataset is highly imbalanced.
How will sampling (either over- or under-sampling) work in that case? Should I remove (...
2
votes
1answer
48 views
Is this dataset with only two features suitable for clustering with k-means?
I am working with the K-means clustering algorithm for unsupervised learning.
Is the following dataset suitable for the k-means clustering task or not? Why or why not? The dataset has only two ...
4
votes
1answer
97 views
How to fill missing values in a dataset where some properties can be inputs and outputs?
I have a dataset with missing values, I would like to use machine learning methods to fill.
In more detail, there are $n$
individuals, for which up to 10 properties are provided, all numerical. The ...
2
votes
0answers
31 views
Why is ANFIS important in general?
I am actually working with the iris dataset from sklearn and try to understand the ANFIS-Package for python. But that does not really matter! I have a more general question.
During thinking about ...
1
vote
0answers
27 views
How can raw data from a motion sensor (like an IMU) reduced to the main points of the data
How can I reduce the caputured movement data of a person in a way, that I have filtered the main features of the movement. Or how can I detect pattern/ main features in that data set?
I captured some ...
1
vote
0answers
13 views
What are the best datasets available for music information retrieval?
I am interested in doing some work in classification problems in music information retrieval. I know that there are some formats of datasets (such as MIDI, Spectrogram, Piano-roll, MusicXML, etc.) for ...
0
votes
0answers
23 views
dataset structure for context aware task-oriented dialogue systems
I have been reading papers on Dialog State Tracking and Intent Detection and Slot Filling for building a simple ...
1
vote
0answers
38 views
How can I approximate a function that determines the priority of objects?
I am facing the following supervised learning problem:
An object is fully characterized by its position in $R^n$. There are $m$ objects. There are fully observable (i.e. their positions are always ...
2
votes
1answer
31 views
How are weights for weighted x-entropy loss on imbalanced data calculated?
I am trying to build a classifier which should be trained with the cross entropy loss. The training data is highly class-imbalanced. To tackle this, I've gone through the advice of the tensorflow docs
...
2
votes
2answers
62 views
What are standard datasets for fully connected neural networks?
I am looking for datasets that are used as a testing standard in the fully connected neural networks (FCNN). For example, in the image recognition and CNN, CIFAR datasets are used in most of the ...
3
votes
6answers
74 views
How can I cluster this data frame with several features and observations?
How can I cluster the data frame below with several features and observations? And how would I go about determining the quality of those clusters? Is k-NN appropriate for this?
...
1
vote
0answers
25 views
Are there any training datasets using standard font text rather than hand written ones?
Are there any training datasets using standard font text rather than handwritten ones?
I tried using the MNIST handwritten one on font based chars, but it didn't work well.
1
vote
6answers
74 views
What is the meaning of test data set in naive bayes classifier or decision trees?
What is the benefit of a test data set, especially for naive bayes estimator or decision tree construction?
When using a naive bayes classifier the probabilities are a fact. As far as I know there is ...
3
votes
1answer
177 views
What are “proxy data sets” in machine learning?
The paper Assessment of Deep Generative Models for High-Resolution Synthetic Retinal Image Generation of Age-Related Macular Degeneration uses the term "proxy data sets" in this way
To develop DL ...
4
votes
2answers
108 views
Why do we need both the validation set and test set?
I know that this has been asked a hundred times before, however, I was not able to find a question (and an answer) which actually answered what I wanted to know, respectively, which explained it in a ...
0
votes
2answers
58 views
What does it mean to have epochs=30 in Keras' fit method given certain data?
I have read a lot of information about several notions, like batch_size, epochs, iterations, but because of explanation was without numerical examples and I am not native speaker, I have some kind of ...
1
vote
0answers
20 views
Will structured knowledge bases continue to be used in question answering with the likes of BERT gaining popularity?
This may come across as an open and opinion-based question, I definitely want to hear expert opinions on the subject but I am also looking for references to materials that I can read deeply.
One of ...
2
votes
0answers
28 views
Boston Housing Current Best Model
I'm currently looking for a standard data-set to test some new regression models I've been developing on, and the "Boston Housing Prices" data-set seemed to stand out, since it's very standard.
...
4
votes
2answers
80 views
How should I select the features for predicting diseases (in particular when patients specify their health issues)?
My aim is to train a model for predicting diseases. Now, according to this Wikipedia article, diseases are classified based on the following criteria in general:
Causes (of the disease)
Pathogenesis (...
1
vote
2answers
115 views
Why is the validation performance better than the training performance?
I am training a classifier to identify 24 hand signs of American Sign Language. I created a custom dataset by recording videos in different backgrounds for each of the signs and later converted the ...
1
vote
1answer
57 views
Are sentences from the same document independent and identically distributed?
I am trying to build an LSTM model to generate Shakspeare-like poems. I have data set $\{s_1, s_2, \dots, s_m \}$, which are sentences of Shakespeare poems, and each sentence contains words $\{w_1, ...
