8
Non-Euclidian geometry can be generally boiled down to the phrase
"the shortest path between 2 points isn't necessarily a straight line".
Or, put in a way that lends itself very much to machine learning,
"things that are similar to each other are not necessarily close if one uses eucidian distance as a metric" (aka the triangle inequality doesn't hold)...
7
The neural network is typically a set size once it's created. You'd have to create a network big enough for your data-set.
6
There are 2 problems you might face.
Your neural net (in this case convolutional neural net) cannot physically accept images of different resolutions. This is usually the case if one has fully-connected layers, however, if the network is fully-convolutional, then it should be able to accept images of any dimension. Fully-convolutional implies that it doesn'...
5
There are various dataset available such as
Pascal VOC dataset: You can perform all your task with these.
size of the dataset is as follows
ADE20K Semantic Segmentation Dataset: you can perform only segmentation here
COCO dataset: This is rich dataset but a size larger then 5 GB so you can try downloading using google colab in your drive and then make ...
5
The difference between the validation and test set in my opinion should be explained in this way:
the validation set is meant to be used multiple times.
the test set is meant to be used only once.
I think that the misunderstanding here arise because machine learning is mostly taught focusing only on a specific part of a large pipeline, which is the model ...
4
In many cases, a production-ready model has everything it needs to make predictions without retaining training data. For example: a linear model might only need the coefficients, a decision tree just needs rules/splits, and a neural network needs architecture and weights. The training data isn't required as all the information needed to make a prediction is ...
4
So I am assuming that you are trying to detect a lego brick from the image. One idea is that you can use transfer learning. Leveraging a pre-trained machine learning model is called transfer learning. The underlying idea behind transfer learning is that one takes a well-trained model from one dataset or domain, and applies it to a new one. François Chollet ...
3
I did a little search and couldn't find any database that has ground truth for aggressiveness. This means that you need to build yourself a database. This might be huge undertaking. Take thousands of messages, and classify them by hand whether they are aggressive or not. This part is quite labor intensive.
Second part is much easier at start but would be ...
3
There is not single answer to the vanishing gradient problem. However, there a few things that can help.
As mentioned in the comments, use of Rectified Linear Units (ReLU) as your activation function can help, since the it does not get saturated for large neuron inputs.
Next, careful choice of weight initialization can help avoid saturation, as well. See ...
3
We can already observe information bubbles on social media, where the circle is that machine learns what content people like and give more similar content based on clicks and so on. From single wrong click you could enter a bubble and never come out if you don't take care or be aware.
This happens with humans, so same may apply to computers. Checking ...
3
I think the crucial point here is what you precisely mean by mislabelled. Google's image classifier will likely do a 'pretty good' job of retrieving images with the given subject included, but how strict or lenient your class requisites are is quite important. For example, if one of your classes is 'dog' there may be hundreds of images procured from scraping ...
3
Recommendation systems can be applied for anything, as long as you have sufficient training data.
The most important inputs to the recommendation system are not "audio files or video files".
Wikipedia gives you the list:
Similarity matrix
Item attributes
User activities and behaviours
User profile
https://en.wikipedia.org/wiki/Recommender_system
3
Speaking to the "collecting data" part of the question, I'll say this:
Keep in mind that not everything requires massive amounts of data. Consider also that large amounts of data about all sorts of things are available in the OpenData / LinkedData realms. Governments in particular are sources of massive amounts of data. See, for example, data.gov or Google ...
3
Your data set would be what is called "unbalanced' and this can lead to problems in developing an accurate classifier.
The best thing to do (which you might not be able to do) is to find more images for those classes with a smaller number of images.
Another alternative is to synthetically produce more images. One way to do that is to use the Keras ...
3
There are a ton of sample datasets our there you can play with. A bunch of good ones install with R in the datasets package. Luckily you can download them independently if you're not an R user. Try https://vincentarelbundock.github.io/Rdatasets/datasets.html
You might also be interested in the MNIST database which is one of the canonical databases used ...
3
There are a few.
Sloth
LabelBox
RectLabel
LabelMe
OpenCV also has some facilitation for annotating.
Annotation is the more common name used in software suites for tools that facilitate adding labels to images and frames of movies because. Simple categorization is of limited use in the real world, especially with moving pictures where action may be ...
3
How can data augmentation reduce overfitting?
You write that you can already maybe see how data augmentation can help prevent overfitting in general, but it sounds a bit uncertain and it's still asked in the title of the question, so I'll address this first:
Generally, when we use Machine Learning for classification problems, we would ideally learn a ...
3
You are quite correct. If you have properly followed the Cross Validation procedure and selected the best model indeed, then you can use the CV set as the training set for the final model. And no it will not cause your hypothesis to worsen (for that set maybe, but not for new examples) if you have selected the model correctly. In-fact you may use the entire ...
3
scikit-learn has a small data sets API http://scikit-learn.org/stable/datasets/index.html
I imagine one can add more data sets locally.
Some data sets are for classification, other for regressions.
This is the only one I know about.
3
I think you'll enjoy this work from Apple on improving the realism of synthetic images. Essentially what you need to do is generate a synthetic image then have your GAN modify the synthetic image so that a 1) a discriminator thinks it is real while also 2) not changing the gross structure of the image very much (so the traffic sign doesn't move) - yes, this ...
3
There are a few tools that you can use to annotate (or label) data. For example, labelme or Labelbox. Have a look at this question for more alternatives.
3
I do not know a specific tool that meets all the mentioned requirements. However, a long time ago, I had to do a very similar task of labeling tons of images into 10 classes. This is how I did this:
Used a very basic clustering tool to cluster images into clusters (I set the number of clusters larger than 10 as I new some classes have very different ...
3
Decision Tree learners, on their own, are not a good way to deal with imbalanced data. The most commonly used algorithms, by default, make no attempt to address this problem.
If you look carefully at the three sources you post, you will find that they actually all agree on this point.
Two of the sources actually propose methods of addressing this ...
3
Of course, it's possible to define a problem where there is no relationship between input $x$ and output $y$. In general, if the mutual information between $x$ and $y$ is zero (i.e. $x$ and $y$ are statistically independent) then the best prediction you can do is independent of $x$. The task of machine learning is to learn a distribution $q(y|x)$ that is as ...
3
Yes, you can weight the loss function for each example, so that instead of your cost function being
$$J = \sum_i \mathcal{L}(y_i, \hat{y}_i)$$
It will be
$$J = \sum_i w_i\mathcal{L}(y_i, \hat{y}_i)$$
Where $i$ iterates over your data set, $\mathcal{L}$ is the loss function you are using, $y_i$ is ground truth for each example and $\hat{y}_i$ is ...
3
The sentences coming from the same document, author, etc., are unlikely to be independent, that is, the occurrence of a sentence $s_i$ in a certain document $d$ is likely correlated with the occurrence of another sentence $s_j$. If they are not independent, they can also not be independent and identically distributed (which is a stronger condition). The same ...
3
In computer science, if you say "A is a proxy for B", then it means that "A replaces B" (temporarily or not), or that "A is used as an intermediary for B".
The term "proxy" usually refers to a server, i.e. there are the so-called proxy servers, which intuitively do the same thing (i.e. they are used as intermediaries). ...
3
Yes it should be possible. You may have a bug in your code, or the wrong hyperparameters. Training ResNet-50 will take a long time. Try training on other sets of images and see what accuracy you get to check if your approach is correct. Or, try loading a pretrained model, and training from that.
2
A simple way to do it would be lexicograpical sentiment analysis. To do that, you'd need a list of words categorized with a score that reflects "friendly" vs "aggressive" sentiment. For an example of setting up a SA system using Spark, see this article. To do what you're talking about, substitute AFINN for a different dataset. You might have to create ...
2
The answer by Cem Kalyoncu mentions the difficulty of building a ground truth database for aggressiveness.
One alternative approach would be to attempt to operate at the concept level, which would allow the use of pre-existing ontologies such as ConceptNet.
Here's a paper that describes this technique.
Only top voted, non community-wiki answers of a minimum length are eligible
