11
There are 2 problems you might face.
Your neural net (in this case convolutional neural net) cannot physically accept images of different resolutions. This is usually the case if one has fully-connected layers, however, if the network is fully-convolutional, then it should be able to accept images of any dimension. Fully-convolutional implies that it doesn'...
7
The neural network is typically a set size once it's created. You'd have to create a network big enough for your data-set.
7
Under US copyright law, this is probably fair use
...but beware of memorization. You may run into more trouble if the AI outputs things very similar to the original work.
Also, consult a lawyer to help you apply the law to your specific situation. This is just information on general legal principles, not any specific situation, and also I'm not a lawyer.
...
5
So I am assuming that you are trying to detect a lego brick from the image. One idea is that you can use transfer learning. Leveraging a pre-trained machine learning model is called transfer learning. The underlying idea behind transfer learning is that one takes a well-trained model from one dataset or domain, and applies it to a new one. François Chollet ...
5
There are various dataset available such as
Pascal VOC dataset: You can perform all your task with these.
size of the dataset is as follows
ADE20K Semantic Segmentation Dataset: you can perform only segmentation here
COCO dataset: This is rich dataset but a size larger then 5 GB so you can try downloading using google colab in your drive and then make ...
5
The difference between the validation and test set in my opinion should be explained in this way:
the validation set is meant to be used multiple times.
the test set is meant to be used only once.
I think that the misunderstanding here arise because machine learning is mostly taught focusing only on a specific part of a large pipeline, which is the model ...
4
There are a ton of sample datasets our there you can play with. A bunch of good ones install with R in the datasets package. Luckily you can download them independently if you're not an R user. Try https://vincentarelbundock.github.io/Rdatasets/datasets.html
You might also be interested in the MNIST database which is one of the canonical databases used ...
4
Use Fine Tuning
You can simply use a pre-trained model on ImageNet, as this data set has multiple snakes classes.
Then you can fine tune the model with your own small data set and outputs.
See this for further understanding :
Fine Tuning in Keras
(if you don't use Keras, there are other tutorials on the internet using other Machine Learning framework)
...
4
In many cases, a production-ready model has everything it needs to make predictions without retaining training data. For example: a linear model might only need the coefficients, a decision tree just needs rules/splits, and a neural network needs architecture and weights. The training data isn't required as all the information needed to make a prediction is ...
4
In computer science, if you say "A is a proxy for B", then it means that "A replaces B" (temporarily or not), or that "A is used as an intermediary for B".
The term "proxy" usually refers to a server, i.e. there are the so-called proxy servers, which intuitively do the same thing (i.e. they are used as intermediaries). ...
4
They are equivalent. When we consider a particular instance as a vector, we are not literally imagining it as an arrow with it's head at the point coordinates and tail at the origin. It's just when you are working with a tuple of numbers in a mathematical context, it is conventional to call it a vector. This language follows into machine learning which is ...
4
I am wondering how to generate datasets when the environment is not as simple as a tic-tac-toe or a maze problem
There is no difference in concept, which is why tic-tac-toe and maze problems are used to teach.
As you have noted, the main difference between reinforcement learning (RL) and supervised learning is that RL does not use labeled datasets. If you ...
4
You can already do this with some neural networks, such as GANs and VAEs, which are generative models that learn a probability distribution over the inputs, so they learn how to produce e.g. images that are similar to the images they were trained with.
Now, if you're interested in whether there is a black-box method, i.e. a method that, for every possible ...
3
There is not single answer to the vanishing gradient problem. However, there a few things that can help.
As mentioned in the comments, use of Rectified Linear Units (ReLU) as your activation function can help, since the it does not get saturated for large neuron inputs.
Next, careful choice of weight initialization can help avoid saturation, as well. See ...
3
We can already observe information bubbles on social media, where the circle is that the ML algorithms learn what content people like and give more similar content based on clicks and so on. From a single wrong click, you could enter a bubble and never come out if you don't take care or be aware.
This happens with humans, so the same may apply to computers. ...
3
I think the crucial point here is what you precisely mean by mislabelled. Google's image classifier will likely do a 'pretty good' job of retrieving images with the given subject included, but how strict or lenient your class requisites are is quite important. For example, if one of your classes is 'dog' there may be hundreds of images procured from scraping ...
3
Recommendation systems can be applied for anything, as long as you have sufficient training data.
The most important inputs to the recommendation system are not "audio files or video files".
Wikipedia gives you the list:
Similarity matrix
Item attributes
User activities and behaviours
User profile
https://en.wikipedia.org/wiki/Recommender_system
3
Your data set would be what is called "unbalanced' and this can lead to problems in developing an accurate classifier.
The best thing to do (which you might not be able to do) is to find more images for those classes with a smaller number of images.
Another alternative is to synthetically produce more images. One way to do that is to use the Keras ...
3
It is somewhat risky to discuss data independently with your learning mechanism. There is actually no such thing as good data or a good learner. There is only data that is good WITH a particular learner. That is even true of human intelligence after all the standardized education and testing done today.
There are also exceptional learners that find data ...
3
I did a little search and couldn't find any database that has ground truth for aggressiveness. This means that you need to build yourself a database. This might be huge undertaking. Take thousands of messages, and classify them by hand whether they are aggressive or not. This part is quite labor intensive.
Second part is much easier at start but would be ...
3
Speaking to the "collecting data" part of the question, I'll say this:
Keep in mind that not everything requires massive amounts of data. Consider also that large amounts of data about all sorts of things are available in the OpenData / LinkedData realms. Governments in particular are sources of massive amounts of data. See, for example, data.gov or Google ...
3
It's important to note that, ultimately, the statistical methods we currently use in ML research are just that: statistical methods. So, when they show some "bad behaviour", it's not because of problems with the statistical methods, but with the data we give them. But if the data we give them are as "genuine and unfiltered" as it gets, ...
3
There are a few.
Sloth
LabelBox
RectLabel
LabelMe
OpenCV also has some facilitation for annotating.
Annotation is the more common name used in software suites for tools that facilitate adding labels to images and frames of movies because. Simple categorization is of limited use in the real world, especially with moving pictures where action may be ...
3
Besides using transfer learning described in other answer, you should consider using siamese network.
This type of network is used in cases when one does not posess many examples of objects he wants to distinguish. General idea is that instead of "telling" the network "This is a cobra", you provide information like: "This is a cobra, and that is a ...
3
How can data augmentation reduce overfitting?
You write that you can already maybe see how data augmentation can help prevent overfitting in general, but it sounds a bit uncertain and it's still asked in the title of the question, so I'll address this first:
Generally, when we use Machine Learning for classification problems, we would ideally learn a ...
3
scikit-learn has a small data sets API http://scikit-learn.org/stable/datasets/index.html
I imagine one can add more data sets locally.
Some data sets are for classification, other for regressions.
This is the only one I know about.
3
Look at Google's Open Image Dataset @ https://storage.googleapis.com/openimages/web/index.html
They provide image-level labels, object bounding boxes, object segmentation masks, and visual relationships.
Here is the link for the traffic signs dataset.
3
I think you'll enjoy this work from Apple on improving the realism of synthetic images. Essentially what you need to do is generate a synthetic image then have your GAN modify the synthetic image so that a 1) a discriminator thinks it is real while also 2) not changing the gross structure of the image very much (so the traffic sign doesn't move) - yes, this ...
3
There are a few tools that you can use to annotate (or label) data. For example, labelme or Labelbox. Have a look at this question for more alternatives.
Only top voted, non community-wiki answers of a minimum length are eligible