5
Though word-embedding is primarily a language modeling tool, it also acts as a feature extraction method because it helps transform raw data (characters in text documents) to a meaningful alignment of word vectors in the embedding space that the model can work with more effectively (than other traditional methods such as TF-IDF, Bag of Words, etc, on a large ...
4
Feature extraction (FE) is not the same as representation learning (RL), but they are similar and related.
You describe accurately what feature extraction typically refers to, i.e. the process of extracting (new) features from existing ones or raw data (e.g. images). For example, let's say you have a dataset associated with a car. You have only two features ...
3
Introduction
Bag-of-features (BoF) (also known as bag-of-visual-words) is a method to represent the features of images (i.e. a feature extraction/generation/representation algorithm). BoF is inspired by the bag-of-words model often used in the context of NLP, hence the name. In the context of computer vision, BoF can be used for different purposes, such as ...
3
There are methods called "scoring systems" where you give a image scores such as "0.9 stripes, 0.0 red, 0.8 hair, ..." and use those scores to classify objects. It's an older idea, not used to determine if the network is learning. It's not in a standard CNN.
To determine if relevant information is being learned or not, it's standard to use the testing ...
2
I think you guys are playing on semantics.
If you consider feature extraction to be an unlearned preprocessing step to get inputs for your model, then no, word embeddings are not a feature extraction technique (examples here would be BoW counts, n-gram features, etc)
If you consider feature extraction to be any form of conversion from text to a set of ...
2
You get what we call high-level features, which are basically abstract representations of the parts that carry information in the image you want to classify.
Imagine you want to classify a car. The image you feed your network could be a car on a road with a driver and trees and clouds, etc. The network, however, if you've trained it to recognize cars, will ...
2
The problem isn't the GAN but the implementation of its discriminator which is typically a convolutional neural network (CNN). CNNs have trouble with sparse data. They require dense data to learn well. There are ways to work around this. See the following for some ideas:
Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation
Sparse ...
2
Yes, neural networks learn features themselves freeing you from the need to manually engineer them. I will illustrate it here with a toy problem.
Let's assume that we want to learn the areas of parallelograms built on pairs of vectors:
The input data are six coordinates: $(x_1, y_1, x_2, y_2, x_3, y_3)$.
import numpy as np
n_tr = 1000 # training data
x_tr = ...
2
Feature engineering may be necessary when one cannot achieve acceptable error rate — within a budget or in principle.
NN may be stalling due to information bottleneck: too many pigeons, not enough holes. In that case, custom features may provide slightly better information compression. (Alas, this is not a panacea: some layer(s) may still be too narrow. That'...
1
I can't go into details of the algorithm but here's some intuition about what's apparently going wrong:
The Sobel transformation identifies mostly-vertical and mostly-horizontal edges. For slanted edges, it also shows a response, just a bit weaker.
By using a window and taking the minimum of vertical and horizontal response, you identify points where you ...
1
It depends on the data. If it is structured like form data, then you might not need AI at all — simple regular expression patterns might be fine. This would apply for example to address data. If you have the word street followed by a colon, followed by some text, it seems fairly obvious that this is the name of a street, and possibly also a house ...
1
Padding is a common practice both in image-processing (typically via CNNs) and in sequence-processing tasks (RNNs, Transformers).
For CNNs all the standard convolutional layers - Conv1D, Conv2D and Conv3D,- have the padding argument. The padding values can be valid or same for 2d and 3d convolutions. And extra causal type of padding is possible for 1d ...
1
The authors use so-called embeddings, it's a form to represent the images in some meaningful vector form.
The procedure to get embedding as follows. First, keep in mind most of the popular convolutional net architectures starts with convolutional layers and then have few fully connected layers. Then do the following.
Train the full network with one-hot ...
1
The Standard Image Captioning Pipeline is to train the model in a single batch(or mini-batch) i.e. get the features from the CNN Image encoder and then feed that into an RNN decoder (features + Real Captions) to produce output captions for the Image.
The training loop in PyTorch would look something like this:
# zero the parameter gradients
decoder.zero_grad(...
1
There are two ways that you could perform data augmentation:
Up front, by expanding the input dataset into a larger one, performing a range of changes to each input then storing the result. This appears to be what you are suggesting.
Just in time, by sampling from possible augmentations on each epoch, or even per sample when building a mini-batch. This ...
1
There could be multiple possible ways to extract the features. One would be to use RNNs for a temporal relationship as the input data is time-series.
Only top voted, non community-wiki answers of a minimum length are eligible
