3

I have had similar thoughts about neural networks before. Convolution layers are layers of two dimensional nodes effectively passing the spacial data so why don't we use two dimensional hidden layers to receive information out of them. I'm sure someone has used this type of implementation before. I believe the papers bellow are using this. Part of the ...


2

Another explanation of deep learning as an end-to-end framework is in deep learning, pre-processing or feature extraction steps are not necessary. So it only uses a single processing step, which is to train the deep learning model. In other traditional machine learning methods, some separated feature extraction steps usually required. For example in image ...


1

Hello and welcome to the community. There are multiple ways you can train a neural network: stochastic, mini-batch and batch. What you explained is the stochastic mode, where you input one training example 01 for example, calculate the gradients and update the networks weights before the next training example is fed. You could also select multiple such ...


1

I think you may have a class imbalance problem here, if I am reading your output correctly. You have 20,000 negative examples, but only 8000 positive ones, and you are minimizing binary cross entropy without re-weighting the examples, so your model can achieve a low-ish loss just by consistently outputing a value close to 0. This forms a local optima in the ...


1

I don't think he said that at all. Going back to the talk you'll see he mentions mode collapse comes from the naivete of using alternating gradient-based optimization steps because then $min_{\phi}max_{\theta}L(G_\phi, D_\theta)$ starts to look a lot like $max_{\theta}min_{\phi}L(G_\phi, D_\theta)$. This is problematic because in the latter case the ...


1

You should only look for the cross-validation score. If this set is large enough, it will give you an accurate prediction of how your model will act for unseen data. Your case is exceptional. The fitted model which is obviously overfitted actually performs better on the cross-validation set. This means in turn that your overfitted model will perform better ...


1

This is relevant when you have two or more neural networks serving as components to a larger architecture. Training this architecture in an end-to-end manner means simultaneously training all components (i.e. training it as a single network). The best example I can think of are image captioning architectures. These usually comprise of two networks: a CNN ...


1

Yes you can, a few years ago I made a simple CNN for a single Arabic phoneme classification. You can use spectogram or using MFCC / MFSC as features, as long all data has the same size (use padding or cropping if needed). You may need RNN if you want to combine some phonemes to recognize a single word or longer.


1

They don't have acces to the original training or test dataset. Machine learning environments are build on the premise of a benign environment. The models are trained on real data (real inputs). When someone sends a made up input (fake input) it is very easy to fool the model. This is used for example in image recognition. Imagine a fotograph of a panda. ...


1

Read on Fully Convolutional Networks (FCN). There is a lot of papers on the subject, first was "Fully Convolutional Networks for Semantic Segmentation" by Long. The idea is quite close to what you describe - preserve spatial locality in the layers. In FCN there is no fully connected layer. Instead there is average pooling on top of last low-resolution/high-...


Only top voted, non community-wiki answers of a minimum length are eligible