# Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

1,708 questions
Filter by
Sorted by
Tagged with
5 views

### Image classification on SVG format

To best of my knowledge, images are usually fed in pixel format to ML models. Is there any work that does image classification where the image format is SVG?
11 views

### Training a Neural Network with Hexadecimal input

I currently started working in the field of Machine Learning and have been stuck on a problem where I have to train a Neural Network with a dataset containing hexadecimal inputs. I found that we can ...
15 views

### Loss increasing on fixed input?

I'm training a neural network on some input data. I know that loss increasing may be related to: overfitting, if the loss increases on test data (while still decreases on training data) obscillations ...
49 views

### How can I find a specific word in an audio file?

I'm trying to train and use a neural network to detect a specific word in an audio file. The input of the neural network is an audio of 2-3 seconds duration, and the neural network must determine ...
15 views

### What are some solutions for dealing with time series data that are recorded at uneven intervals?

Let's say I have a time series data which is a bunch of observations that occur at different time stamps and intervals. For example, my observations come from a camera located at a traffic ...
15 views

### How are the step size and covariance matrix updated in CMA-ES?

I've been following the tutorial The CMA Evolution Strategy: A Tutorial to try and understand the CMA-ES, but I'm having trouble understanding how the step size and the covariance matrix are been ...
17 views

### How to input dataset with multi-value properties

I'm trying to learn to use AI, and so I've followed some basic tutorials like training an MLP to predict the price of a car given properties like its age and manufacturer. Now I want to see if I can ...
17 views

### How to interpret the final neural network generated in NEAT-Python?

I made a simple obstacle collision game in Python and made a neural network which will play the game using the NEAT (NeuroEvolution of Augmenting Topologies) and I got a final network as follows -> ...
16 views

### Finding the energy function given update rule of a single layer non-linear neural network

Consider the network with N neurons, each of which takes a $2 \times k$ input specified by the tuple $(\vec c_t, \vec \theta_t)$ to produce output $\vec{R}_t$ through an update rule on the pairwise ...
40 views

### How to use text as an input for a neural network - regression problem? How many likes/claps an article will get

I am trying to predict the number of likes an article or a post will get using a NN. I have a dataframe with ~70,000 rows and 2 columns: "text" (predictor - strings of text) and "likes&...
38 views

### What are some applications where tree models perform better than neural networks?

Neural networks are known to be better modeling techniques as compared to machine learning tree-based algorithms. Are there any exceptions to this?
30 views

### Is it a good idea to train a neural network to classify images without base-hypothesis?

I'm a relative beginner in deep-learning (understand by that, I'm doing my first kaggle competition right now, and I have loads to learn still) and I was just wondering something. Let's say you have ...
82 views

### What is eager learning and lazy learning?

What is the difference between eager learning and lazy learning? How does eager learning or lazy learning help me build a neural network system? And how can I use it for any target function?
86 views

### Why does a neuron in a multi-layer network need several input connections?

For example, if I have the following architecture: Each neuron in the hidden layer has a connection from each one in the input layer. 3 x 1 Input Matrix and a 4 x 3 weight matrix (for the ...
55 views

### Is there a way to reduce the RMSE error when training a neural network to recognise MNIST digits using ANFIS?

I wanted to build a digit recognition neural network using MATLAB ANFIS kit. I started by using the MNIST database and I figured out it's almost impossible to classify 784 dimension data using ANFIS. ...
43 views

### Recommendations or resources for neural network/deep learning for time series application?

I know there are quite a few good deep learning books out there, but most explain neural networks and deep learning via application on images. If there are examples/code, they are often done on the ...
65 views

### What is the difference between artificial neural network (ANN) and deep learning?

I have read many mixed definitions around these two terms. For example, is it right to say deep learning is any ANN with more than two hidden layers? What are formal definitions for these two?
80 views

### Why L2 loss is more commonly used in Neural Networks than other loss functions?

Why L2 loss is more commonly used in Neural Networks than other loss functions? What is the reason to L2 being a default choice in Neural Networks?
47 views

### What are some scalable approaches to perform anomaly detection (for images with small cracks) with unsupervised learning?

I have some images with anomalies, like small cracks, but it's an imbalanced dataset. Please, suggest some effective scalable approaches. Should I consider convolutional auto-encoders? It's supposed ...
69 views

### How can a de-noising auto-encoder act as an anomaly detection model?

In some research papers, I have seen that, for training the autoencoders, instead of giving the non-anomalous input images, they add some anomalies to the normal input images, and train the auto-...
59 views

### In what situations ELUs should be used instead of RELUs?

I always use RELUs actication functions when I need to and I understand limitations of ELUs. So in what situation do I need to consider ELUs over RELUs?
42 views

### Reference for Transfer Learning via Final Layers of a Neural Network

Problem (Sketch): I'm interested in a particular formulation of the transfer-learning problem, which, given a trained network $f$ seeks to learn a new network $g$ whose last few layers behave very ...
71 views

### What activation functions are currently popular?

I am not asking what activation function is better. I want to know what activation functions are more used in research or deployment. Also, are they used in combination? e.g. ReLU, ELUs, etc. I'd ...
52 views

### Backpropagation implementation with Java

I've been trying to implement a Multilayer Perceptron Network using java language with the ultimate goal of creating and teaching a neural network to recognize handwritten digits. Pretty simple and ...
47 views

### What is the reason for mode collapse in GAN as opposed to WGAN?

In this article I am reading: $D_{KL}$ gives us inifity when two distributions are disjoint. The value of $D_{JS}$ has sudden jump, not differentiable at $\theta=0$. Only Wasserstein metric provides ...
77 views

### What kind of neural network can be trained to recognise patterns?

Is there a type of neural network that can be fed patterns to train itself on to complete new patterns that it has not seen before? What I'm trying to do is train a neural network to transform an ...
43 views

### What is meant by “arranging the final features of CNN in a grid” and how to do it?

In the paper What You Get Is What You See: A Visual Markup Decompiler, the authors have proposed a method to extract the features from the CNN and then arrange those extracted features in a grid to ...
28 views

### What could be the possible strategy and Deep Learning method that MathPix might be using for LaTex detection?

I want to build an open Source OCR just like MathPix. There is already a model to extract LaTex from the image by Harverd NLP's im2markup but the problem is that their data has been trained and tested ...
136 views

### Why does every neuron in a multi-layer perceptron typically have the same activation function?

Why does every neuron in a multi-layer perceptron typically have the same activation function? Is this a requirement, are there any advantages, or maybe is it just a rule of thumb?
72 views

### Forcing a neural network to be close to a previous model - Regularization through given model

I'm wondering, has anyone seen any paper where one trains a network but biases it to produce similar outputs to a given model (such as one given from expert opinion or it being a previously trained ...
56 views

### Is it necessary to standardise the expected output

Normalisation transform data into a range: $$X_i = \dfrac{X_i - Min}{Max-Min}$$ Practically, I found out that the model doesn't generalise well when using normalisation of input data, instead of ...
54 views

### Can residual neural networks use other activation functions different from ReLU?

In many diagrams, as seen below, residual neural networks are only depicted with ReLU activation functions, but can residual NNs also use other activation functions, such as the sigmoid, hyperbolic ...
47 views

### Can I train a neural network with multiple datasets (e.g. 25)?

I want to create a neural network that I can train with many datasets (e.g. 20 - 25 datasets). Can I use transfer learning for this? Or is there a better approach than this?
60 views

### What should the output of a neural network that needs to classify in an unsupervised fashion XOR data be?

XOR data, without labels: [[0,0],[0,1],[1,0],[1,1]] I'm using this network for auto-classifying XOR data: ...
66 views

### Is reinforcement learning reward set for step by step, or the whole sequence until failure?

Reinforcement Learning may start with no data, and the agent receives rewards for correct actions. Are the rewards given out step by step, or only until the agent fails then the reward is a negative ...
57 views

### What should I do with the flatten layer during back-propagation?

I'm creating a CNN network without other frameworks such as PyTorch, Keras, Tensorflow, and so on. During the forward pass, the Flatten layer reshapes the previous ...
145 views

### Why is symbolic AI not so popular as ANN but used by IBM's Deep Blue?

Everybody is implementing and using DNN with, for example, TensorFlow or PyTorch. I thought IBM's Deep Blue was an ANN-based AI system, but this article says that IBM's Deep Blue was symbolic AI. Are ...
52 views

### How can the FCNN reduce the dimensions of the input from $1048 \times 100$ to $523 \times 100$ with max-pooling?

I am trying to implement a paper on Image tempering detection and localization, the paper is Image Manipulation Detection and Localization Based on the Dual-Domain Convolutional Neural Networks, I was ...
35 views

### Using a neural network in a microcontroller to recognize air-written letters and numbers

Recently I finished a project which combined stm32f030r8 microcontroller and MPU9250 sensor to create a system which would detect orientation on all 3 axis using a combination of accel data, gyro data ...
65 views

Is there a place where people can share (or buy) ready made neural networks instead of creating them themselves? Something like a Wikipedia for DNNs?
37 views

### Is there a way to get landmark features automatically learned by a neural network?

Is there a way to get landmark features automatically learned by a neural network without having to manually pre-label them in the images that are being fed into the network?
77 views

### Why can't we train neural networks in a peer-to-peer manner?

I have recently been exposed to the concept of decentralized applications, I know that neural networks require a lot of parallel computing infra for training. What are the technical difficulties one ...
43 views

### When would bias regularisation and activation regularisation be necessary?

For Keras on TensorFlow, a layer class constructor comes with these: kernel_regularizer=... bias_regularizer=... ...
42 views

### Why would the learning rate curve go backwards?

I'm working on recognizing the numbers 3 and 7 using the MNIST data set. I'm using cnn_learner() function from fastai library. When I plotted the learning rate, the ...
32 views

### How to calculate confidence score of OCR system?

I am working on an OCR project and I wonder how I can calculate the confidence score of my OCR system. I have digital multi meter images. There are some measurement results on the screens of devices ...
32 views

### CIFAR-10 can't get above 10% Accuracy with MobileNet, VGG16 and ResNet on Keras

I'm trying to train the most popular Models (mobileNet, VGG16, ResNet...) with the CIFAR10-dataset but the accuracy can't get above 9,9%. I want to do that with the completely model (include_top=True) ...
60 views

### What is the purpose of a Neural Network in Reinforcement Learning when we have a Q-learning update rule?

I'm confused as to the purpose of training a neural network (NN) for reinforcement learning (RL) tasks such as Gridworld. In RL tasks, namely q-learning, we have a q-learning update rule, which is ...
56 views

### Would it be possible to implement the principals of the K means clustering algorithm in a Neural Network

During a Machine Learning course which I have done I have learnt about the K means algorithm. Is it possible to use the principals of K means within a neural network?