Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
0
votes
0answers
21 views

How to think and build an AI project?

I am starting to study artificial intelligence by my own, since my college stopped the classes by the covid. For learning purposes, I want to build a neural network that can optimize the builds of a ...
0
votes
1answer
16 views

How to make input variable as trainable parameter in a neural network?

I am working on an optimization problem. First, I have done forward training to work the network as a surrogate model, then I freeze the output and I want to find an optimal value of input for a given ...
0
votes
0answers
10 views

What is the best way to convert an historical log data file to a Markov Decision Process (MDP) to perform Q-learning?

Hypothetically, I have an historical log file whose entries contain the instantaneous throughput for the transfer of a set of files (25,000 files ranging in size from 101KB to 222MB) recorded every ...
3
votes
0answers
38 views

When do two identical neural networks have uncorrelated errors?

In Chapter 9, section 9.1.6, Raul Rojas describes how committees of networks can reduce the prediction error by training N identical neural networks and averaging the results. If $f_i$ are the ...
2
votes
1answer
59 views

Why is neural networks being a deterministic mapping not always considered a good thing?

Why is neural networks being a deterministic mapping not always considered a good thing? So I'm excluding models like VAEs since those aren't entirely deterministic. I keep thinking about this and my ...
0
votes
0answers
15 views

Architecture for Logistic Regression with Arbitrary Number of Options

Suppose I want to design a neural network to choose one of several mutually exclusive options. This may normally be done via logistic regression, where the input of the network would be a [batch x ...
0
votes
1answer
22 views

Why won't my model train with CTC loss?

I am trying to train an LSTM using CTC loss, but the loss does not decrease when I train it. I have created a minimal example of my issue by creating training data where the network simply has to copy ...
2
votes
0answers
22 views

How to train/update neural networks faster without a decrease in performance?

I noticed that there are many studies in recent years on how to train/update neural networks faster/quicker with equal or better performance. I find the following methods(except the chips arms race): ...
0
votes
0answers
14 views

How to improve de-noise algorithm on low signal-to-noise ratio features?

In this plot I have features that all have a very small predictive power on y, there is a low signal-to-noise ratio. In order to de-noise them, I tried PCA and k-...
0
votes
0answers
12 views

Low Signal-To-Noise Ratio Data Processing and Model Choices

Attached is a plot showing that y regresses on individual features: Scatter Plot And as shown, each feature has minuscule predictive power on y. I have 130 features like this, all of them are more or ...
0
votes
0answers
10 views

Deriving an ANFIS based controller

I have a Fuzzy controller with a desirable response, and I'm wanting to use the input and output data to derive an ANFIS based controller. ANFIS requires data for 2 inputs and 1 output, I have the ...
2
votes
1answer
37 views

Is it possible to train one part of the network with a particular learning rate and the other part with a different one?

I have a combined network consisting of two parts: one is for images and the other is for numerical data. Each sample is matched with a numerical case by an ID. For this combined network, a ...
1
vote
0answers
28 views

Are monotonically increasing functions easier to learn?

A monotonically increasing function is a function that as x gets bigger so does its output. So, if plotted, it will never go down. Although the outputs might stay constant. Logically this seems like ...
0
votes
0answers
39 views

How to extract parameters from a text using AI/NLP

lets say I have three texts: "make a heading that says hello word" "make a heading of hello world" "create heading consist of hello world" How can I fetch those groups ...
2
votes
0answers
20 views

Why does batch norm standardize with sample mean/variance, when it also learns parameters to scale the mean/variance?

Batch norm is a normalizing layer that is shown to help deep networks learn faster and with higher generalization accuracy. It normalizes the activations of the previous layer to a mean $\beta$ and ...
1
vote
0answers
13 views

How can an “architectural motif” be extracted from a trained MLP?

I am trying to reproduce the paper Synthetic Petri Dish: A novel surrogate model for Rapid Architecture Search. In the paper, the authors try to reduce the architecture of an MLP model trained on ...
0
votes
0answers
18 views

Neural Network trains towards 1 despite target

So I'm trying to make my first neural network and have just finished my back propagation functions. I got the algebra from brilliant and thought I'd understood it, but my bug proves otherwise. The bug ...
3
votes
0answers
90 views
+50

Is it really possible to create the “Perfect Cylinder” used in Universal Approximation Theorem for 1-hidden layer Neural Network?

There are proofs for the universal approximation theorem with just 1 hidden layer. The proof goes like this: Create a "bump" function using 2 neurons. Create (infinitely) many of these ...
0
votes
0answers
19 views

Which AI technique should I use for (key)point detection (in an image of a plantar pressure)?

I am relatively new to the field of AI. I have a problem that I would like to solve with AI, but I don't know which buzzwords I should use to search for solutions. I have a plantar pressure scan, like ...
0
votes
0answers
18 views

What's the best machine learning algorithm / neural network architecture to use for a task that maps between images and textual descriptions of them?

Title says it all really. I want to train a network to take images of diagrams and produce a standard textual definition of them. What ML architecture is best for this?
0
votes
0answers
26 views

Multivariate time-series classification with many variables

I am attempting to use time-series classification algorithms for fraud detection applications. I have came across several works in the literature that propose novel techniques for multivariate time-...
2
votes
0answers
61 views

$\nabla \log \pi$ with respect to some parameters constantly being zero

I am new to reinforcement learning. May I ask a simple (and maybe a bit silly) question here? I am trying to use the "one-step actor-critic" method to train a robot on a gridworld. Let's ...
2
votes
1answer
54 views

Can TensorFlow, PyTorch, and other mainstream ML frameworks be used for research-grade work in AI?

Many authors of research papers in AI (e.g. arXiv) write their neural networks from the ground-up, using low-level languages like C++ to implement their theories. Can existing open source frameworks ...
6
votes
1answer
301 views

How to express a fully connected neural network succintly using linear algebra?

I'm currently reading the paper Federated Learning with Matched Averaging (2020), where the authors claim: A basic fully connected (FC) NN can be formulated as: $\hat{y} = \sigma(xW_1)W_2$ [...] ...
1
vote
0answers
17 views

For the generalised delta rule in back-propogation, do you subtract the target from the obtained output, or vice versa?

When I look up the generalised delta rule equation for back-propogation, I am seeing two conflicting equations. For example, here (slide 20), given $o$ (the output, defined in slide 18), $z$ (the ...
0
votes
0answers
28 views

How can I perform the forward pass in a neural network evolved with NEAT, given that some connections may not exist or there may be loopy connections?

I have a problem that arose as part of a NEAT (Neuro Evolution Through Augmenting Topologies) implementation that I am writing. I am wanting it to produce topologies or graphs that describe neural ...
1
vote
0answers
18 views

Is there any research work that shows that we should explicitly mark the word boundaries for 1D CNNs?

I'm doing character embedding for NLP tasks using one-dimensional convolutional neural networks (see Chiu and Nichols (2016) for the motivation). I haven't found any empirical evidence of whether or ...
8
votes
0answers
60 views

What is the number of neurons required to approximate a polynomial of degree n?

I learned about the universal approximation theorem from this guide. It states that a network even with a single hidden layer can approximate any function within some bound, given a sufficient number ...
0
votes
1answer
39 views

What is the derivative of a specific output with respect to a specific weight?

If I have a neural network, and say the 6th output node of the neural network is: $$x_6 = w_{16}y_1 + w_{26}y_2 + w_{36}y_3$$ What does that make the derivative of: $$\frac{\partial x_6}{\partial w_{...
0
votes
0answers
30 views

How to draw a 3-dimensonal shape's neural network

I am reading an exam question about NN (that I cannot publish, for copyright reasons). The question says: 'Construct a rectangle in 2D space. Define the lines, and then define the weights and ...
0
votes
1answer
39 views

How is the latent vector transforming to a feature map in DCGAN (Generator structure)?

I'm working on the code trying to generate new images using DCGAN model. The structure of my code is from the PyTorch tutorial here. I'm a bit confused trying to find and understand how the latent ...
0
votes
1answer
34 views

Backpropogation rule for the output layer of a multi-layer network - What does the rule do in ambiguous cases?

This is the back-propogation rule for the output layer of a multi-layer network: $$W_{jk} := W_{jk} - C \dfrac{\delta E}{\delta W_{jk}}$$ What does this rule do in the more ambiguous cases such as: (1)...
0
votes
0answers
11 views

How to update the error on hidden nodes using back-propogation, given the error on the output nodes and weights

I'm trying to solve question 30 of this paper. The short version of the question is if someone could show me how to do this, I would appreciate it (the answer should be A; -0.0660). The long version ...
2
votes
1answer
37 views

Output volume proof for convolutional neural network

As I've been dabbling into the sliding window concept, I stumbled on a question that asked me to find the number of windows needed on a 1D image of $W$ size, knowing the window size $K$ and the stride ...
0
votes
0answers
15 views

Is there any solution to the problem of detecting whether a user is having trouble finding something while surfing a webpage?

While a user is navigating through a website, we need to detect whether the user is having trouble finding something in realtime. The output is used to trigger an event that should pop up a FAQ page ...
1
vote
0answers
14 views

Drawing verticle lines on video frames in vehicle detection

I am working on an application of vehicle detection, the purpose is to check if road is congested or not. The solution that came to my mind is to detect the vehicles moving in each direction ...
3
votes
0answers
29 views

What's the difference between content-based attention and dot-product attention?

I'm following this blog post which enumerates the various types of attention. It mentions content-based attention where the alignment scoring function for the $j$th encoder hidden state with respect ...
0
votes
0answers
24 views

Python: Recognize actions / keywords in natural text?

This is for a personal project, and I am no very skilled programmer, language is Python, needs to work offline. That said: I have a list of 200 key actions, eg. 'Building sandcastles', 'Paintball' or '...
0
votes
0answers
14 views

Generating numerical output based on multiple inputs

I have been trying to use a linear regression with Turicreate to predict the a certain number based on a variety of input numbers. My process is pretty simple: I have four columns in my training ...
2
votes
1answer
56 views

What is the space complexity for training a neural network using back-propagation?

Suppose that a simple feedforward neural network (FFNN) contains $n$ hidden layers, $m$ training examples, $x$ features, and $n_l$ nodes in each layer. What is the space complexity to train this FFNN ...
4
votes
1answer
47 views

What is a unified neural network model?

In many articles (for example, in the YOLO paper, this paper or this one), I see the term "unified" being used. I was wondering what the meaning of "unified" in this case is.
1
vote
0answers
32 views

Designing Policy-Network for Deep-RL with Large, Variable Action Space

I am attempting a project involving training an agent to play a game using deep reinforcement learning. This project has a few features that complicate the design of the neural network: The action ...
0
votes
1answer
33 views

Loss randomly changing, incorrect output (even for low loss) when trying to overfit on a single set of input and output

I am trying to make a neural network framework from scratch in C++ just for fun, and to test my backpropagation, I thought it would be an easy way to test the functionality if I give it one input - a ...
1
vote
0answers
24 views

The last target name is missed in the test set

I am training a neural network with a dataset that has 51 classes and 6766 data in it. I used 80% for the training set, 10% for validation, and 10% for the test. After training I got confusion matrix ...
1
vote
0answers
57 views

Are Graph Neural Networks generalizations of Convolutional Neural Networks?

In lecture 4 of this course, the instructor argues that GNNs are generalizations of CNNs, and that one can recover CNNs from GNNs. He presents the following diagram (on the right) and mentions that it ...
0
votes
1answer
49 views

Can you use machine learning for binary data?

I am totally new to artificial intelligence and neural networks and have a broad question that I hope is appropriate to ask here. I am an ecologist working in animal movement and I want to use AI to ...
2
votes
2answers
29 views

Appropriate convolutional neural network architecture when the input consists of two distinct signals

I have a dataset consisting of a set of samples. Each sample consists of two distinct desctized signals S1(t), S2(t). Both signals are synchronous; however, they show different aspects of a phenomena. ...
0
votes
0answers
9 views

Is the information in the hiddenstate of a RNN worth processing further after the input passes the RNN?

I hope the question is understandable. I just wanted to ask if the hidden state, which is passed through the timesteps/cells of an RNN/LSTM/GRU to deliver information from $\text{cell}_{i-1}$ to the $\...
1
vote
0answers
21 views

Papers on using symbolic methods as constraint on neural network?

Given a set of constraints on the input data, I am looking for papers that discuss using symbolic methods (decision trees, rule based, etc.) as a separate source of certainty in a classification task. ...
2
votes
0answers
19 views

Interpretation of Inner Product in a two-tower model

I have seen at quite a few places the use of two-tower architecture. This(Fig 6) is one of the examples. Each tower computes embedding of a concept which is orthogonal to the concepts in the rest of ...

1
2 3 4 5
39