Questions tagged [neural-networks]
For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.
2,508
questions
0
votes
0
answers
3
views
Why are thresholds in NN so important?
I am currently working on a paper on the approximation properties of NN. One part of this topic is of course the universal approximation theorem(UAT), which gets discussed in various papers with ...
- 1
0
votes
0
answers
3
views
How to get Llama-2 Rotary Embeddings?
I want to get the Llama-2 rotary embeddings. I do print(model) and get the following output:
In the picture I highlight the rotary embeddings.
How can get the ...
- 101
0
votes
0
answers
29
views
VAE for Motion Sequence Generation - Convergence Issue with Scheduled Sampling
I have implemented a Variational Autoencoder (VAE) in PyTorch for motion sequence generation using human pose data (joint angles and angular velocities in radians) from the CMU dataset. The VAE ...
- 1
-1
votes
0
answers
12
views
Simulate all the graphs and figures attached in the article related to VCSEL laser in the MATLAB
Please simulate all or some of the diagrams and figures present in this article related to VCSEL lasers in MATLAB software.
The attached paper for download is available at the following link.
https://...
0
votes
0
answers
17
views
Cooperation between pattern recognition and neural networks for better design of AI
I continue to try to understand how a human brain works and how we can use it to set up AI which would copy the behavior of a actual person. A few days ago I uploaded a question where I described a ...
- 1
0
votes
0
answers
4
views
What is the policy model in RLHF for LLMs?
What is the policy model doing explicitly in an LLM with RLHF setup?
From my understanding, LLMs generate in a way that is no different from any of their predecessors: beam search decoding, ...
0
votes
0
answers
27
views
Why are convolutions and pooling described as layers in a network?
Whenever I look at resources on convolutions and max-pooling in CNNs they always seem to describe these algorithms as being part of the network - a preliminary set of layers before the main ...
- 13
0
votes
0
answers
39
views
Why does my loss function fluctuate so much?
I have a loss function that I'm trying to maximise using a neural network.
While it does appear to increase and plateau over the training, it does so in a very "noisy" manner, spiking up and ...
- 73
1
vote
2
answers
57
views
Tips and tricks when training a very large language model?
Have never trained a (very) large language model, so I am wondering if the process is the same as training a (regular) language model, i.e. you prepare the data, set up the architecture, ...
-1
votes
0
answers
17
views
If I have a pre trained neural network with all bias and weight files, where and how can I check it's accuracy
Tell me the software I can use to check the accuracy of the neural network and how can I use the software to implement the pre trained layer files in verilog
0
votes
2
answers
32
views
Must I "prime" my normalizer with the same data I trained it with in order to use it?
I trained a Keras Network. During training, I would first initialize a normalizer from the values in the entire dataset, then partition into train, test and validation datasets. After partitioning, I ...
- 149
0
votes
1
answer
25
views
Can a concept/feature be represented using more than one layer of a Neural Network?
I was reading Goodfellow. At the start of the text it was mentioned that there are two ways to represent depth of a deep neural network. One is using the depth of the computation graph and the other ...
- 101
0
votes
0
answers
42
views
Are we missing correlation to achieve a artificial intelligence comparable to humans?
I don't think neural networks are enough to copy human intelligence because in the end a machine only using a neural network to decide on something still is unable to recognize patterns.
So I was ...
- 1
0
votes
0
answers
39
views
Optimizing Stop Loss Percentage for a Specific Model Based on Stock Price to Maximize Expected Value
I'm fine-tuning a specific trading model, and a crucial parameter I'm keen on optimizing is the stop loss percentage. The primary objective is to maximize the Expected Value (EV), formulated as:
$$EV =...
- 11
0
votes
0
answers
21
views
Pixel-wise regression only focus on edge
I am trying to use unet to learn pixel-wise regression from one image to one groundtruth with the same image size. The network seems to focus too much on the edge of the image, and it does not learn ...
- 1
0
votes
0
answers
11
views
Challenges in Developing AI Algorithms for X-ray Image Analysis on Large Datasets
Hello everyone,
I'm currently working on a research project involving X-ray imaging
and the development of AI algorithms to detect diseases from large
...
- 134
0
votes
0
answers
14
views
Machine learning network design for 2D point prediction labeling
I'm looking to develop a basic neural network for labeling a detected set of 2D points. I've generated a random set of 3D points within a certain radius, all of which lie on a flat plane (Y value is 0)...
- 1
2
votes
1
answer
46
views
Training an RL model with an environment where some of the variables do not change as a result of the agent actions
Typically training an RL model requires an action and an observation space, and the agent learns how its actions affect the observations. Even though there are cases where the observation space ...
- 121
1
vote
1
answer
44
views
Find maximum value of unknown functions f(x,y)=z using reinforcement learning & neural network
is it possible to train a neural network to find the global maximum value of unknown functions like f(x,y)=z with reinforcement learning?
Up until now I had only had experience with simple ...
- 13
7
votes
4
answers
4k
views
Do neural network weights need to add up to one?
The idea that weights determine how much influence each input value from the current layer will have when calculating the input to the following layer reminds me of when my professors would say that ...
- 181
1
vote
1
answer
64
views
How to generate a 3D model from only 1 image?
I'm posting this on the AI stack exchange because even though this can be solved with a "regular" complex and sophisticated algorithm, it seems that trying to generate something for which ...
- 11
0
votes
1
answer
23
views
Is It Reasonable to Get NaN Outputs With Identity Activation Functions
I made my own Neural Network from scratch in unity with C# and I am using it as DQN. I set up my network which has 4 layers: 9 input values, 20 nodes in the second layer, 15 nodes in the third layer, ...
- 25
2
votes
1
answer
104
views
Calculating mutual information between layer outputs and targets in a neural network
I've seen in several papers that it is possible to calculate the mutual information between a layer's outputs and the desired outputs. For example:
Source: https://www.ncbi.nlm.nih.gov/pmc/articles/...
- 73
-2
votes
2
answers
58
views
Size of a neural network [closed]
For any type of ANN, is there a common formula which can be used to calculate size of a neural network?
3
votes
0
answers
64
views
Relation between SDE diffusion and DDPM/DDIM
Background & Definitions
In DDPM, the diffusion backward step is described as follows (where $z\sim \mathcal{N}(0,I)$ and $x_{T}\sim \mathcal{N}(0,I)$):
and in DDIM we have
while in the SDE ...
- 79
0
votes
1
answer
31
views
How can BERT/Transformer models accept input batches of different sizes?
I understand that all inputs in a batch need to be of the same size. However, it seems BERT/Transformers models can accept batches with different sizes as input.
How is that possible? I thought we ...
- 1
0
votes
0
answers
23
views
How to change network that works on a subset of the input?
To clarify my question, I'll take an example: let's say you're trying to classify pictures, by determining whether there's a dog or not in the picture. Let's assume the pictures to be 1000x1000 pixels....
0
votes
1
answer
56
views
Why don't people use their own random noise to counter adversarial attacks on computer vision systems?
Why couldn't you take the image an AI is given and apply several different random noise filters to the image and take the democratically most common response and use that for the output of the AI. As ...
- 103
0
votes
0
answers
25
views
Optimal Neural Network Structure for 'Crystal Rush': Single vs. Modular Networks?
Problem
I'm attempting to design a neural network for the bot programming game Crystal Rush. Given the game's complexity, I anticipate needing a vast neural network to manage bot movements, resource ...
user75399
0
votes
0
answers
23
views
Interpreting TensorFlow playground simple neural network
The hidden layer neuron output images are meaningful to me. But I don't understand how the model can output any number except zero for the III quadrant when all hidden layer neurons output 0 for any ...
- 101
2
votes
0
answers
25
views
What's the result of multiple neurons using ReLU activation function?
This question comes from a doubt that I recently had on an amazing book called "Neural Networks from Scratch With Python by Harrison Kinsley & Daniel Kukieła"
Let's suppose that I have ...
- 21
0
votes
0
answers
19
views
Create samples out of documents for Causal Language Modelling
I want to create an input source for Causal Language model using Llama 2 model in hugging face. I have a set of documents which are scraped from a specific website and want to fine-tune on them. Each ...
- 1
1
vote
1
answer
44
views
Convolution layer, with biases too?
We already know that the kernel slides around the image, multiplying the pixels with the parameters, so, what if additionally, we also have a kernel slide around the image and add values(different ...
- 11
0
votes
0
answers
18
views
Is it possible to combine SGD with an unsupervised learning approach effectively
Before I undertake quite a large project I would like to clarify whether my idea for training a multi-layer neural network will work. I plan to make an AI that can land a rocket from randomly ...
- 1
0
votes
0
answers
5
views
Multi Output Regression but num of objects to predict vary per sample
so recently I came across a problem of predicting the positions of objects from a pulse wave. My biggest concern here is that for each data sample, the number of objects varies. I know that this ...
- 1
0
votes
1
answer
25
views
CNN how to measure the amount of FPS that can be processed?
This is my first question in the AI stack exchange. I want to ask about how to measure how many FPS can a CNN model process during real time detection. I am working on a real time detection system ...
- 3
1
vote
1
answer
61
views
How Can I Backpropagate My Network with PPO
I am trying to implement PPO to my reinforcement agents. I have a classic neural network that represents the policy. I didn't quite understand how the PPO updates the network, according to what? There ...
- 25
0
votes
1
answer
71
views
Confusion over taking gradients in Variational Autoencoders (VAE)
I am confused as to when to hold certain parameters constant in a VAE. I will explain with a concrete example.
We can write $\operatorname{ELBO}(\phi, \theta) = \mathbb{E}_{q_{\phi}(z)}\left[\log
\...
- 23
0
votes
0
answers
21
views
Implementing momentum is causing calculation exceptions/errors
I am developing my own neural network in order to learn about how they work. I am implementing via C++ and the Eigen library (for matrix multiplication). I have a working implementation that seems to ...
0
votes
1
answer
46
views
Easiest way to train a neural-network with neurons that deviate from $f_{nl}(x \cdot A)$
I want to model how a neural network would behave for a system of input-output devices that are only approximately similar to a neuron. I think I have a resonable plan for how to do this, but I'm ...
- 221
3
votes
0
answers
53
views
how do you train a neural network to determine shortest path in a 4-node graph
Suppose I have the following graph:
...
- 131
0
votes
1
answer
54
views
Are All the Target Q Values in DQN same?
So I am trying to understand and make a DQN. But I didn't understand a part. So basically state's Q values computed with the network and the target Q values will also compute with a target network ...
- 25
0
votes
1
answer
22
views
Fully Connected Neural Networks That Act Partially Connected
This might sound like a slightly odd question, but is there a good technique to make a fully connected neural network act as if it's partially connected - by having certain edges not propagate a ...
0
votes
0
answers
28
views
Using Neural Networks to Measure Distances on Google Maps
This is a problem I have been wondering about for a while.
Suppose I have the following screenshot from Google Maps:
My Question: I want to find out a series of longitude/latitude coordinates that ...
- 269
1
vote
2
answers
82
views
Maximize a scoring function within the latent space of a generative model
Given a generative model, G, trained on a dataset D. This generative model can be either GAN or Diffusion based. Supposed each sample, x_i, generated by G, can be evaluated by a readily available ...
- 81
0
votes
1
answer
34
views
Can neural-networks solve a system of non-linear algebraic equation
I am currently doing research as a PhD student in theoretical physics. Currently we are calculating physical quantities described by coupled non-linear algebraic systems of equations. These equations ...
- 1
3
votes
2
answers
582
views
Implementing a GAN with control over the output class
I am trying to accomplish the reverse of the typical MNIST in machine learning using a GAN - instead of predicting a number from an image of a digit, I want to reconstruct an image of a digit from a ...
- 143
0
votes
0
answers
20
views
What machine learning methods can be used to minimise the difference to a target 2D boolean matrix with differing sizes?
I have a problem whereby given an input, the difference that is generated from the model to a target 2D boolean matrix needs to be minimised.
The target boolean matrix can be of varying shape, i.e: 10 ...
- 101
4
votes
2
answers
121
views
Why are some Neural Networks more forgiving on Quantization?
I know this might be a bit general question and concerning a rather active research field, much beyond my expertise, but I do believe there're some answers.
The use of NN parameters quantization can ...
- 91
0
votes
1
answer
50
views
Is validation data needed with generated training data?
I have two systems, a system A that generates some data X and a system B that calculates ...
- 113