Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
0 votes
0 answers
3 views

Why are thresholds in NN so important?

I am currently working on a paper on the approximation properties of NN. One part of this topic is of course the universal approximation theorem(UAT), which gets discussed in various papers with ...
Lopsio's user avatar
  • 1
0 votes
0 answers
3 views

How to get Llama-2 Rotary Embeddings?

I want to get the Llama-2 rotary embeddings. I do print(model) and get the following output: In the picture I highlight the rotary embeddings. How can get the ...
Christian01's user avatar
0 votes
0 answers
29 views

VAE for Motion Sequence Generation - Convergence Issue with Scheduled Sampling

I have implemented a Variational Autoencoder (VAE) in PyTorch for motion sequence generation using human pose data (joint angles and angular velocities in radians) from the CMU dataset. The VAE ...
RTn's user avatar
  • 1
-1 votes
0 answers
12 views

Simulate all the graphs and figures attached in the article related to VCSEL laser in the MATLAB

Please simulate all or some of the diagrams and figures present in this article related to VCSEL lasers in MATLAB software. The attached paper for download is available at the following link. https://...
Ezzatallah Bakhshkandi's user avatar
0 votes
0 answers
17 views

Cooperation between pattern recognition and neural networks for better design of AI

I continue to try to understand how a human brain works and how we can use it to set up AI which would copy the behavior of a actual person. A few days ago I uploaded a question where I described a ...
Cerise's user avatar
  • 1
0 votes
0 answers
4 views

What is the policy model in RLHF for LLMs?

What is the policy model doing explicitly in an LLM with RLHF setup? From my understanding, LLMs generate in a way that is no different from any of their predecessors: beam search decoding, ...
information_interchange's user avatar
0 votes
0 answers
27 views

Why are convolutions and pooling described as layers in a network?

Whenever I look at resources on convolutions and max-pooling in CNNs they always seem to describe these algorithms as being part of the network - a preliminary set of layers before the main ...
Gamaray's user avatar
  • 13
0 votes
0 answers
39 views

Why does my loss function fluctuate so much?

I have a loss function that I'm trying to maximise using a neural network. While it does appear to increase and plateau over the training, it does so in a very "noisy" manner, spiking up and ...
VJ123's user avatar
  • 73
1 vote
2 answers
57 views

Tips and tricks when training a very large language model?

Have never trained a (very) large language model, so I am wondering if the process is the same as training a (regular) language model, i.e. you prepare the data, set up the architecture, ...
information_interchange's user avatar
-1 votes
0 answers
17 views

If I have a pre trained neural network with all bias and weight files, where and how can I check it's accuracy

Tell me the software I can use to check the accuracy of the neural network and how can I use the software to implement the pre trained layer files in verilog
Soham Bhakat's user avatar
0 votes
2 answers
32 views

Must I "prime" my normalizer with the same data I trained it with in order to use it?

I trained a Keras Network. During training, I would first initialize a normalizer from the values in the entire dataset, then partition into train, test and validation datasets. After partitioning, I ...
Pittsburgh DBA's user avatar
0 votes
1 answer
25 views

Can a concept/feature be represented using more than one layer of a Neural Network?

I was reading Goodfellow. At the start of the text it was mentioned that there are two ways to represent depth of a deep neural network. One is using the depth of the computation graph and the other ...
rsonx's user avatar
  • 101
0 votes
0 answers
42 views

Are we missing correlation to achieve a artificial intelligence comparable to humans?

I don't think neural networks are enough to copy human intelligence because in the end a machine only using a neural network to decide on something still is unable to recognize patterns. So I was ...
Cerise's user avatar
  • 1
0 votes
0 answers
39 views

Optimizing Stop Loss Percentage for a Specific Model Based on Stock Price to Maximize Expected Value

I'm fine-tuning a specific trading model, and a crucial parameter I'm keen on optimizing is the stop loss percentage. The primary objective is to maximize the Expected Value (EV), formulated as: $$EV =...
David's user avatar
  • 11
0 votes
0 answers
21 views

Pixel-wise regression only focus on edge

I am trying to use unet to learn pixel-wise regression from one image to one groundtruth with the same image size. The network seems to focus too much on the edge of the image, and it does not learn ...
K.Nguyen's user avatar
0 votes
0 answers
11 views

Challenges in Developing AI Algorithms for X-ray Image Analysis on Large Datasets

Hello everyone, I'm currently working on a research project involving X-ray imaging and the development of AI algorithms to detect diseases from large ...
kibromhft's user avatar
  • 134
0 votes
0 answers
14 views

Machine learning network design for 2D point prediction labeling

I'm looking to develop a basic neural network for labeling a detected set of 2D points. I've generated a random set of 3D points within a certain radius, all of which lie on a flat plane (Y value is 0)...
colyton's user avatar
2 votes
1 answer
46 views

Training an RL model with an environment where some of the variables do not change as a result of the agent actions

Typically training an RL model requires an action and an observation space, and the agent learns how its actions affect the observations. Even though there are cases where the observation space ...
Jesuspc's user avatar
  • 121
1 vote
1 answer
44 views

Find maximum value of unknown functions f(x,y)=z using reinforcement learning & neural network

is it possible to train a neural network to find the global maximum value of unknown functions like f(x,y)=z with reinforcement learning? Up until now I had only had experience with simple ...
Bubble's user avatar
  • 13
7 votes
4 answers
4k views

Do neural network weights need to add up to one?

The idea that weights determine how much influence each input value from the current layer will have when calculating the input to the following layer reminds me of when my professors would say that ...
Garrett's user avatar
  • 181
1 vote
1 answer
64 views

How to generate a 3D model from only 1 image?

I'm posting this on the AI stack exchange because even though this can be solved with a "regular" complex and sophisticated algorithm, it seems that trying to generate something for which ...
OGOG's user avatar
  • 11
0 votes
1 answer
23 views

Is It Reasonable to Get NaN Outputs With Identity Activation Functions

I made my own Neural Network from scratch in unity with C# and I am using it as DQN. I set up my network which has 4 layers: 9 input values, 20 nodes in the second layer, 15 nodes in the third layer, ...
Ege's user avatar
  • 25
2 votes
1 answer
104 views

Calculating mutual information between layer outputs and targets in a neural network

I've seen in several papers that it is possible to calculate the mutual information between a layer's outputs and the desired outputs. For example: Source: https://www.ncbi.nlm.nih.gov/pmc/articles/...
VJ123's user avatar
  • 73
-2 votes
2 answers
58 views

Size of a neural network [closed]

For any type of ANN, is there a common formula which can be used to calculate size of a neural network?
Indika Alahapperuma's user avatar
3 votes
0 answers
64 views

Relation between SDE diffusion and DDPM/DDIM

Background & Definitions In DDPM, the diffusion backward step is described as follows (where $z\sim \mathcal{N}(0,I)$ and $x_{T}\sim \mathcal{N}(0,I)$): and in DDIM we have while in the SDE ...
snatchysquid's user avatar
0 votes
1 answer
31 views

How can BERT/Transformer models accept input batches of different sizes?

I understand that all inputs in a batch need to be of the same size. However, it seems BERT/Transformers models can accept batches with different sizes as input. How is that possible? I thought we ...
PS1's user avatar
  • 1
0 votes
0 answers
23 views

How to change network that works on a subset of the input?

To clarify my question, I'll take an example: let's say you're trying to classify pictures, by determining whether there's a dog or not in the picture. Let's assume the pictures to be 1000x1000 pixels....
WINTERSDORFF Raphael's user avatar
0 votes
1 answer
56 views

Why don't people use their own random noise to counter adversarial attacks on computer vision systems?

Why couldn't you take the image an AI is given and apply several different random noise filters to the image and take the democratically most common response and use that for the output of the AI. As ...
Ethan's user avatar
  • 103
0 votes
0 answers
25 views

Optimal Neural Network Structure for 'Crystal Rush': Single vs. Modular Networks?

Problem I'm attempting to design a neural network for the bot programming game Crystal Rush. Given the game's complexity, I anticipate needing a vast neural network to manage bot movements, resource ...
user avatar
0 votes
0 answers
23 views

Interpreting TensorFlow playground simple neural network

The hidden layer neuron output images are meaningful to me. But I don't understand how the model can output any number except zero for the III quadrant when all hidden layer neurons output 0 for any ...
DimitrijeCiric's user avatar
2 votes
0 answers
25 views

What's the result of multiple neurons using ReLU activation function?

This question comes from a doubt that I recently had on an amazing book called "Neural Networks from Scratch With Python by Harrison Kinsley & Daniel Kukieła" Let's suppose that I have ...
Tomorrow's user avatar
0 votes
0 answers
19 views

Create samples out of documents for Causal Language Modelling

I want to create an input source for Causal Language model using Llama 2 model in hugging face. I have a set of documents which are scraped from a specific website and want to fine-tune on them. Each ...
Dimits's user avatar
  • 1
1 vote
1 answer
44 views

Convolution layer, with biases too?

We already know that the kernel slides around the image, multiplying the pixels with the parameters, so, what if additionally, we also have a kernel slide around the image and add values(different ...
akzytr's user avatar
  • 11
0 votes
0 answers
18 views

Is it possible to combine SGD with an unsupervised learning approach effectively

Before I undertake quite a large project I would like to clarify whether my idea for training a multi-layer neural network will work. I plan to make an AI that can land a rocket from randomly ...
Gamaray's user avatar
0 votes
0 answers
5 views

Multi Output Regression but num of objects to predict vary per sample

so recently I came across a problem of predicting the positions of objects from a pulse wave. My biggest concern here is that for each data sample, the number of objects varies. I know that this ...
mim96's user avatar
  • 1
0 votes
1 answer
25 views

CNN how to measure the amount of FPS that can be processed?

This is my first question in the AI stack exchange. I want to ask about how to measure how many FPS can a CNN model process during real time detection. I am working on a real time detection system ...
Jo Sky's user avatar
  • 3
1 vote
1 answer
61 views

How Can I Backpropagate My Network with PPO

I am trying to implement PPO to my reinforcement agents. I have a classic neural network that represents the policy. I didn't quite understand how the PPO updates the network, according to what? There ...
Ege's user avatar
  • 25
0 votes
1 answer
71 views

Confusion over taking gradients in Variational Autoencoders (VAE)

I am confused as to when to hold certain parameters constant in a VAE. I will explain with a concrete example. We can write $\operatorname{ELBO}(\phi, \theta) = \mathbb{E}_{q_{\phi}(z)}\left[\log \...
Joel's user avatar
  • 23
0 votes
0 answers
21 views

Implementing momentum is causing calculation exceptions/errors

I am developing my own neural network in order to learn about how they work. I am implementing via C++ and the Eigen library (for matrix multiplication). I have a working implementation that seems to ...
user1311627's user avatar
0 votes
1 answer
46 views

Easiest way to train a neural-network with neurons that deviate from $f_{nl}(x \cdot A)$

I want to model how a neural network would behave for a system of input-output devices that are only approximately similar to a neuron. I think I have a resonable plan for how to do this, but I'm ...
Steven Sagona's user avatar
3 votes
0 answers
53 views

how do you train a neural network to determine shortest path in a 4-node graph

Suppose I have the following graph: ...
user1068636's user avatar
0 votes
1 answer
54 views

Are All the Target Q Values in DQN same?

So I am trying to understand and make a DQN. But I didn't understand a part. So basically state's Q values computed with the network and the target Q values will also compute with a target network ...
Ege's user avatar
  • 25
0 votes
1 answer
22 views

Fully Connected Neural Networks That Act Partially Connected

This might sound like a slightly odd question, but is there a good technique to make a fully connected neural network act as if it's partially connected - by having certain edges not propagate a ...
Steve Harding's user avatar
0 votes
0 answers
28 views

Using Neural Networks to Measure Distances on Google Maps

This is a problem I have been wondering about for a while. Suppose I have the following screenshot from Google Maps: My Question: I want to find out a series of longitude/latitude coordinates that ...
stats_noob's user avatar
1 vote
2 answers
82 views

Maximize a scoring function within the latent space of a generative model

Given a generative model, G, trained on a dataset D. This generative model can be either GAN or Diffusion based. Supposed each sample, x_i, generated by G, can be evaluated by a readily available ...
terenceflow's user avatar
0 votes
1 answer
34 views

Can neural-networks solve a system of non-linear algebraic equation

I am currently doing research as a PhD student in theoretical physics. Currently we are calculating physical quantities described by coupled non-linear algebraic systems of equations. These equations ...
SaFeHe's user avatar
  • 1
3 votes
2 answers
582 views

Implementing a GAN with control over the output class

I am trying to accomplish the reverse of the typical MNIST in machine learning using a GAN - instead of predicting a number from an image of a digit, I want to reconstruct an image of a digit from a ...
JS4137's user avatar
  • 143
0 votes
0 answers
20 views

What machine learning methods can be used to minimise the difference to a target 2D boolean matrix with differing sizes?

I have a problem whereby given an input, the difference that is generated from the model to a target 2D boolean matrix needs to be minimised. The target boolean matrix can be of varying shape, i.e: 10 ...
Jag's user avatar
  • 101
4 votes
2 answers
121 views

Why are some Neural Networks more forgiving on Quantization?

I know this might be a bit general question and concerning a rather active research field, much beyond my expertise, but I do believe there're some answers. The use of NN parameters quantization can ...
edmz's user avatar
  • 91
0 votes
1 answer
50 views

Is validation data needed with generated training data?

I have two systems, a system A that generates some data X and a system B that calculates ...
n-l-i's user avatar
  • 113

1
2 3 4 5
51