Questions tagged [neural-networks]
For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.
2,610
questions
0
votes
0
answers
6
views
How to train a simple network to predict 2D location without knowing ground truth?
There is a 2D table of known dimension width=4cm, length=6cm. We can place a disc(diameter=0.5cm) at position (x,y).
If the disc stays on the table, there is a function (evaluate_position) that says ...
- 13
0
votes
0
answers
27
views
Model suggestion for AI based scaling
We are exploring the idea of scaling elements within a UI container based on the given size.
The container is represented by a json object, for example:
...
- 101
0
votes
0
answers
21
views
How to train continuous probability distributions as output from a neural network?
I'm training time series models on numerical forecasting, and I'm seeing inherent difficulty in modeling the uncertainty of the values. Time series forecasting generally has a pattern of uncertainty ...
0
votes
0
answers
25
views
AI / Machine Learning Project Collaboration Community?
I'm working on a time series prediction project using neural nets and looking for collaborators. Does someone know of a AI/Machine Learning community that is projects driven, where members can find ...
- 11
0
votes
0
answers
11
views
1D CNN with Single vs. Two Channels for Number Image Recognition
I am taking images of numbers as input, in a convolutional neural network and building a model to predict the number.
In particular, I am building a one dimensional convolutional neural network with ...
- 121
0
votes
0
answers
7
views
Can you iteratively freeze and unfreeze parts of a neural network for efficient training?
I know you can do efficient training by freezing parts of a NN, but is there any work done where part 1 of a NN is frozen and part 2 is trained, and then part 2 is frozen and part 1 is trained?
- 203
0
votes
0
answers
11
views
Can low rank adaptation (Lora) speed up (or reduce memory) for fine-tuning on small neural networks?
Lora has many examples of speeding up fine-tuning on large language models, but is it possible to use Lora on small neural networks?
As my understanding, from this blog,
as the number of neural ...
0
votes
0
answers
18
views
High-resolution 3D human digitization from a single image or multiple images
Reference
A 2020 tool converts a single image or multiple images of a full body to a 3D OBJ file. Internally, it creates the SDF - signed distance field - and then uses the marching cubes to generate ...
- 141
0
votes
2
answers
70
views
Which type of ML algorithm takes the least amount of time for training?
I am doing research on proteins.
I have 17,000 *.CSV files on my hard disk. These files represent the chains of proteins. I want to use these ...
- 319
11
votes
2
answers
5k
views
Can you train a neural network by simply giving it ratings each time it runs?
I am currently trying to train a bot for a game I am creating. It is a 2d game with a complex map made of various shapes. The bot and character shoot bullets that are capable of ricocheting. The ...
- 113
1
vote
1
answer
69
views
Is it possible that an RLNN generates actions by itself based on the info and observation state provided by the environment?
Is it possible that an RLNN generates actions by itself based on the info and observation state provided by the environment? For example a function G(s) where it takes in the state as input and ...
- 13
0
votes
0
answers
23
views
Do neural networks have a perception of space, regardless of dimensionality?
Suppose I have a model M which outputs a three-dimensional tensor of size 3x3x3.
I have another model N which outputs a one-dimensional tensor of size 27.
Train both models on some arbitrary objective ...
0
votes
0
answers
11
views
How to knowing number of clusters when using SOM?
SOM uses neural network. The output layer of SOM should be neurons position. As the model is training, neuron's position started to moving to the closer of centroid of clusters.
The output layer was ...
0
votes
1
answer
30
views
Literature suggestions for transformers
What are the best educational sources for learning about transformers, what is the go to literature for a mathematician who considers themself a beginner in the subject?
Books, lecture notes, research ...
- 101
0
votes
0
answers
19
views
How to Optimize model for faster Training
Below is the forward pass of my model. The input x is split about time-dimension (last-dim) which has indices till 250. Below is the code...
...
0
votes
0
answers
23
views
Influence of Unused FFN on Model Accuracy in PyTorch
I am encountering a peculiar issue with my PyTorch model where the presence of an initialized but unused FeedForward Network (FFN) affects the model's accuracy. Specifically, when the FFN is ...
- 1
2
votes
2
answers
422
views
Is it easier to use back-propagation or genetic algorithms to teach an artificial intelligence?
I am making a very simple neural network for a school project, and I would like to know what the best and easiest way to "teach" a neural network would be. From what I know, backpropagation ...
- 23
1
vote
0
answers
36
views
History of Neural Networks and Deep Learning
I'm interested in learning about the history of neural networks and deep learning. I've been reading about the field and am familiar with many of the developments since the 1950s. Is there a textbook,...
- 11
0
votes
1
answer
25
views
How do I input multi-channel Numpy array to U-net for semantic segmentation
I had lidar 3D point cloud data from semantckitti. I want to perform Semantic Segmentation on the data using U-Net. I converted the 3d point cloud data into 2D using spherical conversion and saved the ...
0
votes
0
answers
13
views
Simplifying equation with maximum inside absolute sign
This is the formula I am working on.
$|a\times \max(x,0)+b\times \max(y,0)+c \times \max(z,0)|$
Is it possible to take the maximum out of the absolute sign?
0
votes
1
answer
53
views
Is it possible to simplify max(max(a,b),c)?
Like the title says, is there any way to simplify the piece wise linear function of max(max(a,b),c) into some linear combination of max(a,b), max(b,c), max(a,c)?
0
votes
1
answer
20
views
Loss goes down but never below a certain treshold
I made a neural network in C#, I observe the loss goes down but never below a certain treshold.
This is XOR function error graph:
(The graph is every 4 samples, so for all the 4 possible combinations ...
- 301
0
votes
0
answers
11
views
Adding Feature in HGNN to Count Connections to Types of Nodes
So I'm making a HGNN currently in which the number of connections a node has to other nodes of a certain type matters. Its a social network, so I care about how many person-person connections a person ...
0
votes
1
answer
37
views
Is there a way to convert 3 layer network to 2 layer network with specific math formula?
Currently, my professor asked me to find an explicit formula to convert 3 layer network to 2 layer network. I've read some paper about the general properties of neural network, how its complexity is ...
0
votes
1
answer
27
views
CNN multioutput regression architecture modification
I am working on a regression task where the model has to predict two values at the same time. The idea is that the dataset consists of 16 features, where the first 8 features represent the first value ...
0
votes
0
answers
18
views
Is my C# Adam implementation correct?
I have some doubt because I incurred in different papers proposing different implementations. Also implementations on opensource projects looks different.
In example there is a C++ library that ...
- 301
1
vote
0
answers
32
views
Is there any standardized notation for drawing neural network diagrams?
Is there any standardized notation for drawing neural network diagrams? For example, for circuits there is a universal set of symbols used to draw different types of circuits why not for neural ...
- 33
0
votes
0
answers
8
views
Finding invariant feature areas within representation vector for each meta-class/group?
I have pairs of images which are not the same class, but are from the same meta-class/group. I have a standard CNN which produces a representation for each sample.
If I have several pairs of images ...
- 11
0
votes
1
answer
42
views
Total loss in backpropagation
I'd say I have some understanding of backpropagation, however I am not really sure of the total loss being calculated.
Let us take the example below:
After 1 forward pass when I have to update the ...
- 9
2
votes
1
answer
54
views
Is it possible to use Mini-Batches with Adam optimization?
Is it possible/advised to use Mini-Batch like accumulation with Adam optimization?
How would that works?
Do I accumulate the loss function for each sample in the batch and then run Adam, or should I ...
- 301
0
votes
0
answers
23
views
Validating bundles of XMLs with inconsistent structure
The problem:
We have a large number of XML bundles, and each bundle needs to meet certain criteria to be considered valid; specifically, certain values (or types of values) should belong to certain ...
0
votes
0
answers
11
views
Predicition of future batches in time series
i am working with neural networks and i want to predict the time series further ahead. I did a course on neural networks where this kind of problem is faced. But i dont really understand how it works.
...
0
votes
0
answers
13
views
19 characters to one prediction
I want to read the KELM natural sentences 19 characters at a time and predict the 20th character. So i made AN enumerator that returns that moving window of characters and already encode each of the ...
- 301
1
vote
0
answers
39
views
What is the best way to train a neural network with a variable number of inputs?
Suppose I have a neural network with 5 inputs: [A,B,C,D,E]
There is only 1 output. The expected accuracy of the model should increase when all 5 inputs are ...
- 11
0
votes
1
answer
63
views
How do I know that my dataset is good enough for training a neural network?
Suppose I have a clean (no outliers and normalized) dataset for training a neural network.
The training process is expected to take almost a week. So, before I start training, I want to know if this ...
- 319
2
votes
1
answer
61
views
What are these special "AI chips" actually used for?
To make an AI / LLM, like ChatGPT, you need two things:
To create the LLM. This includes training it, etc. Very expensive from computation perspective.
Run the LLM to answer user queries. For ...
- 121
4
votes
3
answers
147
views
In the VAE, why is $z \sim \mathcal{N}(\mu, \sigma^2)$ equivalent to $z = \mu + \sigma \odot \epsilon$?
In the reparameterization trick of a Variational Autoencoder (VAE), instead of sampling noise $z$ from $z \sim \mathcal{N}(\mu, \sigma^2)$, we can use a different method: $z = \mu + \sigma \odot \...
- 153
1
vote
1
answer
37
views
Fine tuning or just feature extraction or both using Roberta?
I'm reading a program that use the pre-trained Roberta model (roberta-base). The code first extracts word embeddings from each caption in the batch, using the last hidden state of the Roberta model. ...
- 153
0
votes
0
answers
11
views
What model is good for learning both within and across categories?
How can I incorporate both general trends and subcategory-specific trends into a model?
Let's say I am predicting factors that affect import volume, for example. There are many industries which have ...
- 113
0
votes
1
answer
13
views
0
votes
2
answers
91
views
What is the difference between densenet and resnet?
Is the only difference between the two how the skip connection is combined? Resnet combines skip connections through addition and Densenet through concatenating.
The Densenet paper appears to be ...
- 203
0
votes
0
answers
27
views
How exactly is backquery supposed to work in this situation?
Context: This code is based on a 3 layer fully connected neural network trained on had written numbers 0-9. This back query code will then take in an output value of 0.99,0.01,0.01,0.01,0.01,0.01,0.01,...
- 95
1
vote
0
answers
44
views
Why do mix models work?
Is there research on why models mixes work? One would expect that averaging the weights of two models would produce garbage, but many models mixes created by amateurs show that they not only work, but ...
- 310
1
vote
2
answers
64
views
Books that contains exclusively math problems/assignments in Deep Learning & Neural Networks
I am doing a Deep Learning Course.Suggest some books that contains exclusively math problems/assignments in Deep Learning & Neural Networks. I can understand that majority of the replies suggest &...
- 13
1
vote
1
answer
70
views
Marking object on a map from the image
I have been researching if there are any existing machine learning models that would help mark objects (for example: cars) on the map having only image, camera location, and camera orientation.
For ...
- 19
0
votes
1
answer
27
views
Gradually increasing CPU load on using sentence embeddings model with kmeans
I am having a ML based production application, using flask, deployed on GCP server using gunicorn workers. In each incoming request, a text sentence is received.
It is using sentence transformers (All-...
- 1
0
votes
1
answer
46
views
Using a neural network to predict a single discrete number
I am working on a project that uses a categorical and non categorical dataset to predict a Success/Fail rate. Each entry/data point has multiple categorical and numerical parameters tied to a rate.
We ...
0
votes
1
answer
63
views
How LSTM really decide what to forget and not?
Currently, I am learning about LSTM, and I understand the intuition behind it, such as how forget gate works (sigmoid function yields a value between 0 and 1; if it is 0 it "completely" ...
- 3
1
vote
3
answers
70
views
Does transformers' self-attention mechanism process tokens independently, or entire sequence at a time?
About attention: the Query, Key and Value vectors (before the linear transformations) are just the entire sequence, that is being inputted, or just each token? Chat-GPT nor Youtube didn't give me a ...
0
votes
0
answers
63
views
What are the differences between Inception Score and Fréchet Inception Distance?
From the articles I've read about image generation using GANs, the Inception Score measures two things simultaneously: the variety of images (diversity) and the distinct quality of each image. Does ...
- 153