Questions tagged [neural-networks]
For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.
2,541
questions
0
votes
0
answers
9
views
Prefix tuning in LLM uses learnable vectors to fine tune the model
I would like to implement a new architecture for Transformer.
Below description is my thought.
Prefix tuning in LLM uses learnable vectors to fine tune the model.
Is there a way to use the output ...
- 1
0
votes
0
answers
10
views
Is size of trained model on disk a good measure of model complexity?
I am writing a research paper on my own custom CNN model for image classification. I am comparing my model architecture with pre-trained architectures, like DenseNet121 and InceptionV3.
I want to ...
- 13
0
votes
0
answers
7
views
Could an analysis of GPT4's WAIS score be published?
Similar to this: https://www.scientificamerican.com/article/i-gave-chatgpt-an-iq-test-heres-what-i-discovered/
But more detailed and in depth (subtest breakdown, including image analysis, etc.), WAIS-...
- 113
1
vote
0
answers
28
views
Is there a state-based approach to neural networks?
It occurred to me that neural networks operate like straight-line programs: an input is processed once by the layers, and an output is always immediately formed, in the same amount of time for each ...
- 111
1
vote
1
answer
16
views
Can I do incremental learning with different loss function in neural networks?
I have a saved tensorflow neural network model. I was wondering if it's possible to incrementally train the model but with different nt loss function.
- 111
1
vote
0
answers
14
views
Looking for a way to train a model to learn optimal parameters/hyperparameters of clustering
I have 5000 docs, each is a review. For each review, i'm plotting the sentences in a semantic dimension. Now, I'm applying clustering to these points for each review. The success of my model depends ...
- 11
1
vote
1
answer
92
views
Why in one article to calcule the delta uses the weights of current layer, while in another article uses the weights of neuron inputs of next layer?
Article 1:
https://pyimagesearch.com/2021/05/06/backpropagation-from-scratch-with-python/
Article 2:
https://machinelearningmastery.com/implement-backpropagation-algorithm-scratch-python/
I was ...
- 233
0
votes
0
answers
17
views
Can't get a correct accuracy on tabular data using deep learning
This is my first message here, and I would like to seek some assistance !
I have a technical test for a job that I really want, and I have 10 days to complete it. I've attempted to work on it, but I'm ...
0
votes
1
answer
51
views
In multilayer perceptron neural networks, the more complex a dataset is, the more difficult it will be to find adequate initial random weights?
I know that neural networks don't need to have exactly the same weights, and that different networks trained on the same problem may result in different final weights, but produce comparable results. ...
- 233
2
votes
1
answer
56
views
How can I deal with random weights initialisation when predicting a time-series sine function?
I am training a simple RNN model in keras to predict a time series.
The time series I am considering is just a sine function
The task to solve is the following:
...
- 123
0
votes
0
answers
20
views
Base rate effects on object detection neural networks?
Suppose that we are training an neural network for object detection. It is natural to train the neural network on images that contain the object in question. However, this might have the effect of ...
- 171
0
votes
0
answers
28
views
How was the word2vec model trained?
Let's take the CBOW (continuous bag of words) model as the example.
Suppose that, there are $c$ context words, each of which is a one-hot encoding vector. So the total number of elements of input ...
- 1
0
votes
0
answers
32
views
Why is masked self attention necessary on GPT decoder
I'm currently reading the paper for the first GPT model and I'm confused about why masked self attention is necessary and I haven’t found any good answers online.
The consensus seems to be that we don'...
0
votes
1
answer
43
views
What is the potential issue of nested neural networks
everyone. I am working on a nested neural network architecture. For the sake of better understanding my question, simply assume the loss is
$L = G(k’) - H(k'')$
where $G$ and $H$ are two functions we ...
1
vote
1
answer
40
views
If I manually trained a multilayer percetron neural network, always following exactly the same steps meticulously, would I always get the same result?
Reference:
https://home.agh.edu.pl/~vlsi/AI/backp_t_en/backprop.html
If I trained a multilayer percetron neural network manually, following exactly the backpropagation steps described in the article, ...
- 233
0
votes
0
answers
20
views
When to know if I am "on the right track" for a CNN architecture
Context
Very new to CNNs and ML in general. I am building a simple binary image segmentation network for generating black and white image masks (white pixels = desired object; black pixels = all else)....
0
votes
0
answers
14
views
Keeping the grad of the tensor when using inverse_transform
To train a network, I scaled both the input and outputs of my data like the following:
...
- 1
0
votes
1
answer
70
views
Why is it called multi-headed attention?
Why do we call the attention layer in transformers multi-headed attention when in practice all the attention matrices from different heads (W,K,V) for a single layer are concatenated to perform the ...
- 103
0
votes
0
answers
24
views
Neural Networks Model Predictions on Same Sample
I have built a tensorflow model to reconstruct the given input data sample by using an Autoencoder to predict the Missingness in the sample. However I do not have enough data sample where i don't have ...
1
vote
0
answers
44
views
The more a MLP neural network is trained, the value of correct class should increasingly closer to 1, and the other classes to 0? Is this expected?
I am using a multilayer perceptron network with the Iris dataset. This network has 3 output neurons (to represent the 3 classes of Iris, that is, Iris Versicolor, Iris Virginia and Iris Setosa). ...
- 233
0
votes
0
answers
17
views
What kind of learning architecture can I use to solve a set of nonlinear thermodynamic equations?
I am in the business of solving equations based in thermodynamics. Technically speaking, I am trying to solve for the binodals for a ternary system. The binodal is a curve in a phase diagram.
...
- 101
2
votes
1
answer
1k
views
Neural network for game
There is a game for two, which is an NxN field (always the same size). Players take turns. The first player's goal is to connect the two points (not necessarily at the corners) given on this field. ...
- 31
2
votes
1
answer
39
views
An Adaline neuron can solve problems that are not linearly separable?
https://en.wikipedia.org/wiki/ADALINE
I was confused about this because, for example, the XOR problem is not linearly separable, and a simple Perceptron obviously cannot solve it, so we would need a ...
- 233
1
vote
1
answer
33
views
Can I train a Madaline network, just training each neuron individually, and then combining the output of all of them to be the classification result?
I've heard that Madaline networks generally have a single hidden layer, which have multiple Adaline units, where each Adaline receives different subsets of the input data. And the output of the ...
- 233
0
votes
1
answer
28
views
Is Madaline neural network and ther train almost the same as a multilayer perceptron(MLP)? what's the difference?
https://www.geeksforgeeks.org/adaline-and-madaline-network/
Is the Madaline neural network and its training mode almost the same or identical to a multilayer perceptron (MLP)? what's the difference?
- 233
0
votes
0
answers
22
views
What are general techniques of structuring an image classification Neural Network for very large numbers of output classes?
I am aware of Neural Networks that have 100K+ classes and I would like to build one myself (yes, I have lots of training data) but I am unsure which technique to use because most of the nets I have ...
0
votes
0
answers
22
views
Encoding categorical data with "many" unique values in neural network
I am new to machine learning, in fact, I am implementing my first deep neural network from scratch without any framework.
The dataset has 3500 rows, and 4 categorial columns of which two have about ...
- 113
0
votes
0
answers
26
views
In multilayer perceptron neural networks, is normalization required? or is optional? Why the network only seem to work when I use normalization?
I followed the steps in the following article (which teaches how to program a multilayer perceptron neural network from scratch in Python): https://machinelearningmastery.com/implement-backpropagation-...
- 233
0
votes
0
answers
11
views
How to implement a link constraint between the weights/bias of a neural network?
I'm new to the forum, and although I've tried my best, I hope not to write a duplicate post.
Here's the background:
I'm trying to build a neural network from scratch where certain weights are related ...
0
votes
0
answers
31
views
Multiple text inputs to neural network
How do I best input multiple text fields in a neural network for classification? For example, each data point has a title and an abstract:
...
- 1
0
votes
0
answers
14
views
Sampling dollar bars for ML model of multiple tickers
I have a Neural Network model that provides predictions for the future returns of a portfolio comprising stocks and cryptocurrencies. The original model operates on standard time bars (sampled at a ...
- 123
0
votes
0
answers
22
views
Deep learning to identify and classify first and last names into gender, age range and probable region of origin [closed]
I have a bit of a question as to how I could architect a pipeline of deep learning for the following use case:
gender identification from a full name (joseph is male, josephine is female, sasha is ...
- 1
0
votes
0
answers
11
views
Hopfield networks with shared neurons
Has there been research or even a thorough analysis of several Hopfield networks coupled by sets of shared neurons? Would this work? What about their respective storage capacities and robustness? Any ...
1
vote
1
answer
41
views
How do I train a CNN on multiple images that have the same shape?
I want to train a convolutional neural network on multiple input images. My image is 240x360 and is in RGB. Therefore my input image has a shape of 3x240x360. Now I want to use multiple images of the ...
- 61
0
votes
0
answers
17
views
Does the training time of a neural network increase more if we add a layer at the beginning or at the end?
Let's consider a fixed NN architecture, dataset and hardware. We add a layer, either at the beginning or at the end of the NN. In which case the training time will increase more? Intuitively, I ...
- 194
0
votes
0
answers
13
views
How can a RNN with 256 cells accept a input of any size?
I built a 3 layered RNN model with 256 cells each using torch. Input feature size is set to 40. Below give a basic Idea on the model.
...
1
vote
0
answers
39
views
Are there neural networks that compute weights dynamically based on geometric attributes of neurons?
I am interested in exploring neural network architectures where the weights are not stored but are computed dynamically based on certain attributes or "dimensions" of the connected neurons.
...
0
votes
0
answers
24
views
Seeking methods to incorporate arbitrary actuator faults for Control Optimization
I am working on a problem where a control method, backed by a Neural Network (NN), dictates the movement of a 1D actuator to influence a specific process. This actuator can move linearly within a set ...
0
votes
1
answer
65
views
Time Series Classification using Transformer Encoder
Lets say I have a collection of tensors, each tensor representing a time series with 64 points and 4 features. The dimension of each tensor would be [64,4]. I am trying to classify these series. For ...
0
votes
1
answer
39
views
Which algorithm do I need to use to calculate the error?
I am building a neural network for a AI whose goal is to learn to implement the XOR logical function.I have this code so far:
...
- 121
0
votes
0
answers
20
views
How to compensate the receptive field offset in a Fully Convolutionl Network?
I'm studying the Fully Convolutional Networks right now and when it's clear that the receptive field is not dependent on the input size (the whole network in a way is independent from the input size), ...
- 1
2
votes
0
answers
32
views
How do I balance context and history when creating prompts for LLM's?
A conversation through the OpenAI API looks something like this
...
- 121
0
votes
0
answers
14
views
Is it possible tofind a subset of F_p for large p such that no solutions exist
I'm aware that neural networks are probably not designed to do that, however asking hypothetically:
I have a question regarding the possibility of identifying a subset of $\mathbb{F}_p$ in which a ...
- 101
0
votes
1
answer
69
views
How to force Transformer to give more weight to certain tokens
I'm developing an encoder-decoder based transformer model and I would like to ask if there are ways to incentivize or penalize certain tokens during training.
I'm working on a translation task where ...
- 101
1
vote
2
answers
42
views
How to do image classification with optional metadata?
I have a vanilla image classification problem. The image may optionally have some numerical metadata associated with it. We don't assume uniform availability of this metadata, i.e., the model should ...
1
vote
1
answer
61
views
Which process is better to understand images?
What is the difference between this process of recognizing objects in a image:
(The correlation function calculate the correlation coefficient between the input and a image containing the object we ...
- 121
0
votes
0
answers
13
views
What are some good pairs of transfer learning source and target datasets for image classification?
As the title says, I'm curious about some well used transfer learning tasks.
ImageNet to other datasets is common, but what are something good pairs I can try and mess around with ?
Like CIFAR10 to ...
- 11
0
votes
0
answers
33
views
Pruning a specific dimension in a neural network using L1-norm
I've been playing around with pruning neural networks. One interesting thing I've found is that pruning the weights with the lowest L1-norm along a specific dimension seems to give better results than ...
- 101
0
votes
0
answers
55
views
Why are thresholds in NN so important?
I am working on a paper on the approximation properties of NN. One part of this topic is of course the universal approximation theorem (UAT), which gets discussed in various papers with different ...
- 1
-1
votes
1
answer
216
views
How to get Llama-2 Rotary Embeddings?
I want to get the Llama-2 rotary embeddings. I do print(model) and get the following output:
In the picture I highlight the rotary embeddings.
How can get the ...
- 99