Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
0 votes
0 answers
27 views

Relating two small datasets with Poisson in a neural network or random network

I'm looking for a way to "relate" two small datasets, in Python, for predictive purposes. The two datasets are both independent and have no correlation (the numbers are randomly positioned). ...
Horiatiki's user avatar
  • 101
0 votes
1 answer
23 views

What are the techniques used to initialize weights for neural networks?

When creating a neural network to predict the impact of risks on the project cost, what techniques are used to initialize the weights provided to the hidden layers and the output layer?
maya sy's user avatar
  • 11
2 votes
1 answer
411 views

Which epoch is the best for me to choose?

I have trained my deep learning model. I also saved the validation loss to a file and plotted on a graph I have $2$ questions for this: Does the validation loss look normal? Is there any issue with ...
user's user avatar
  • 23
1 vote
1 answer
52 views

Types of activation functions used in neural networks

I need to specify the points and conditions that require me to use one activation function over another, Why is it not possible to use the sigmoid FUNCTION in hidden layers, and it is preferable to ...
maya sy's user avatar
  • 11
0 votes
0 answers
22 views

Prefix tuning in LLM uses learnable vectors to fine tune the model

I would like to implement a new architecture for Transformer. Below description is my thought. Prefix tuning in LLM uses learnable vectors to fine tune the model. Is there a way to use the output ...
jackson's user avatar
0 votes
1 answer
25 views

Is size of trained model on disk a good measure of model complexity?

I am writing a research paper on my own custom CNN model for image classification. I am comparing my model architecture with pre-trained architectures, like DenseNet121 and InceptionV3. I want to ...
Dawood Ahmad's user avatar
0 votes
0 answers
11 views

Could an analysis of GPT4's WAIS score be published?

Similar to this: https://www.scientificamerican.com/article/i-gave-chatgpt-an-iq-test-heres-what-i-discovered/ But more detailed and in depth (subtest breakdown, including image analysis, etc.), WAIS-...
BigMistake's user avatar
2 votes
0 answers
37 views

Is there a state-based approach to neural networks?

It occurred to me that neural networks operate like straight-line programs: an input is processed once by the layers, and an output is always immediately formed, in the same amount of time for each ...
HiddenBabel's user avatar
1 vote
1 answer
17 views

Can I do incremental learning with different loss function in neural networks?

I have a saved tensorflow neural network model. I was wondering if it's possible to incrementally train the model but with different nt loss function.
SUNITA GUPTA's user avatar
1 vote
0 answers
15 views

Looking for a way to train a model to learn optimal parameters/hyperparameters of clustering

I have 5000 docs, each is a review. For each review, i'm plotting the sentences in a semantic dimension. Now, I'm applying clustering to these points for each review. The success of my model depends ...
Prithvi's user avatar
  • 11
0 votes
1 answer
97 views

Why in one article to calcule the delta uses the weights of current layer, while in another article uses the weights of neuron inputs of next layer?

Article 1: https://pyimagesearch.com/2021/05/06/backpropagation-from-scratch-with-python/ Article 2: https://machinelearningmastery.com/implement-backpropagation-algorithm-scratch-python/ I was ...
will The J's user avatar
0 votes
0 answers
17 views

Can't get a correct accuracy on tabular data using deep learning

This is my first message here, and I would like to seek some assistance ! I have a technical test for a job that I really want, and I have 10 days to complete it. I've attempted to work on it, but I'm ...
Enzo Durand's user avatar
0 votes
1 answer
51 views

In multilayer perceptron neural networks, the more complex a dataset is, the more difficult it will be to find adequate initial random weights?

I know that neural networks don't need to have exactly the same weights, and that different networks trained on the same problem may result in different final weights, but produce comparable results. ...
will The J's user avatar
2 votes
1 answer
56 views

How can I deal with random weights initialisation when predicting a time-series sine function?

I am training a simple RNN model in keras to predict a time series. The time series I am considering is just a sine function The task to solve is the following: ...
apt45's user avatar
  • 123
0 votes
0 answers
20 views

Base rate effects on object detection neural networks?

Suppose that we are training an neural network for object detection. It is natural to train the neural network on images that contain the object in question. However, this might have the effect of ...
Him's user avatar
  • 171
0 votes
0 answers
29 views

How was the word2vec model trained?

Let's take the CBOW (continuous bag of words) model as the example. Suppose that, there are $c$ context words, each of which is a one-hot encoding vector. So the total number of elements of input ...
J. Doe's user avatar
  • 1
0 votes
0 answers
32 views

Why is masked self attention necessary on GPT decoder

I'm currently reading the paper for the first GPT model and I'm confused about why masked self attention is necessary and I haven’t found any good answers online. The consensus seems to be that we don'...
Kiran Manicka's user avatar
0 votes
1 answer
44 views

What is the potential issue of nested neural networks

everyone. I am working on a nested neural network architecture. For the sake of better understanding my question, simply assume the loss is $L = G(k’) - H(k'')$ where $G$ and $H$ are two functions we ...
Zuba Tupaki's user avatar
0 votes
1 answer
40 views

If I manually trained a multilayer percetron neural network, always following exactly the same steps meticulously, would I always get the same result?

Reference: https://home.agh.edu.pl/~vlsi/AI/backp_t_en/backprop.html If I trained a multilayer percetron neural network manually, following exactly the backpropagation steps described in the article, ...
will The J's user avatar
0 votes
0 answers
20 views

When to know if I am "on the right track" for a CNN architecture

Context Very new to CNNs and ML in general. I am building a simple binary image segmentation network for generating black and white image masks (white pixels = desired object; black pixels = all else)....
gladshire's user avatar
0 votes
0 answers
14 views

Keeping the grad of the tensor when using inverse_transform

To train a network, I scaled both the input and outputs of my data like the following: ...
jasw's user avatar
  • 1
0 votes
1 answer
71 views

Why is it called multi-headed attention?

Why do we call the attention layer in transformers multi-headed attention when in practice all the attention matrices from different heads (W,K,V) for a single layer are concatenated to perform the ...
Tarique's user avatar
  • 103
0 votes
0 answers
24 views

Neural Networks Model Predictions on Same Sample

I have built a tensorflow model to reconstruct the given input data sample by using an Autoencoder to predict the Missingness in the sample. However I do not have enough data sample where i don't have ...
siva sai kumar reddy's user avatar
0 votes
0 answers
44 views

The more a MLP neural network is trained, the value of correct class should increasingly closer to 1, and the other classes to 0? Is this expected?

I am using a multilayer perceptron network with the Iris dataset. This network has 3 output neurons (to represent the 3 classes of Iris, that is, Iris Versicolor, Iris Virginia and Iris Setosa). ...
will The J's user avatar
0 votes
0 answers
17 views

What kind of learning architecture can I use to solve a set of nonlinear thermodynamic equations?

I am in the business of solving equations based in thermodynamics. Technically speaking, I am trying to solve for the binodals for a ternary system. The binodal is a curve in a phase diagram. ...
bad_chemist's user avatar
2 votes
1 answer
1k views

Neural network for game

There is a game for two, which is an NxN field (always the same size). Players take turns. The first player's goal is to connect the two points (not necessarily at the corners) given on this field. ...
user3576767's user avatar
1 vote
1 answer
40 views

An Adaline neuron can solve problems that are not linearly separable?

https://en.wikipedia.org/wiki/ADALINE I was confused about this because, for example, the XOR problem is not linearly separable, and a simple Perceptron obviously cannot solve it, so we would need a ...
will The J's user avatar
0 votes
1 answer
33 views

Can I train a Madaline network, just training each neuron individually, and then combining the output of all of them to be the classification result?

I've heard that Madaline networks generally have a single hidden layer, which have multiple Adaline units, where each Adaline receives different subsets of the input data. And the output of the ...
will The J's user avatar
0 votes
1 answer
29 views

Is Madaline neural network and ther train almost the same as a multilayer perceptron(MLP)? what's the difference?

https://www.geeksforgeeks.org/adaline-and-madaline-network/ Is the Madaline neural network and its training mode almost the same or identical to a multilayer perceptron (MLP)? what's the difference?
will The J's user avatar
0 votes
0 answers
22 views

What are general techniques of structuring an image classification Neural Network for very large numbers of output classes?

I am aware of Neural Networks that have 100K+ classes and I would like to build one myself (yes, I have lots of training data) but I am unsure which technique to use because most of the nets I have ...
AnalogDigital's user avatar
0 votes
0 answers
22 views

Encoding categorical data with "many" unique values in neural network

I am new to machine learning, in fact, I am implementing my first deep neural network from scratch without any framework. The dataset has 3500 rows, and 4 categorial columns of which two have about ...
M a m a D's user avatar
  • 113
0 votes
0 answers
26 views

In multilayer perceptron neural networks, is normalization required? or is optional? Why the network only seem to work when I use normalization?

I followed the steps in the following article (which teaches how to program a multilayer perceptron neural network from scratch in Python): https://machinelearningmastery.com/implement-backpropagation-...
will The J's user avatar
0 votes
0 answers
11 views

How to implement a link constraint between the weights/bias of a neural network?

I'm new to the forum, and although I've tried my best, I hope not to write a duplicate post. Here's the background: I'm trying to build a neural network from scratch where certain weights are related ...
Sourisimos's user avatar
0 votes
0 answers
31 views

Multiple text inputs to neural network

How do I best input multiple text fields in a neural network for classification? For example, each data point has a title and an abstract: ...
raywib's user avatar
  • 1
0 votes
0 answers
14 views

Sampling dollar bars for ML model of multiple tickers

I have a Neural Network model that provides predictions for the future returns of a portfolio comprising stocks and cryptocurrencies. The original model operates on standard time bars (sampled at a ...
apt45's user avatar
  • 123
0 votes
0 answers
11 views

Hopfield networks with shared neurons

Has there been research or even a thorough analysis of several Hopfield networks coupled by sets of shared neurons? Would this work? What about their respective storage capacities and robustness? Any ...
Hans-Peter Stricker's user avatar
1 vote
1 answer
42 views

How do I train a CNN on multiple images that have the same shape?

I want to train a convolutional neural network on multiple input images. My image is 240x360 and is in RGB. Therefore my input image has a shape of 3x240x360. Now I want to use multiple images of the ...
Erik Storm's user avatar
0 votes
0 answers
17 views

Does the training time of a neural network increase more if we add a layer at the beginning or at the end?

Let's consider a fixed NN architecture, dataset and hardware. We add a layer, either at the beginning or at the end of the NN. In which case the training time will increase more? Intuitively, I ...
DeltaIV's user avatar
  • 194
0 votes
0 answers
13 views

How can a RNN with 256 cells accept a input of any size?

I built a 3 layered RNN model with 256 cells each using torch. Input feature size is set to 40. Below give a basic Idea on the model. ...
D Star Let's Explore's user avatar
1 vote
0 answers
39 views

Are there neural networks that compute weights dynamically based on geometric attributes of neurons?

I am interested in exploring neural network architectures where the weights are not stored but are computed dynamically based on certain attributes or "dimensions" of the connected neurons. ...
Deadbeef Development's user avatar
0 votes
0 answers
24 views

Seeking methods to incorporate arbitrary actuator faults for Control Optimization

I am working on a problem where a control method, backed by a Neural Network (NN), dictates the movement of a 1D actuator to influence a specific process. This actuator can move linearly within a set ...
IsolatedSushi's user avatar
0 votes
1 answer
67 views

Time Series Classification using Transformer Encoder

Lets say I have a collection of tensors, each tensor representing a time series with 64 points and 4 features. The dimension of each tensor would be [64,4]. I am trying to classify these series. For ...
Zohaib Hamdule's user avatar
0 votes
1 answer
39 views

Which algorithm do I need to use to calculate the error?

I am building a neural network for a AI whose goal is to learn to implement the XOR logical function.I have this code so far: ...
Cerise's user avatar
  • 121
0 votes
0 answers
20 views

How to compensate the receptive field offset in a Fully Convolutionl Network?

I'm studying the Fully Convolutional Networks right now and when it's clear that the receptive field is not dependent on the input size (the whole network in a way is independent from the input size), ...
Antoni's user avatar
  • 1
2 votes
0 answers
32 views

How do I balance context and history when creating prompts for LLM's?

A conversation through the OpenAI API looks something like this ...
Ian Purton's user avatar
0 votes
0 answers
14 views

Is it possible tofind a subset of F_p for large p such that no solutions exist

I'm aware that neural networks are probably not designed to do that, however asking hypothetically: I have a question regarding the possibility of identifying a subset of $\mathbb{F}_p$ in which a ...
laura's user avatar
  • 101
0 votes
1 answer
70 views

How to force Transformer to give more weight to certain tokens

I'm developing an encoder-decoder based transformer model and I would like to ask if there are ways to incentivize or penalize certain tokens during training. I'm working on a translation task where ...
jasperagrante's user avatar
1 vote
2 answers
42 views

How to do image classification with optional metadata?

I have a vanilla image classification problem. The image may optionally have some numerical metadata associated with it. We don't assume uniform availability of this metadata, i.e., the model should ...
Vardaan Pahuja's user avatar
1 vote
1 answer
61 views

Which process is better to understand images?

What is the difference between this process of recognizing objects in a image: (The correlation function calculate the correlation coefficient between the input and a image containing the object we ...
Cerise's user avatar
  • 121
0 votes
0 answers
13 views

What are some good pairs of transfer learning source and target datasets for image classification?

As the title says, I'm curious about some well used transfer learning tasks. ImageNet to other datasets is common, but what are something good pairs I can try and mess around with ? Like CIFAR10 to ...
v1998199904's user avatar

1
2 3 4 5
51