Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
10 views

Strange date behavior in ChatGPT - why?

I gave ChatGPT this data: ...
0 votes
0 answers
6 views

How to create a model in TensorFlow that always gets the same output?

I am new to neural networks and currently working with TensorFlow. For an experiment, I would like to build a model that consistently produces the same output for identical inputs. However, my initial ...
28 votes
4 answers
11k views

How could we build a neural network that is invariant to permutations of the inputs?

Given a neural network $f$ that takes as input $n$ data points: $x_1, \dots, x_n$. We say $f$ is permutation invariant if $$f(x_1 ... x_n) = f(\sigma(x_1 ... x_n))$$ for any permutation $\sigma$. How ...
0 votes
1 answer
56 views

How LSTM really decide what to forget and not?

Currently, I am learning about LSTM, and I understand the intuition behind it, such as how forget gate works (sigmoid function yields a value between 0 and 1; if it is 0 it "completely" ...
0 votes
0 answers
10 views

How to train an AI model to identify GUI elements

Goal: I would like to make an app that can detect UI elements, extract them, and let the user manipulate them in some way. Existing Solutions: There are 2 tools that I used that try to do this, but ...
2 votes
1 answer
170 views

Fit Q Evaluation in offline reinforcement learning

I am working on a PyTorch implementation of Implicit Q-Learning (IQL) (paper), given a dataset $\mathcal D = \left\{ (\mathbf s_i, \mathbf a_i, \mathbf s_i', r_i ) \right\}$ of transitions. I think I ...
1 vote
3 answers
59 views

Does transformers' self-attention mechanism process tokens independently, or entire sequence at a time?

About attention: the Query, Key and Value vectors (before the linear transformations) are just the entire sequence, that is being inputted, or just each token? Chat-GPT nor Youtube didn't give me a ...
1 vote
4 answers
2k views

Neural networks with sparse inputs

I have a task I want to solve with neural networks. The task is predicting a certain vector of dimension K. The problem is that the inputs to the networks are sparse. The input is a vector of size N, ...
1 vote
3 answers
252 views

How can I implement 2D CNN filter with channelwise-bound kernel weights?

I would like to bind kernel parameters through channels/feature-maps for each filter. In a conv2d operation, each filter consists of HxWxC parameters I would like to have filters that have HxW ...
1 vote
1 answer
69 views

Why does averaging attention-weighted positions reduce the effective resolution in transformers?

I was reading this blog post from Harvard and it says in its background paragraph about transformers that the number of operations required to relate signals from two arbitrary input or output ...
0 votes
2 answers
163 views

If I freeze pre-trained model weights and than train a classifier on top of its embeddings does that called fine-tunning?

In the context of machine learning. If I freeze pre-trained model weights (for example, BERT) and then train a classifier on top of its embeddings, does that called fine-tuning?
0 votes
1 answer
101 views

How to configure a neural network to selectively change only certain characters in a string?

I'm trying to figure out how to train a neural network to macronize Latin text. Essentially, in Latin, vowels can either be long or short, and length is indicated with a macronized character: i.e. o ...
1 vote
1 answer
133 views

Is it possible to know the distance objects are from camera based on only knowing one object's height?

I am doing a project where I have to know distance a particular object is from camera. In the photo I only know one of the object's height, but I don't know how far away that object is and I don't ...
0 votes
2 answers
406 views

Please help me understand the role of loss function in neural networks

I've been studying NNs with tensorflow and decided to code a simple NN from scratch to get a better idea on hwo they work. It my understanding that the cost is used in backpropagation, so basically ...
1 vote
1 answer
673 views

How can I train a neural network to detect subliminal messages?

Is there a way to train a neural network to detect subliminal messages? Where can I find the dataset on which to train the neural network? If I have to create the dataset, how would I go about it? ...
2 votes
1 answer
899 views

What are the state-of-the-art AI methods to recognize elements on webpages or the purpose of webpage?

I'm curious to know about the capabilities of AI today in 2022. I know that AI has become pretty good at recognizing things like objects in photos. But what about when it comes to elements in HTML? ...
0 votes
0 answers
25 views

How exactly is backquery supposed to work in this situation?

Context: This code is based on a 3 layer fully connected neural network trained on had written numbers 0-9. This back query code will then take in an output value of 0.99,0.01,0.01,0.01,0.01,0.01,0.01,...
0 votes
0 answers
21 views

CNN accuracy is too low using VGG16

Hello i'm working on a simple problem, my model is simple it consists of VGG16 as a base_model and a fully connected layer that has 16 units and relu as activation and finally the output layer which ...
5 votes
2 answers
947 views

What is the intuition behind self-attention?

I've been watching a few lectures on transformers, especially for language translation, though it seemingly becomes more confusing the more I watch. In this lecture, there seems to be two conflicting ...
1 vote
2 answers
58 views

Books that contains exclusively math problems/assignments in Deep Learning & Neural Networks

I am doing a Deep Learning Course.Suggest some books that contains exclusively math problems/assignments in Deep Learning & Neural Networks. I can understand that majority of the replies suggest &...
1 vote
1 answer
41 views

Image classification of more than 60,000 classes

I am working on a problem that requires the classification of more than 60k classes. I have around 1k to 1.5k images per class. I am using synthetic data for training and want to evaluate it on real ...
1 vote
0 answers
33 views

Why do mix models work?

Is there research on why models mixes work? One would expect that averaging the weights of two models would produce garbage, but many models mixes created by amateurs show that they not only work, but ...
1 vote
1 answer
96 views

Hand-Signs Recognition using Deep Learning Convolutional Neural Networks

I am developing a CNN model to recognize 24 hand-signs of American Sign Language. I have 2500 Images/hand-sign. The data split is: Training = 1250 Images/hand-sign Validation = 625 Images/hand-sign ...
0 votes
1 answer
35 views

Using a neural network to predict a single discrete number

I am working on a project that uses a categorical and non categorical dataset to predict a Success/Fail rate. Each entry/data point has multiple categorical and numerical parameters tied to a rate. We ...
0 votes
1 answer
20 views

Gradually increasing CPU load on using sentence embeddings model with kmeans

I am having a ML based production application, using flask, deployed on GCP server using gunicorn workers. In each incoming request, a text sentence is received. It is using sentence transformers (All-...
1 vote
1 answer
43 views

Marking object on a map from the image

I have been researching if there are any existing machine learning models that would help mark objects (for example: cars) on the map having only image, camera location, and camera orientation. For ...
1 vote
3 answers
167 views

How can I model any structure for a neural network?

Hello I am currently doing research on the effect of altering a neural network's structure. Particularly I am investigating what affect would putting a random DAG (directed acyclic graph) in the ...
1 vote
1 answer
140 views

Why do we use a delay when feeding our input data to the echo state network?

I'm new to working with neural networks and have recently began implementing neural networks for time series forecasting in some of my work. I've been particularly using Echo State Networks and have ...
0 votes
0 answers
52 views

What are the differences between Inception Score and Fréchet Inception Distance?

From the articles I've read about image generation using GANs, the Inception Score measures two things simultaneously: the variety of images (diversity) and the distinct quality of each image. Does ...
2 votes
1 answer
82 views

Neural Networks are universal approximators? - Exercice 20.1 UML

I'm working on this question which can be found at page 282 of "Understanding Machine Learning: From Theory to Algorithms" by Shai Shalev-Shwartz and Shai Ben-David. The statement is as ...
3 votes
2 answers
130 views

What is a working configuration of a neuronal network (number of layers, lerning rate and so on) for a specific dataset?

I try to solve some easy functions with a neuronal network (aforge-lib): This is how I generate the dataset: ...
2 votes
1 answer
535 views

Why can't we train neural networks in a peer-to-peer manner?

I have recently been exposed to the concept of decentralized applications, I know that neural networks require a lot of parallel computing infra for training. What are the technical difficulties one ...
1 vote
1 answer
624 views

Why gradients are used in Layer-wise Relevance Propagation (LRP)?

To give you an overview, Layer-wise Relevance Propagation is a technique by which we can get relevance values at each node of the neural network. These calculated relevance values (per node) are ...
1 vote
1 answer
58 views

Activation function intuition question

I just want to verify the my intuition of why activation functions are necessary. For this example lets consider a network that classifies numbers 0-9. A network WITHOUT an activation function will be ...
0 votes
0 answers
34 views

Is there a way to design neural networks with symmetric Jacobians?

Is there a way to design neural networks with symmetric Jacobians-the Jacobian of the output with respect to the input? Could you point me to any relevant literature in this area?
3 votes
1 answer
169 views

Is batch learning with gradient descent equivalent to "rehearsal" in incremental learning?

I am learning about incremental learning and read that rehearsal learning is retraining with old data. In essence, isn't this the exact same thing as batch learning (with stochastic gradient descent)? ...
2 votes
1 answer
175 views

How to perform back-propagation in Decoupled Neural Interfaces?

I am attempting to create a fully decoupled feed-forward neural network by using decoupled neural interfaces (DNIs) as explained in the paper Decoupled Neural Interfaces using Synthetic Gradients (...
2 votes
1 answer
74 views

How to decode P bits that represent a random weight generator?

So I've been tasked by my neural network professor at university to replicate the following research: Intelligent Breast Cancer Diagnosis Using Hybrid GA-ANN. Each chromosome represents a possible net,...
1 vote
4 answers
282 views

How to improve neural network training against a large data set of points with varying magnitude

I am currently using TensorFlow and have simply been trying to train a neural network directly against a large continuous data set, e.g. $y = [0.014, 1.545, 10.232, 0.948, ...]$ corresponding to ...
1 vote
1 answer
77 views

Is there any advantage of genetic algorithm (or programming) over Neural Networks? [closed]

I am planning to switch from neural networks to genetic algorithms (GA) and programming (GP). One of the main hassles of working with neural networks is that it requires a large amount of training ...
5 votes
1 answer
3k views

Does the input layer of a neural network have bias and are there bias neurons?

I have seen two different representations of neural networks when it comes to bias. Consider a "simple" neural network, with just an input layer, a hidden layer and an output layer. To ...
0 votes
1 answer
149 views

How to handle random order of inputs and get same output?

I am a beginner with DL. I did some tutorials and I know the basics of TensorFlow. But I have a problem understanding how to construct more advanced NNs. Let's say I have 6 inputs and a list of 500 ...
4 votes
2 answers
218 views

Which neural network can I use to solve this constrained optimisation problem?

Let $\mathcal{S}$ be the training data set, where each input $u^i \in \mathcal{S}$ has $d$ features. I want to design an ANN so that the cost function below is minimized (the sum of the square of ...
2 votes
2 answers
95 views

What's the best criterion for evaluating activation maps in a CNN?

I'm currently studying CNNs and I had the idea of building a model without a fully connected layer at the end. I think this could be beneficial, if one can somehow model the desired outputs as a ...
2 votes
1 answer
108 views

Can I train a model starting from another model trained on a subset of the dataset?

I want to create a neural network and train it on some data, however I want to be able to create a new model without retraining it from the start. An example, I have 1000 data points in my training ...
5 votes
1 answer
618 views

How can the discriminator determine the sample is fake or real?

Based on the articles I've read, the discriminator can identify whether a sample is fake or real. However, the articles don't clarify the conditions used to determine if a sample is fake or real. I ...
2 votes
1 answer
58 views

What are meaning of parameters $\theta$ in this context?

I'm reading the article about generative model from Open AI, here is the section where they explain them: Our network is a function with parameters $\theta$, and tweaking these parameters will tweak ...
0 votes
1 answer
80 views

Changing the number of epochs change the loss at the the `x`th epoch

During a training of a neural network, the test loss was reached the minimum at the x-th epoch, after which I reran the training with the maximum epoch set as ...
0 votes
0 answers
10 views

Solving an ODE with factors that span over orders of magnitude in the region of interest with PINN

I am trying to solve the following ordinary differential equation (ODE) with a physics informed neural network (PINN) $$ \frac{dZ}{dx} = A(x) (1-Z^2) \exp(-Z) - B(x) $$ where A(x) function varies in ...
0 votes
2 answers
165 views

Why does the accuracy of my neural network stay constant?

I'm testing my own implementation of a neural network on recognising the type of a function. I generate sine, linear and quadratic functions with random parameters, compute their values for a linspace ...

1
2 3 4 5
52