Questions tagged [neural-networks]

For questions about a artificial networks, such as MLPs, CNNs, RNNs, LSTM, and GRU networks, their variants or any other AI system components that qualify as a neural networks in that they are, in part, inspired by biological neural networks.

Filter by
Sorted by
Tagged with
0 votes
0 answers
19 views

CNN accuracy is too low using VGG16

Hello i'm working on a simple problem, my model is simple it consists of VGG16 as a base_model and a fully connected layer that has 16 units and relu as activation and finally the output layer which ...
Gabovix's user avatar
0 votes
0 answers
25 views

How exactly is backquery supposed to work in this situation?

Context: This code is based on a 3 layer fully connected neural network trained on had written numbers 0-9. This back query code will then take in an output value of 0.99,0.01,0.01,0.01,0.01,0.01,0.01,...
Stef's user avatar
  • 95
1 vote
0 answers
33 views

Why do mix models work?

Is there research on why models mixes work? One would expect that averaging the weights of two models would produce garbage, but many models mixes created by amateurs show that they not only work, but ...
allo's user avatar
  • 310
1 vote
2 answers
57 views

Books that contains exclusively math problems/assignments in Deep Learning & Neural Networks

I am doing a Deep Learning Course.Suggest some books that contains exclusively math problems/assignments in Deep Learning & Neural Networks. I can understand that majority of the replies suggest &...
kreety kishore's user avatar
1 vote
1 answer
42 views

Marking object on a map from the image

I have been researching if there are any existing machine learning models that would help mark objects (for example: cars) on the map having only image, camera location, and camera orientation. For ...
user3500960's user avatar
0 votes
1 answer
20 views

Gradually increasing CPU load on using sentence embeddings model with kmeans

I am having a ML based production application, using flask, deployed on GCP server using gunicorn workers. In each incoming request, a text sentence is received. It is using sentence transformers (All-...
racdev's user avatar
  • 1
0 votes
1 answer
34 views

Using a neural network to predict a single discrete number

I am working on a project that uses a categorical and non categorical dataset to predict a Success/Fail rate. Each entry/data point has multiple categorical and numerical parameters tied to a rate. We ...
J. Bringas's user avatar
0 votes
1 answer
47 views

How LSTM really decide what to forget and not?

Currently I am learning LSTM and I understand the intuition behind it, like how forget gate works(sigmoid function gives a value between 0 and 1, if it is 0 it "completely" forgets if it is ...
Ashraf's user avatar
  • 3
1 vote
3 answers
50 views

Does transformers' self-attention mechanism process tokens independently, or entire sequence at a time?

About attention: the Query, Key and Value vectors (before the linear transformations) are just the entire sequence, that is being inputted, or just each token? Chat-GPT nor Youtube didn't give me a ...
CyberLight 64's user avatar
0 votes
0 answers
51 views

What are the differences between Inception Score and Fréchet Inception Distance?

From the articles I've read about image generation using GANs, the Inception Score measures two things simultaneously: the variety of images (diversity) and the distinct quality of each image. Does ...
abcd's user avatar
  • 35
1 vote
1 answer
39 views

Image classification of more than 60,000 classes

I am working on a problem that requires the classification of more than 60k classes. I have around 1k to 1.5k images per class. I am using synthetic data for training and want to evaluate it on real ...
pks's user avatar
  • 11
1 vote
1 answer
58 views

Activation function intuition question

I just want to verify the my intuition of why activation functions are necessary. For this example lets consider a network that classifies numbers 0-9. A network WITHOUT an activation function will be ...
Stef's user avatar
  • 95
2 votes
1 answer
74 views

Neural Networks are universal approximators? - Exercice 20.1 UML

I'm working on this question which can be found at page 282 of "Understanding Machine Learning: From Theory to Algorithms" by Shai Shalev-Shwartz and Shai Ben-David. The statement is as ...
mabed's user avatar
  • 21
-1 votes
0 answers
20 views

Request for AI image generation feature in PK [closed]

please allow us to use gemini image generator in Pakistan there is a huge craze about AI image generation so.. this is the time to take over in PK . please allow pk . THANKS
unknown person's user avatar
-2 votes
1 answer
48 views

Predicting Global pandemics in future [closed]

Epidemiology dataset Machine learning prediction algorithms Can AI, Machine learning (Prediction algorithms) predict those pathogens similar to COVID19 Sars virus (Dec 2019 origin) causing global ...
Prashant Akerkar's user avatar
0 votes
0 answers
29 views

Is there a way to design neural networks with symmetric Jacobians?

Is there a way to design neural networks with symmetric Jacobians-the Jacobian of the output with respect to the input? Could you point me to any relevant literature in this area?
Roni's user avatar
  • 1
0 votes
0 answers
47 views

Neural Network ODE solution [closed]

From Bishop's Deep Learning Foundations and Concepts, it defines the Neural ODE (https://arxiv.org/abs/1806.07366). I did not understand from $\frac{dz(t)}{dt}=f(z(t))$ and an initial value $z(0)$, it ...
piero's user avatar
  • 123
-1 votes
3 answers
46 views

Papers on gradient descent w.r.t inputs for optimization [closed]

I'm currently doing research regarding some optimization of antenna configurations. I would love to see if there are any papers discussing gradient descent w.r.t inputs to "change the inputs to ...
dlu's user avatar
  • 1
5 votes
1 answer
612 views

How can the discriminator determine the sample is fake or real?

Based on the articles I've read, the discriminator can identify whether a sample is fake or real. However, the articles don't clarify the conditions used to determine if a sample is fake or real. I ...
David's user avatar
  • 75
2 votes
1 answer
57 views

What are meaning of parameters $\theta$ in this context?

I'm reading the article about generative model from Open AI, here is the section where they explain them: Our network is a function with parameters $\theta$, and tweaking these parameters will tweak ...
David's user avatar
  • 75
0 votes
0 answers
10 views

Solving an ODE with factors that span over orders of magnitude in the region of interest with PINN

I am trying to solve the following ordinary differential equation (ODE) with a physics informed neural network (PINN) $$ \frac{dZ}{dx} = A(x) (1-Z^2) \exp(-Z) - B(x) $$ where A(x) function varies in ...
Maxim's user avatar
  • 1
1 vote
1 answer
34 views

How to make a model forget specific training it has received?

Does L1/L2 (NAdam weight decay) really make the model "unlearn"? Ok so my question might be dumb but is there any way to "unlearn" a model - and yeah I know there is wieght_decay ...
AnArrayOfFunctions's user avatar
0 votes
0 answers
12 views

Explanation for the expression of positional encoding in NeRFs

I was reading the NeRF paper recently (https://arxiv.org/pdf/2003.08934.pdf), and under the positional encoding section, I see that the authors propose usage of the following function for transforming ...
user185887's user avatar
1 vote
1 answer
77 views

Is there any advantage of genetic algorithm (or programming) over Neural Networks? [closed]

I am planning to switch from neural networks to genetic algorithms (GA) and programming (GP). One of the main hassles of working with neural networks is that it requires a large amount of training ...
user366312's user avatar
1 vote
0 answers
18 views

What is the right way to make a neural network learn a period function with known period

I want a neural network to learn the representation of a periodic function whose period is known to be $T$. What is the correct way to achieve that? From my reading, I could infer two things: This ...
user185887's user avatar
0 votes
0 answers
10 views

Neural Net Convergence for Batch SGD

I've built a dynamically sized neural network framework with for multi class classification—just to strengthen my understanding of the deep networks. I'm training and predicting my network to classify ...
arjaras's user avatar
2 votes
2 answers
94 views

What's the best criterion for evaluating activation maps in a CNN?

I'm currently studying CNNs and I had the idea of building a model without a fully connected layer at the end. I think this could be beneficial, if one can somehow model the desired outputs as a ...
Pietro's user avatar
  • 23
0 votes
0 answers
4 views

Realtime Inference of Stateful Variable-Length Time Series LSTM - Getting around Sequence Length vs Inference Frequency Tradeoff

I am trying to train a neural net for low-latency signal filtering; I've been recommended to use a stateful LSTM architecture for this task, however, having corrected some , it seems to me that trying ...
stellarpower's user avatar
1 vote
1 answer
62 views

Why aren't encoders decoders trivial?

If you have an encoder decoder with 10 input neurons for X then 3 hidden in one layer then another 10 in the output which are the same X is it not trivial to set the weights whatever you want and w1 ...
J_Bake's user avatar
  • 11
0 votes
1 answer
55 views

Does ResNext split data or copy it?

I have been learning how to create ResNext neural networks, and am confused how input works with cardinality. In this answer, it seems that it's saying that the data is added together, which I assumed ...
eop3's user avatar
  • 11
0 votes
0 answers
7 views

Can T-Net be substituted by data augmentation in the PointNet architecture?

I am looking at the PointNet architecture at I was wondering why T-Net was preferred over data augmentation. Correct me if I am wrong, but I think of T-Net as trying to find a transformation that will ...
ado sar's user avatar
  • 150
0 votes
0 answers
11 views

Why in Vanilla Gradient for saliency map, we set other classes to zero?

I am reading Pixel Attribution (Saliency Maps) and I have stumbled on the following. For the Vanilla Gradient, if we want to calculate the saliency map for image $I$, then we start with the following ...
ado sar's user avatar
  • 150
0 votes
1 answer
49 views

Handling Variable Output Token Dimensionality in Transformer Decoders During Inference

I'm curious about something in the decoder part of the Transformers architecture. From what I understand, the Keys and Values come from the output of the encoder part of the Transformer. I understand ...
FluidMechanics Potential Flows's user avatar
0 votes
0 answers
20 views

How does the relu nonlinearity allow for nonlinear polynomial transformations?

Since the neural network nonlinearities allow for nonlinear transformations that can stretch and squish the function, how can the ReLU activation function do this? I think for the sigmoid nonlinearity ...
Vivek Reddy's user avatar
0 votes
0 answers
9 views

How to train ViT on smaller datasets?

I know ViTs aren't made for small datasets and low resolution. But have you ever reached traditional CNN accuracy using ViT on CIFAR10/100. I have been playing around with ViT on CIFAR10 and 100. But ...
v1998199904's user avatar
1 vote
1 answer
31 views

Impact of scaling in loss terms when loss function is a composition of multiple functions

I am training a deep learning model, the loss function of which is of the form $$ \cal{L} = \cal{L_1} + \cal{L_2} $$ where $\cal{L_1}$ and $\cal{L_2}$ are of very different orders. WLOG, let's assume ...
user185887's user avatar
0 votes
1 answer
75 views

How the "quantizantion" of models does work?

Consider that a model using FP16 precision is quantized to a lower precision like INT8. Does this reduce the accuracy of the model? From what I know it is designed to reduce the size and required RAM ...
Poseidon of Milos's user avatar
0 votes
0 answers
36 views

Understanding the concepts of embedding in Roberta architecture?

I'm reading the implementation file of Roberta architecture, specifically in the RobertaEmbedding class, this class has the comment: ...
David's user avatar
  • 75
0 votes
0 answers
12 views

Periodic feature layer for Lotka-Volterra approximation

I am working with DeepXDE, a SciML library that can be used to solve differential equations. I came across this demo page for solving a Lotka-Volterra system. Since the solutions are known to be ...
mpnm's user avatar
  • 101
-1 votes
1 answer
79 views

Difference between FLAX and pytorch

I am going through a code written in FLAX instead of pytorch .Can someone please explain what is the difference between these two deep learning frameworks?
akhil's user avatar
  • 9
0 votes
0 answers
19 views

Is Carlini-Wagner dependent on the network under attack?

In my understanding, black-box attacks such as Carlini-Wagner are, in contrast to white-box attacks like FGSM, independent of the classifier's parameters, but various sources seem to disagree with ...
Value_Investor's user avatar
0 votes
1 answer
88 views

In the paper "LLM in a flash," what is meant by an up projection or down projection layer?

In the paper, they first use the terms "up projection layer," and similarly for down projection, in this paragraph in the introduction: Row-column bundling: We store a concatenated row and ...
Tyler's user avatar
  • 103
0 votes
1 answer
25 views

How to Create a 1D Embedding from Tensors of Varying Sizes?

I am a newbie in AI and playing with some computer vision algorithms. I have three tensors with different sizes. Noise augmentation levels tensor with size (N, C, H, W), diffusion timestep tensor of ...
Pooya Kamranjam's user avatar
0 votes
0 answers
26 views

Neural Networks vs Logistic regression

I'm new to Neural Network and would like understand its essential parts and difference from simple logistic regression. Let's take an example of Coffee Roasting prediction (example from Andre NG ...
Teimuraz's user avatar
  • 109
0 votes
2 answers
53 views

Can too much trainingdata have a negative impact?

I have to detect objects in an image. I want to use a neural network for this (yolov8). Since my objects are stacked, most of them are partially hidden and only front and side is visible. My dataset ...
Ef Ge's user avatar
  • 113
0 votes
0 answers
16 views

What are efficient methods for classifying tremor intensity, given challenges with 3D CNN and memory issues with a pretrained VideoMAE model?

I have a training dataset of 80 videos (without augmentation). They are 15 seconds video recorded at 30fps and 240*480 dimensions. What options do I have to train a classification model? The problem ...
Ravideep Singh's user avatar
0 votes
0 answers
23 views

How to implement neural networks for the Pendulum Swing-Up Environment

I have recently completed the Prediction and Control with Function Approximation course on Coursera, which is part of a reinforcement learning specialisation from the University of Alberta. One of the ...
dML's user avatar
  • 1
-1 votes
1 answer
25 views

Why does the number of parameters differ in each layer when each layer is defined the exact same way [closed]

...
Mohd. Farhan Hassan's user avatar
0 votes
0 answers
51 views

Neural networks with a continuous function as the input

I am rethinking image recognition algorithms, and want to use an idea I saw in a 3Blue1Brown video: The basic idea is to map pixels on an image to a specific frequency—every point is distinct—and if ...
Ank i zle's user avatar
  • 101
-1 votes
1 answer
47 views

PINN - Why Differential equations only? [closed]

Im recently studying Physics informed neural networks. I found that it is designed for solving Differential equations. Is there any reason why it is used for differential equations only?
COTHE's user avatar
  • 3

1
2 3 4 5
52