Skip to main content

Questions tagged [deep-neural-networks]

For questions related to deep neural networks, which are artificial neural networks with "many" layers, where "many" can vary depending on the context.

Filter by
Sorted by
Tagged with
0 votes
1 answer
29 views

Is deep learning suitable/preferable for string similarity detection and application automation? If so, which type?

newbie here. I have developed an app that basically does: Perform OCR, check if words are contained in the resulting text and then perform an action. If no words are detected from the given list, ...
zaxunobi's user avatar
  • 101
0 votes
0 answers
9 views

Do I need model calibration/feature normalization for reliable path integrated gradient/sampled Shapley feature attribution in a dnn model?

Are model calibration and feature normalization required for path integrated gradient and sampled Shapley-based feature attribution analyses to work properly in a deep neural network model? I read ...
AiWannabe's user avatar
0 votes
0 answers
18 views

Why does the ECE increase after applying temperature scaling?

I am planning to apply temperature scaling to the output of my model. To determine the optimal temperature, I employed cross-validation and utilized the code available at this link: https://github.com/...
Sara's user avatar
  • 1
1 vote
0 answers
19 views

Compare two songs content using Audio Spectogram Transformer

I'm trying to establish a similarity metric between two songs. To do this I'm using the AST model on HuggingFace. This model basically works in a way very similar to a ViT but applied to spectograms ...
user491880's user avatar
0 votes
0 answers
14 views

Optuna Hyperband Algorithm Not Following Expected Model Training Scheme

I have observed an issue while using the Hyperband algorithm in Optuna. According to the Hyperband algorithm, when min_resources = 5, max_resources = 20, and reduction_factor = 2, the search should ...
Tnb Marketplace's user avatar
0 votes
0 answers
16 views

input_tensor is not connected to anything in onnx but can still do inference. How?

I converted an effdet model trained on custom dataset from tensorflow to onnx and tried to further convert it to tensorrt engine file. However, while doing so, I got uint8 issue (tensorrt does not ...
you_know_who's user avatar
0 votes
1 answer
28 views

CNN multioutput regression architecture modification

I am working on a regression task where the model has to predict two values at the same time. The idea is that the dataset consists of 16 features, where the first 8 features represent the first value ...
lukachu03's user avatar
0 votes
0 answers
19 views

Ideal method for finding function that satisfies a set of constraints

I'm new to playing around with deep learning. I'm trying to find a function that satisfies a bunch of constraints, and looking for tips on how to better my approach. Let $F:X\times Y \mapsto [0,1]$, ...
Clempe's user avatar
  • 1
0 votes
0 answers
16 views

Enhancing Soil Moisture Predictions Using Multimodal Data Integration in Agriculture

I am exploring an interdisciplinary research area involving multimodal data, focusing on agriculture. My study incorporates both visual and tabular data: crop and soil images from three distinct ...
Md Rakib's user avatar
0 votes
0 answers
41 views

The end-to-end Training Process for Knowledge Distillation

I'm a bit confused on the complete training process for Knowledge Distillation. I was reading the Geoffrey Hinton "Distilling the Knowledge in a Neural Network" 2015 paper and some random ...
Chuu's user avatar
  • 1
0 votes
1 answer
83 views

What is the potential issue of nested neural networks

everyone. I am working on a nested neural network architecture. For the sake of better understanding my question, simply assume the loss is $L = G(k’) - H(k'')$ where $G$ and $H$ are two functions we ...
Zuba Tupaki's user avatar
1 vote
0 answers
38 views

Replicating conv autoencoder for anomaly detection, very blurry reconstructions

I’m trying to train an autoencoder on the hazelnut dataset of MVTec AD for reconstruction to detect anomalies. I’m am trying to replicate the results of this study: https://arxiv.org/pdf/2008.12977....
JeanMi's user avatar
  • 165
1 vote
0 answers
139 views

In the figure of Stable Diffusion, when does the switch part change?

In the illustration of Stable Diffusion, there is a concatenation part through Cross-Attention. Why is there a switch in the concatenation part?
diffusion stable's user avatar
1 vote
0 answers
80 views

Pointers to (deep) latent variable models that admit analytical approximations

I am aware that there is a plethora of deep generative models out there (e.g. variational autoencoders (VAE), GANs) that can model high-dimensional data as the images of latent variables under a non-...
ngiann's user avatar
  • 111
0 votes
0 answers
27 views

How to differentiate fuzzy matching from artificial intelligence [duplicate]

I am curious about the term "fuzzy matching," and whether it falls under the category of artificial intelligence. Specifically, when can we say that a website or system is using AI, and when ...
CollarKaniz's user avatar
1 vote
2 answers
204 views

How to do image classification with optional metadata?

I have a vanilla image classification problem. The image may optionally have some numerical metadata associated with it. We don't assume uniform availability of this metadata, i.e., the model should ...
Vardaan Pahuja's user avatar
0 votes
0 answers
32 views

Independent parameter update in backpropagation

When we calculate the gradient wrt to each paramters, we consider the other parameters remain constant, but the moment their is a change in any of the other parameters, shouldn't all the other changes ...
In progress...'s user avatar
1 vote
2 answers
388 views

Why is dot-product and not Euclidean distance used for attention?

In models using attention (eg Transformer architectures) we used scaled dot-product to measure similarity rather than (negative or inverse) Euclidean distance. Why is this the case? Does Layer ...
Betterthan Kwora's user avatar
0 votes
1 answer
43 views

Temporally Non-Aware RNN

I am trying to classify whether or not a specific object is in panoramic photos. The issue is, a panoramic photo can be any width, so the input to my neural network can't be fixed in that dimension. I'...
user avatar
0 votes
0 answers
22 views

Can we use bounding box cropping to avoid shortcut learning (achieve explainable AI)?

Deep neural networks sometime use shortcut features (pseudo correlation) to predict. For example, in cat-dog classification, the network may use the background information (e.g. floor, grass) as a ...
Mingzhou Liu's user avatar
0 votes
0 answers
29 views

Is it possible to combine SGD with an unsupervised learning approach effectively

Before I undertake quite a large project I would like to clarify whether my idea for training a multi-layer neural network will work. I plan to make an AI that can land a rocket from randomly ...
Gamaray's user avatar
0 votes
0 answers
15 views

Linear Discriminant Analysis on a transformed space

Let $S$ be a finite subset of a $\mathbb{R}^k$ partitioned into $N$ subsets $S_1, \ldots, S_N$ and let $n_j = |S_j|$. The between-groups sum of squares of the partition is defined as $$bSS(S_1,\ldots, ...
Alberto's user avatar
  • 101
1 vote
1 answer
45 views

How to handle large dimensionality differences between state and action inputs in a reinforcement learning predictor?

I'm currently writing code for a reward predictor function r=f(s,a) in reinforcement learning, where 's' is the state with 256 dimensions (the embedding dimension after visual input is processed by an ...
XiaoBanni's user avatar
0 votes
0 answers
69 views

How to tell an agent that some actions in the action space are currently not available in gym and the design of action space

I want to make a task allocation decision by reinforcement learning. Suppose there are N tasks to be allocated and M severs to serve these task. However, there is a constraint that one task should be ...
Reese's user avatar
  • 1
0 votes
1 answer
45 views

When training a DNN model, how can I make some training data points more important than others?

In other words, is it possible to assign "weights" to data points during model training? Is there a standard technique for it? It seems like it the math would be straightforward enough for ...
PlinyTheElder's user avatar
1 vote
0 answers
38 views

Can a Fully Connected Neural Network represent all Neural Networks of smaller size?

A fully connected Neural Network architecture can be characterized by a vector $\mathbf a = (a_0,a_1,\ldots,a_L)\in\mathbb N^{L+1}$ and an activation function $\sigma :\mathbb R\to\mathbb R$. In this ...
Stratos supports the strike's user avatar
0 votes
0 answers
28 views

How to identify location from a given input in a semantic way

I have a sample use case, where user will give us a xls file with some location data. Where may be location information is given in different ways , sometime its IATA code, sometimes its standard ...
Sujoy's user avatar
  • 1
2 votes
0 answers
22 views

In DQN how does the Q network not converge to the incorrect target? [duplicate]

Whenever you are doing reinforcement learning you periodically update the target network based on the weights of the Q network. While I do understand this helps create a stable target I do not ...
Stef's user avatar
  • 95
2 votes
0 answers
80 views

Is orthogonal initialization still useful when hidden layer sizes vary?

Pytorch's orthogonal initialization cites "Exact solutions to the nonlinear dynamics of learning in deep linear neural networks ", Saxe, A. et al. (2013), which gives as reason for the ...
Gabi's user avatar
  • 121
0 votes
1 answer
44 views

Classify sequence of flags

I am not able to find an answer to how I should classify a varying number of sequence of binary flags + other features. My data looks like this (these are events, so the order is important and I may ...
Zaba's user avatar
  • 1
0 votes
1 answer
108 views

Difference in gradient calculation for the last layer activation in neural networks

I'm currently working on implementing a neural network using the sigmoid activation function and the binary cross-entropy cost function. In my implementation, I've noticed that the gradient ...
Richard AR's user avatar
0 votes
1 answer
22 views

What architecture is used for deep quadruplet network for person re-identification

I am trying to implement the paper Beyond triplet loss: a deep quadruplet network for person re-identification. In the paper, they provide a figure (attached below) containing the network architecture,...
Varghese Kuruvilla's user avatar
0 votes
0 answers
27 views

Longer DNN training times when using evolutionary algorithms

I am comparing my deep neural network (DNN) performance when using 2 types of optimizers: gradient-based Adam (properly tuned) and a population-based optimization algorithm (e.g., genetic algorithm (...
knowledge_seeker's user avatar
2 votes
1 answer
91 views

Can models like chatGPT learn functions with infinite domain or range

Lets assume two types of prompt: A fixed prompt for which reasonable responses can be infinite. For example: > output a random number > output a palindrome ...
Sayam Qazi's user avatar
3 votes
0 answers
30 views

Why does training converges when the norm of gradient increases?

This is from deep learning book by Ian Goodfellow and Yoshua Bengio and Aaron Courville. When training converges well, I thought the gradient should be at local minima. But the book says it often does ...
tesio's user avatar
  • 205
0 votes
1 answer
138 views

How vision models based on CNNs learn the relative positions of each pixel in the image?

A CNN model is based on a series of filters applied to an image. However, these filters can only "see" a small portion of the image and they have no information of the relative position of ...
IgnacioGaBo's user avatar
1 vote
2 answers
237 views

What should I do, reinforcement learning agent gives different result on every train?

I'm using PPO+LSTM to create a trading bot. The agent is trained on 3 years of data and tested on 1 year. Every time I train the agent with same set of hyper-parameters, I get very different results ...
ad124j2's user avatar
  • 11
0 votes
1 answer
54 views

How to improve classification accuracy in TF deep neural network model?

I need help in increasing the accuracy of a classification model using Neural Networks on Tensorflow. I am trying to train a model on sequential data ...
Fr_nkenstien's user avatar
1 vote
0 answers
153 views

What does Deep Q-Learning (DQL) do?

Hello :) I'm required to write a document where I describe what DQL does in short. This is what I wrote: DQL: instead of a Q-table, a DNN is used to approximate the Q-values for each action based on a ...
Ness's user avatar
  • 206
1 vote
1 answer
132 views

Formally, what are the layers in an Artificial Neural Network?

You may not believe it, but I am an ANN expert. Perhaps, for that reason, I am unable to grasp completely what the layers are in a Deep Forward Artificial Neural Network (DFANN). According to the Deep ...
neoglez's user avatar
  • 11
0 votes
2 answers
470 views

How to handle the size difference of highway network or residual network in cnn?

For highway network, it looks like this: For residual network, it looks like this: Pictures are from What is the name of this neural network architecture with layers that are also connected to non-...
liaoming999's user avatar
0 votes
0 answers
39 views

Does the number of epochs measure a correlation?

i have built a two-layers neural network (1000 => 1000) to predict a dynamical system driven by two real-world parameters. When using the first parameter as input to the first layer, training the ...
Bouarfa Mahi's user avatar
1 vote
2 answers
118 views

Does it make sense to build a two-layer neural network with a triangular weight matrix?

I need to implement a rule and have defined a lower triangular boolean mask for the weights that I want to keep static for a zero value. In which condition triangular weight matrix will be used?
Bouarfa Mahi's user avatar
1 vote
2 answers
2k views

Why should one expect the backward pass to take twice as long as the forward pass?

I have seen it stated that, as a rule of thumb, a backward pass in a neural network should take about twice as long as the forward pass. Examples: From DeepSpeed's Flops Profiler docs, the profiler: ...
user26866's user avatar
  • 111
0 votes
1 answer
742 views

Is it redunant to add more layers to a neural network with same number of neurons as the previous layer?

Lets say I have a neural network with three layers and the last layer has 3 outputs. If I added additional layer of 3 neurons to the end of the network, would that be a more powerful neural network? ...
Steven Sagona's user avatar
1 vote
1 answer
556 views

Why MSE and MAE yield poor results when used with gradient-based optimization for classification?

Deep learning book chapter 6: In 6.2.1.2 last paragraph: Unfortunately, mean squared error and mean absolute error often lead to poor results when used with gradient-based optimization. Some output ...
vivian.ai's user avatar
0 votes
3 answers
1k views

why validation accuracy be greater than training accuracy for deep learning models? [closed]

I hope you are well. I had a problem and didn't understand the answers given on questions similar to my question. If possible, please answer this problem in a simpler way. Val_acc : %99.4 _ Train_acc :...
maserati urm's user avatar
1 vote
1 answer
174 views

MobileNetV2 - Some particularities

So I was studying MobileNetV2 architecture and came across this table from the original paper that represents its architecture: Table Description: "Table 2: MobileNetV2 : Each line describes a ...
Blue Ross's user avatar
0 votes
1 answer
122 views

Training a neural network in full batch training

It is a trend in deep learning to train models using multi-batches, i.e., to show the model a subset of the entire dataset for each weight update. In some cases, as in continual learning, we see that ...
Alfred's user avatar
  • 165
1 vote
4 answers
159 views

Why, in deep learning, do we get computational power by going deeper?

I know by the expressiveness of a neural networks that it can be seen as a chain of function compositions, i.e. $g(f(.. z(x)..))$ and also that, if we go deep, we can approximate complex functions $f: ...
f_s's user avatar
  • 11

1
2 3 4 5