All Questions
172 questions with no upvoted or accepted answers
5
votes
1
answer
553
views
How does SGD escape local minima?
SGD is able to jump out of local minima that would otherwise trap BGD
I don't really understand the above statement. Could someone please provide a mathematical explanation for why SGD (Stochastic ...
- 1,171
5
votes
3
answers
1k
views
What's the difference between architectures and backbones?
In the paper "ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagery", the authors talk about using:
Feature Pyramid Networks (as the ...
4
votes
1
answer
255
views
Is there any relation between the recursive neural network and recurrent neural network?
Recurrent neural networks, abbreviated as RNNs, are widely used in deep learning literature, especially for text processing.
Are they related to recursive neural networks in any way?
I am asking for ...
- 3,990
4
votes
0
answers
292
views
Why are most commonly used activation functions continuous?
I have come to notice that the most commonly used activation functions are continuous. Is there any specific reason behind this? Results such as this paper have worked on training networks with ...
- 564
4
votes
0
answers
61
views
What is the difference between "out-of-distribution (generalisation)" and "(meta)-transfer learning"?
I'm trying to develop a better understanding of the concept of "out-of-distribution" (generalization) in the context of Bengio's "Moving from System 1 DL to System 2 DL" and the concept of "(meta)-...
4
votes
1
answer
617
views
What are the differences between Bytenet and Wavenet?
I recently read Bytenet and Wavenet and I was curious why the first model is not as popular as the second. From my understanding, Bytenet can be seen as a seq2seq model where the encoder and the ...
- 1,158
4
votes
0
answers
614
views
What is the difference between GAT and GaAN?
I was looking at two papers
Graph Attention Networks (GAT) by Petar Veličković and
GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs by Jiani Zhang.
I'm trying to ...
- 1,158
4
votes
1
answer
58
views
How to use MOPSO to align characters vertically?
I need to efficiently align characters vertically using Multi Objective PSO. Alignment is achieved by adding spaces in between a given set of characters.
...
- 253
3
votes
1
answer
634
views
What's the difference between domain randomization and domain adaptation?
In my understanding, domain randomization is one method of diversifying the dataset to achieve a better shot at domain adaptation. Am I wrong?
- 131
3
votes
0
answers
201
views
What is the difference between fuzzy neural networks and adaptive neuro fuzzy inference systems?
I have, like you see, just a general question about the combination of fuzziness and neural networks. I understood it as follows
Fuzzy neural networks as a hybrid system: the neural network helps me ...
- 51
3
votes
0
answers
43
views
How do reinforcement learning and collaborative learning overlap?
How do reinforcement learning and collaborative learning overlap? What are the differences and similarities between these fields?
I feel like the results I get via google do not make the distinction ...
- 297
3
votes
0
answers
133
views
Are No Free Lunch theorem and Universal Approximation theorem contradictory in the context of neural networks?
To my understanding NFL states that, we cannot have an hypothesis (let's assume it is an approximator like NN in this case) class that can't achieve certain accuracy parameters $\leq \epsilon$ with ...
user9947
3
votes
0
answers
78
views
Can the importance sampling estimator have a non-stationary behaviour policy even if the target policy is stationary?
The inverse propensity score (IPS) estimator, which is used for off-policy evaluation in a contextual bandit problem, is well explained in the paper Doubly Robust Policy Evaluation and Optimization.
...
- 227
3
votes
0
answers
220
views
What is the difference between Squeeze-and-excite and bottleneck modules from Mobilenet v2?
Squezee-and-excite networks introduced SE blocks, while MobileNet v2 introduced linear bottlenecks.
What is the effective difference between these two concepts?
Is it only implementation (depth-wise ...
- 101
3
votes
0
answers
122
views
Does Retina-net's focal loss accomplish its goal?
Taking out the weighting factor we can define focal loss as
$$FL(p) = -(1-p)^\gamma log(p) $$
Where $p$ is the target probability. The idea being that single stage object detectors have a huge ...
- 2,389
3
votes
0
answers
366
views
What are the differences between CRF and HMM?
What I know about CRF is that they are discriminative models, while HMM are generative models, but, in the inference method, both use the same algorithm, that is, the Viterbi algorithm, and forward ...
- 161
3
votes
0
answers
310
views
What are the key differences between cellular neural network and convolutional neural network?
What are the key differences between cellular neural networks and convolutional neural networks in terms of working principle, implementation, potential performance, and applicability?
3
votes
0
answers
231
views
What is the relation between the definition of learnability of Vapnik and Gold and learnability of neural networks?
Gold showed that a language can be learned only if it contains a finite set of sentences.
We know that deep neural networks can implement any function. Does this contradict the Gold's result?
What ...
2
votes
1
answer
38
views
Is there a better way to do this type of optimization?
I have an image classification task that uses an object detection model as its basis. For each image, I get a vector of confidences (one value for each class), and I take the class with the highest ...
- 21
2
votes
0
answers
70
views
Can local learning rules minimize a global loss?
It is widely believed that synaptic plasticity is the way biological brains learn. Artificial implementations of this mechanism are for instance local weight-update rules in Spiking Neural Networks. ...
- 121
2
votes
1
answer
140
views
Multi-objective training involving maximization of one loss function and minimization of another
I need my model to predict $s$ from my data $x$. Additionally, I need the model to not use signals in $x$ that are predictive of a separate target $a$. My approach is to transform $x$ into a ...
- 121
2
votes
0
answers
112
views
When are traditional image processing methods preferable to machine learning and why?
By traditional image processing I understand, e. g. using filters to improve the image, extracting edges and then classifying objects using template matching.
My current decision criteria are:
large ...
- 21
2
votes
0
answers
32
views
How to create a loss function that penalizes duplicate indices in the output tensor?
We're working on a sequence-to-sequence problem using pytorch, and are using cross-entropy to calculate the loss when comparing the output sequence to the target sequence. This works fine and ...
- 121
2
votes
0
answers
61
views
What are the specific differences between vision transformers variants?
I have tried 4 different types of attacks on vision transformers (ViT small and tiny, DeiT small and tiny) but the attack successes on smaller versions are higher than the tiny versions. My ...
- 31
2
votes
0
answers
28
views
What are the benefits of using spectral k-means over simple k-means?
I have understood why k-means can get stuck in local minima.
Now, I am curious to know how the spectral k-means helps to avoid this local minima problem.
According to this paper A tutorial on Spectral,...
- 121
2
votes
0
answers
119
views
When to model decision-making problem as single agent vs multi-agent problem?
I understand the goals and purposes of RL in the case of a single agent and the underlying model, i.e. MDPs, for RL problems (or sequential decision making with uncertainty in general).
My question is ...
- 121
2
votes
1
answer
74
views
Can teacher forcing in RNN ensure Turing completeness?
RNN has the same capability as a universal Turing machine. But I am confused whether RNN holds the same capabilities if we use teacher forcing.
Consider the following excerpts from paragraphs taken ...
- 3,990
2
votes
0
answers
153
views
What is the difference between Probabilistic Graphical models and Graph Neural networks?
While going over PGMs and GNNs, it seems like both leverage the graph data structure. The former has been used to represent causal associations (among other things), while the latter has a varied set ...
- 670
2
votes
1
answer
173
views
Closed networks vs Networks with a removed delay to predict new data
I've come across two types of neural networks to predict, both from Matlab, the closed structure and the net that removes one delay to find new data.
From Matlab's app generated scripts we see:
% ...
2
votes
1
answer
860
views
Is there any difference between "image generation" and "image synthesis"?
Generative Adversarial networks (aka GANs) are used for image generation. The phrase image synthesis is also used in literature.
I know that the phrase image generation stands for
An act of ...
- 3,990
2
votes
0
answers
94
views
Do the terms multi-task and multi-output refer to the same thing in the context of deep learning?
Do the terms multi-task and multi-output refer to the same thing in the context of deep learning (with neural networks)? For example, do neural networks for multi-task learning use multiple outputs?
...
- 339
2
votes
0
answers
32
views
Is it possible to ensure the convergence when training a RNN weight on its SVD decomposition?
I'm reading the following paper in which the author seems to do 2 things interesting:
The hidden-to-hidden weight matrix of the RNN is SVD decomposed and train separately.
Each orthogonal part of the ...
- 121
2
votes
0
answers
72
views
How and why do state-of-the-art models in medical segmentation differ from general segmentation models?
I am just getting into medical image segmentation and have been able to understand the state-of-the-art architectures, like Double UNet, UNet++, and Multiresunet.
What I haven't understood yet: Why ...
- 635
2
votes
0
answers
63
views
How does the support vector machine constraint imply that sample selection bias will not systematically affect the output of the optimisation?
I am currently studying the paper Learning and Evaluating Classifiers under Sample Selection Bias by Bianca Zadrozny. In section 3.4. Support vector machines, the author says the following:
3.4. ...
- 603
2
votes
0
answers
57
views
How are the lower and upper bound values of the moths determined in the Moth-Flame Optimization algorithm?
I am currently implementing the Moth-Flame Optimization (MFO) Algorithm, based on the paper: Moth-Flame Optimization Algorithm: A Novel Nature-inspired Heuristic Paradigm.
To calculate the values of ...
- 139
2
votes
0
answers
103
views
What is the difference between text-based image retrieval and natural language object retrieval?
I'm working on creating a model that locates the object in the scene (2D image or 3D scene) using a natural language query. I came across this paper on natural language object retrieval, which ...
- 21
2
votes
1
answer
623
views
Continuous state and continuous action Markov decision process time complexity estimate: backward induction VS policy gradient method (RL)
Model Description: Model based (assume known of the entire model) Markov decision process.
Time($t$): Finite horizon discrete time with discounting factor
State($x_t$): Continuous multi-dimensional ...
- 21
2
votes
0
answers
99
views
How are the classical MDP and the object-oriented MDP views different?
I've been reading the attached paper - which aims to model entities in the world as objects, including the learning agent itself!
To say the least, the goal is to navigate through what seems like a ...
- 1,171
2
votes
0
answers
132
views
What is the relationship between PAC learning and classic parameter estimation theorems?
What are the differences and similarities between PAC learning and classic parameter estimation theorems (e.g. consistency results when estimating parameters, e.g. with MLE)?
- 847
2
votes
0
answers
89
views
What is the difference between training a model with RGB images and using only the color channels separately?
What is the difference between training a model with RGB images and using only the color channels separately (like only the red channel, green channel, etc.)? Would the model also learn patterns ...
- 175
2
votes
0
answers
262
views
What are the advantages and disadvantages of extrinsic and perplexity model evaluation in NLP?
In the video Evaluation and Perplexity by Dan Jurafsky, the author talks about extrinsic and perplexity evaluation in the context of natural language processing (NLP).
What are the advantages and ...
- 1,763
2
votes
0
answers
299
views
What is the difference between using a backbone architecture and transfer learning?
I'm super new to deep learning and computer vision, so this question may sound dumb.
In this link (https://github.com/GeorgeSeif/Semantic-Segmentation-Suite), there are pre-trained models (e.g., ...
- 21
2
votes
0
answers
48
views
How can I compare EEG data with accelerometer data in 1 algorithm?
I have frequency EEG data from fall and non-fall events and I am trying to incorporate it with accelerometer data that was collected at the same time. One approach is, of course, to use two separate ...
- 21
2
votes
0
answers
99
views
What is the difference between tracking and mapping (TAM) and localization and mapping (LAM)?
In the paper Visual SLAM algorithms: a survey from 2010 to 2016 by Takafumi Taketomi, Hideaki Uchiyama and Sei Ikeda it is mentioned
It should be noted that tracking and mapping (TAM) is used instead ...
- 151
2
votes
0
answers
65
views
Is logistic regression used for unconstrained or constrained optimisation problems?
Is logistic regression used for unconstrained or constrained optimization problems, and why?
- 21
2
votes
0
answers
122
views
What is the difference between evolutionary game theory and meta-heuristics?
Here is a list of meta-heuristic algorithms
Ant colony optimization,
Ant lion optimizer,
Artificial bee colony algorithm,
Bat algorithm,
Cat swarm optimization,
Crow search algorithm,
Cuckoo ...
- 1,763
2
votes
0
answers
121
views
How can I assign agents to tasks based on time and affinity?
I am working on an assignment problem.
Consider $K$ agents $A_1, \dots A_K$ and $N$ tasks $T_1, \dots T_N$. Each task has a certain time $t(T_i)$ to be completed and each agent has a matching (or ...
2
votes
0
answers
92
views
What is the complexity of policy gradient algorithms compared to discrete action space algorithms?
I am using a policy gradient algorithm (actor-critic) for wireless networks. The policy gradient-based algorithm helps because it considers continuous action space.
But how much does a policy ...
- 21
2
votes
0
answers
66
views
Is a neural network the correct approach to optimising a fitness function in a genetic algorithm?
I've written an application to help players pick the optimal heroes during the draft phase of the Heroes of the Storm MOBA. It can be daunting to pick from 80+ characters that have synergies/counters ...
- 121
2
votes
1
answer
130
views
Are there any advantages of using rules-based approaches versus models for detecting spam?
Suppose that we have unlabeled data. That is, all we have are a collection of emails and want to determine whether any of them is spam or not. Let's say we have $1,000$ rules to determine whether a ...
- 21