All Questions

3,715 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
14 votes
1 answer
244 views

Will parameter sweeping on one split of data followed by cross validation discover the right hyperparameters?

Let's call our dataset splits train/test/evaluate. We're in a situation where we require months of data. So we prefer to use the evaluation dataset as infrequently as possible to avoid polluting our ...
user avatar
13 votes
3 answers
494 views

Can some one help me understand this paragraph from Nvidia's progressive GAN paper?

Furthermore, we observe that mode collapses traditionally plaguing GANs tend to happen very quickly, over the course of a dozen mini-batches. Commonly they start when the discriminator overshoots, ...
user avatar
  • 393
12 votes
0 answers
260 views

What is the number of neurons required to approximate a polynomial of degree n?

I learned about the universal approximation theorem from this guide. It states that a network even with a single hidden layer can approximate any function within some bound, given a sufficient number ...
user avatar
  • 663
10 votes
0 answers
276 views

Extending FaceNet’s triplet loss to object recognition

FaceNet uses a novel loss metric (triplet loss) to train a model to output embeddings (128-D from the paper), such that any two faces of the same identity will have a small Euclidean distance, and ...
user avatar
9 votes
0 answers
149 views

Why is my GAN more unstable with bigger networks?

I am working with generative adversarial networks (GANs) and one of my aims at the moment is to reproduce samples in two dimensions that are distributed according to a circle (see animation). When ...
user avatar
  • 134
9 votes
2 answers
366 views

Back-of-the-envelope machine learning (specifically neural networks) calculations

There is a popular story regarding the back-of-the-envelope calculation performed by a British physicist named G. I. Taylor. He used dimensional analysis to estimate the power released by the ...
user avatar
  • 281
9 votes
0 answers
764 views

Are Cellular Neural Networks one type of Neural Networks?

I am researching Cellular Neural Networks and have already read Chua's two articles (1988). In cellular neural networks, a cell is only in relation with its neighbors. So it is easy to use them for ...
user avatar
  • 273
8 votes
1 answer
196 views

How to deal with a small amount of labeled samples?

I'm trying to develop skills to deal with very small amounts of labeled samples (250 labeled/20000 total, 200 features) by practicing on Kaggle "Don't Overfit" dataset (Traget_Practice have ...
user avatar
  • 181
8 votes
1 answer
179 views

Why are documents kept separated when training a text classifier?

Most of the literature considers text classification as the classification of documents. When using the bag-of-words and Bayesian classification, they usually use the statistic TF-IDF, where TF ...
user avatar
  • 236
7 votes
1 answer
111 views

Validation accuracy higher than training accurarcy

I implemented the unet in TensorFlow for the segmentation of MRI images of the thigh. I noticed I always get a higher validation accuracy by a small gap, independently of the initial split. One ...
user avatar
7 votes
0 answers
200 views

Is the Bellman equation that uses sampling weighted by the Q values (instead of max) a contraction?

It is proved that the Bellman update is a contraction (1). Here is the Bellman update that is used for Q-Learning: $$Q_{t+1}(s, a) = Q_{t}(s, a) + \alpha*(r(s, a, s') + \gamma \max_{a^*} (Q_{t}(s', ...
user avatar
7 votes
2 answers
256 views

How can I solve the zero subset sum problem with hill climbing?

I want to solve the zero subset sum problem with the hill-climbing algorithm, but I am not sure I found a good state space for this. Here is the problem: consider we have a set of numbers and we want ...
user avatar
7 votes
0 answers
76 views

Normalizing Normal Distributions in Thompson Sampling for online Reinforcement Learning

In my implementation of Thompson Sampling (TS) for online Reinforcement Learning, my distribution for selecting $a$ is $\mathcal{N}(Q(s, a), \frac{1}{C(s,a)+1})$, where $C(s,a)$ is the number of times ...
user avatar
  • 71
7 votes
1 answer
79 views

What is the impact of using multiple BMUs for self-organizing maps?

Here's a sort of a conceptual question. I was implementing a SOM algorithm to better understand its variations and parameters. I got curious about one bit: the BMU (best matching unit == the neuron ...
user avatar
7 votes
1 answer
104 views

How does the network know which objects to track in the paper "Label-Free Supervision of Neural Networks with Physics and Domain Knowledge"?

I was reading the paper Label-Free Supervision of Neural Networks with Physics and Domain Knowledge, published at AAAI 2017, which won the best paper award. I understand the math and it makes sense. ...
user avatar
7 votes
1 answer
222 views

Why do layered neural nets struggle with continous data?

In this article here, the writer claims that a new type of neural net is required to deal with data that is both continuous, and also sparsely sampled. It was my understanding that this was the ...
user avatar
  • 171
7 votes
0 answers
1k views

Is there a difference in the architecture of deep reinforcement learning when multiple actions are performed instead of a single action?

I've built a deep deterministic policy gradient reinforcement learning agent to be able to handle any games/tasks that have only one action. However, the agent seems to fail horribly when there are ...
user avatar
  • 393
6 votes
0 answers
81 views

How can a neural network distinguish a rotated 6 and 9 digits?

Rotated MNIST is a popular dataset for benchmarking models equivariant to rotations on $\mathbb{R}^2$, described by $SO(2)$ group or its discrete subgroups like $\mathbb{Z}^{n}$: Group equivariant ...
user avatar
6 votes
1 answer
263 views

What are the state-of-the-art results in OpenAI's gym environments?

What are the state-of-the-art results in OpenAI's gym environments? Is there a link to a paper/article that describes them and how these SOTA results were calculated?
user avatar
6 votes
1 answer
112 views

Are there transformer-based architectures that can produce fixed-length vector encodings given arbitrary-length text documents?

BERT encodes a piece of text such that each token (usually words) in the input text map to a vector in the encoding of the text. However, this makes the length of the encoding vary as a function of ...
user avatar
6 votes
2 answers
80 views

Can training a model on a dataset composed by real images and drawings hurt the training process of a real-world application model?

I'm training a multi-label classifier that's supposed to be tested on underwater images. I'm wondering if feeding the model drawings of a certain class plus real images can affect the results badly. ...
user avatar
  • 91
6 votes
1 answer
157 views

How could an AI detect whether an enemy in a game can be blocked off/trapped?

Imagine a game played on a 10x10 grid system where a player can move up down left or right and imagine there are two players on this grid: An enemy and you. In this game, there are walls on the grid ...
user avatar
  • 61
6 votes
0 answers
100 views

How do neural network topologies affect GPU/TPU acceleration?

I was thinking about different neural network topologies for some applications. However, I am not sure how this would affect the efficiency of hardware acceleration using GPU/TPU/some other chip. If, ...
user avatar
6 votes
1 answer
315 views

An intuitive explanation of Adagrad, its purpose and its formula

It (Adagrad) adapts the learning rate to the parameters, performing smaller updates (i.e. low learning rates) for parameters associated with frequently occurring features, and larger updates (i.e. ...
user avatar
  • 103
6 votes
0 answers
289 views

Why don't people use nonlinear activation functions after projecting the query key value in attention?

Why don't people use nonlinear activation functions after projecting the query key value in attention? It seems like doing this would lead to much-needed nonlinearity, otherwise, we're just doing ...
user avatar
  • 530
6 votes
0 answers
196 views

What are the main benefits of using Bayesian networks?

I have some trouble understanding the benefits of Bayesian networks. Am I correct that the key benefit of the network is that one does not need to use the chain rule of probability in order to ...
user avatar
6 votes
0 answers
94 views

What are the current trends/open questions in logics for knowledge representation?

What are the future prospects in near future from a theoretical investigation of description logics, and modal logics in the context of artificial intelligence research?
user avatar
  • 61
6 votes
0 answers
4k views

Does it make sense to use batch normalization in deep (stacked) or sparse auto-encoders?

Does it make sense to use batch normalization in deep (stacked) or sparse auto-encoders? I cannot find any resources for that. Is it safe to assume that, since it works for other DNNs, it will also ...
user avatar
  • 211
6 votes
1 answer
300 views

Which neural networks are suitable for visual place recognition?

I am doing a project on visual place recognition in changing environments. The CNN used here is mostly AlexNet, and a feature vector is constructed from layer 3. Does anyone know of similar work ...
user avatar
5 votes
0 answers
58 views

Is there a venue to publish negative results in AI/ML domain?

Negative results occur frequently in AI/ML research (and perhaps in other domains too). Most of the time, these results are not published. This is mostly because your typical AI/ML conference doesn't ...
user avatar
5 votes
0 answers
67 views

Proof that there always exists a dominating policy in an MDP

I think that it is common knowledge that for any infinite horizon discounted MDP $(S, A, P, r, \gamma)$, there always exists a dominating policy $\pi$, i.e. a policy $\pi$ such that for all policies $\...
user avatar
  • 175
5 votes
0 answers
103 views

Why can't cognitive architectures achieve general intelligence?

Newbie here. I recently read about cognitive architectures (see: https://en.wikipedia.org/wiki/Cognitive_architecture). They are supposed to be modeled after the human mind and represent a promising ...
user avatar
5 votes
0 answers
45 views

Are generative models actually used in practice for industrial drug design?

I just finished reading this paper MoFlow: An Invertible Flow Model for Generating Molecular Graphs. The paper, which is about generating molecular graphs with certain chemical properties improved the ...
user avatar
  • 51
5 votes
0 answers
52 views

Origins of the name of convolutional neural networks

Convolutional neural networks (CNNs) contain convolutional layers. In modern deep learning libraries such as Tensorflow and PyTorch among others, convolutional layers are implemented by using the ...
user avatar
  • 371
5 votes
0 answers
54 views

It is possible to use deep learning to give approximate solutions to NP-hard graph theory problems?

It is possible to use deep learning to give approximate solutions to NP-hard graph theory problems? If we take, for example, the travelling salesman problem (or the dominating set problem). Let's say ...
user avatar
  • 171
5 votes
0 answers
445 views

Wasserstein GAN: Implemention of Critic Loss Correct?

The WGAN paper concretely proposes Algorithm 1 (cf. page 8). Now, they also state what their loss for the critic and the generator is. When implementing the critic loss (so lines 5 and 6 of Algorithm ...
user avatar
5 votes
0 answers
81 views

Why is there a Uniform and Normal version of He / Xavier initialization in DL libraries?

Two of the most popular initialization schemes for neural network weights today are Xavier and He. Both methods propose random weight initialization with a variance dependent on the number of input ...
user avatar
  • 600
5 votes
0 answers
105 views

$\frac{P(x_1 \mid y, s = 1) \dots P(x_n \mid y, s = 1) P(y \mid s = 1)}{P(x \mid s = 1)}$ indicates that naive Bayes learners are global learners?

I am currently studying the paper Learning and Evaluating Classifiers under Sample Selection Bias by Bianca Zadrozny. In section 3. Learning under sample selection bias, the author says the following: ...
user avatar
5 votes
0 answers
105 views

Why Pixel RNN (Row LSTM) can capture triangular contexts?

I'm reading the paper Pixel Recurrent Neural Network. I have a question about Row LSTM. Why Row LSTM can capture triangular contexts? In this paper, the kernel of the one-dimensional convolution ...
user avatar
5 votes
1 answer
115 views

How to classify human actions?

I'm quite new to machine learning (I followed the Coursera course of Andrew Ng and now starting deeplearning.ai courses). I want to classify human actions real-time like: Left-arm bended Arm above ...
user avatar
5 votes
0 answers
2k views

How many training data is required for GAN?

I'm beginning to study and implement GAN to generate more datasets. I'll just try to experiment with state-of-the-art GAN models as described here https://paperswithcode.com/sota/image-generation-on-...
user avatar
  • 195
5 votes
0 answers
117 views

Are there any easy ways to create annotated training images for object detection?

For the purposes of object detection, are there any easy ways to create annotated training images? For example, if we have $10,000$ images and want to draw bounding boxes on 2 objects for each image, ...
user avatar
  • 61
5 votes
2 answers
1k views

What evaluation metric are used for sequence-to-sequence prediction problems?

I am solving many sequence-to-sequence prediction problems using RNN/LSTM. What type of evaluation metrics can be used for sequence prediction problems? One metric is the mean squared error (MSE) ...
user avatar
  • 171
5 votes
0 answers
431 views

How is the rollout from the MCTS implemented in both of the AlphaGo Zero and the AlphaZero algorithms?

In the vanilla Monte Carlo tree search (MCTS) implementation, the rollout is usually implemented following a uniform random policy, that is, it takes random actions until the game is finished and only ...
user avatar
5 votes
1 answer
168 views

How do big companies, like Facebook, model individuals and their interaction?

As a layman in AI, I want to get an idea of how big data players, like Facebook, model individuals (of which they have so many data). There are two scenarios I can imagine: Neural networks build ...
user avatar
5 votes
0 answers
63 views

What is meant by "model discriminability for local patches within the receptive field"?

In the abstract of the paper Network In Network, the authors write We propose a novel deep network structure called "Network In Network"(NIN) to enhance model discriminability for local ...
user avatar
5 votes
0 answers
309 views

What are the ways to calculate the error rate of a deep Convolutional Neural Network, when the network produces different results using the same data?

I am new to the object recognition community. Here I am asking about the broadly accepted ways to calculate the error rate of a deep CNN when the network produces different results using the same data....
user avatar
5 votes
0 answers
114 views

Speciation in NEAT - Advantages of keeping stable number of species

I found several methods for setting the compatibility distance in NEAT: some normalize it, some don't, some automatically adjust it. In a few tests I am running, using normalized static compatibility ...
user avatar
  • 331
5 votes
0 answers
210 views

Is a mathematical formula a form of intelligence?

Warning: This question takes us into VALIS territory, but I wouldn't underestimate the profundity of that particular philosopher. There is a non-AI definition of intelligence which is simply "...
user avatar
  • 6,077
5 votes
1 answer
94 views

Correcting 'bad' translations in a sequence-to-sequence neural machine translation model

In working with basic sequence-to-sequence models for machine translation I have been able to achieve decent results. But inevitably some translations are not optimal or just flat-out incorrect. I am ...
user avatar

15 30 50 per page
1
2 3 4 5
75