# Tag Info

13

The paper's authors needed to implement their models anyway in order to conduct their experimentations, so why not publish the implementation? Some papers and authors actually provide a link to their own implementation, but most of the papers (that I have read) don't provide it, although some third-party implementations may already be available on Github (...

6

When crossover happens and one parent is fitter than the other, the nodes from the more fit parent are carried over to the child. This is the case as disjoint and excess genes are only carried over from the fittest parent. Here's an example: // Node Crossover Parent 1 Nodes: {[0][1][2]} // more fit parent Parent 2 Nodes: {[0][1][2][3]} Child Nodes: {[0]...

5

Someone can argue to some human adequate reasons, but there is a bad trend of falsified results in deep learning research papers that propose some nowel solutions or even update state-of-the-art model performance. And that's not just a few papers that lie, it's a large portion of them. And the reason for that is even more sad - most of so-called deep ...

5

The first reason described in nbro's answer can definitely be an important one; authors may have implemented their software using code that they can't share. There's a lot of research coming out of companies (large and small), and they may use all sorts of proprietary libraries that were built in the company and cannot be distributed outside. As described in ...

4

Go predictions were included in the paper: The experts are far from infallible. They predicted that AI would be better than humans at Go by about 2027. (This was in 2015, remember.) SOURCE: Experts Predict When Artificial Intelligence Will Exceed Human Performance (MIT Tech Review)

4

A non-starving policy is a (behavior) policy that is theoretically guaranteed to visit each state and take all possible actions from each state an infinite number of times, so that to always update $Q(s, a)$, $\forall s, \forall a$, an infinite number of times. In the context of off-policy prediction, this criterion implies that any trajectory will have no ...

4

In language theory, there are generally several admitted levels that can be studied in relation with one another or independently. The semantic level is the one dealing with the meaning of the text ("semantic" comes from the greek and means "to signify"). The semantic level is therefore generally independent from the syntax and even the language used to ...

3

The square brackets $[]$ in $[\tau_{ij}]^\alpha$ and $[\eta_{ij}]^\beta$ may be just a way of emphasing that the elements $\tau_{ij} \in \mathbb{R}$ and $\eta_{ij} \in \mathbb{R}$ of respectively the matrices $\mathbf{\tau} \in \mathbb{R}^{n \times n}$ and $\mathbf{\eta} \in \mathbb{R}^{n \times n}$ (where $n$ is the number of nodes in the graph) are ...

3

locality of pixel dependencies probably means that neighboring pixels tend to be correlated, while faraway pixels are usually not correlated. This assumption is usually made in several image processing techniques (e.g. filters). Of course, the size and the shape of the neighborhood could vary, depending on the region of the image (or whatever), but, in ...

3

I recommend you focus on quality over quantity. Publishing a paper will boost your reputation and make you more recognised within your academic field (AI), however this is only if the paper provides useful insights into an important issue. Your paper is more likely to be accepted if it is well written and easy to understand, stimulates new important ...

3

Dennis Soemers provides an important point that from a theoretical standpoint, this can be seen as a non-issue. However, what you bring up is an important practical issue of potential-based reward shaping (PBRS). The issue is actually worse than you describe---it's more general than s = s'. In particular, the issue presents itself differently based on the ...

3

I don't think the situation you're sketching should be a problem at all. If $P(s)$ is high (e.g. $P(s) = 1000$), this means (according to your shaping / "heuristic") that it's valuable to be in the state $s$, that you expect to be able to get high future returns from that state. If you then continuously take actions that keep you in the same state $s$, it ...

3

It was difficult to find because recurrent network designs predate LSTM extensions of that earlier idea by decades. Although the term recurrent was not yet used as a primary description of the technology advancement, recurrence was an essential feature of the theoretical treatment of artificial networks that learned actions in Attractor dynamics and ...

3

All right, I figured it out. trajectories need not have the same starting state because the distribution of $s_0$ is drawn from a distribution D (mentioned in the paper). Had been confused because many of the code implementations on github focus on trajectories starting from the same state. Hope this helps everyone !

3

2

It looks like you're asking about the difference between using conditional and joint probabilities. The joint probability $$D(x,y)$$ is the probability of x and y both happening together. The conditional probability $$D(x | y)$$ is the probability that x happens, given that y has already happened. So, $$D(x,y) = D(y) * D(x | y)$$. Notice that, in a C-GAN,...

2

One important consideration here: in the last decade or two the machine learning and artificial intelligence fields, which contains the majority of reinforcement learning work, researchers have considered conferences to be the more impactful publishing venues than journals. The particular venue a researcher chooses depends on the data and/or application ...

2

Your interpretation is quite correct. I could not understand how it would speed up the convergence though. What they are doing is basically re-assigning the magnitude of the weight vector (also called norm of the weight vector). To put things in the perspective, the conventional approach to any Machine Learning cost function is to not only check the ...

2

In laymen's terms, a non-expansive operator is a function that brings points closer together or at least no further apart. An example of a non-expansive operator is the function $f(x) = x/2$. The two numbers $0$ and $5$ are a distance of $5$ apart. The two output numbers $f(0) = 0$ and $f(5) = 2.5$ are 2.5 apart (which is smaller than $5$ apart). It is easy ...

2

First of all you made a mistake, equation 8 in the paper is defined with $\frac{\partial L(\theta)}{\partial s_t}$ not $\frac{\partial L(\theta)}{\partial\theta}$. Loss is defined as: $L(\theta) = - \mathbb{E}_{w^s \sim p_{\theta}}[r(w^s)]$ If we use definition of expectation (for discrete case): $\mathbb{E}[X] = \sum\limits_{i} p_i(x_i)x_i$ we get ...

2

The adjective bad isn't mathematically descriptive. A better term is sub-optimal, which implies the state of learning might appear optimal based on current information but the optimal solution from among all possibilities is not yet located. Consider a graph representing a loss function, one of the names to measure disparity between the current learning ...

2

Poel's paper on Translating Values into Design Requirements articulates a framework for mapping abstract values and norms into concrete design constraints that an engineer could work with. The example used in the paper is mapping beliefs about animal welfare to design constraints on chicken coups. The newer paper by Tubella et al. on Governance by Glass-...

2

So this paper is by google, but is very similar where they use 2D positional embeddings and perform MHA on the flattened image. Are you talking about Attention Augmented Convolutional Networks

2

The paper you are citing is the paper that introduced the cascaded convolution neural network. In fact, in this paper, the authors say To realize 3DDFA, we propose to combine two achievements in recent years, namely, Cascaded Regression and the Convolutional Neural Network (CNN). This combination requires the introduction of a new input feature which ...

2

I think that making some draws might help. Below I tried to draw the model architecture. We start with classic feed-forward structure: input represented by a vector I with length f (number of features), a hidden layer H which does not have a fixed size, and output O of length c (number of classes). Then we have 3 extra vectors than usual: a vector U they ...

2

In full: The limit, as standard deviation $\sigma$ tends towards zero, of the gradient with respect to vector $\mathbf{x}$, of the expectation - where perturbation $\epsilon$ follows the normal distribution with mean 0 and variance $\sigma^2$ times identity vector $[1,1,1,1...]$ * - of any function $f$ of $\mathbf{x}$ plus $\epsilon$ is equal to the ...

2

Well, here in the picture we have the unrolled or unfold RNN on the right side. Siamese network is formed when it is said to be "unrolled over two time-steps". So, take part where there is two first iterations of RNN and yes, you have kind of Siamese network. One take from the source of the image: Unlike a traditional deep neural network, which uses ...

2

Here, $H$ is a $n * d$ matrix where $n$ is the number of total nodes in the graph and $d$ is the dimension of embedding of each node. Using the notation in the question, the basic GNN formulation without self loop is: $\text{D}^{-1}\text{A}\text{H}$. If you study this equation closer then you will find that the $i^{th}$ row of $\text{A}\text{H}$ generates ...

1

There are definitely approaches that are theory driven (like SVMs), and others where the theory comes after the practice (like a lot of deep neural networks). I think it would be difficult to argue strongly that either direction is more common "in general" within AI, or indeed, within any other branch of science. The approach that is currently in favor ...

1

Temporal Depth is a third parameter of a time series data. For example if you have a video clip of length 25 frames and on training a model you are giving first five frames with respect to time. Your temporal depth will be 5.

Only top voted, non community-wiki answers of a minimum length are eligible