Questions tagged [implementation]

For questions about implementation of software/algorithms related to Artificial Intelligence.

Filter by
Sorted by
Tagged with
1 vote
1 answer
41 views

OpenAI Gym implementation of the delayed rewards

My question is about if is it possible to implement delayed reward logic within Gym environment. More specifically, I work on ride-pooling RL algorithm, when the action (choice of the parameters of ...
Klavdiia Bochenina's user avatar
1 vote
1 answer
145 views

Why in Multi-Head Attention implementation should we use $3$ linear layers for Q, K, V instead of $3 * h$ layers?

I have been trying to implement a Transformer architecture using PyTorch by following the Attention Is All You Need paper as well as the The Annotated Transformer blog post to compare my code with ...
Daviiid's user avatar
  • 573
0 votes
0 answers
56 views

A2C unable to solve Cartpole

I have coded my own A2C implementation using PyTorch. However, despite having followed the algorithm pseudo-code from several sources, my implementation is not able to achieve a proper Cartpole ...
Antonio's user avatar
  • 26
0 votes
0 answers
29 views

How to find AI research papers where authors provide source codes that will reproduce the experiments described in the paper?

It is important for a research paper to include raw data and code for scientific replicability, verifiability, and falsifiability. However, as noted by nbro, roughly 5% percent of papers have code ...
High GPA's user avatar
  • 163
2 votes
1 answer
96 views

How to sample the tuples during the initial time steps of the DDPG algorithm?

I am facing an issue in understanding the following line from the pseudocode of the DDPG algorithm Sample a random minibatch of $N$ transitions $(s_i, a_i, r_i, s_{i+1})$ from $R$ Here $N$ is a ...
hanugm's user avatar
  • 3,612
0 votes
1 answer
187 views

Is it possible to add states to the Q-table after the game has started?

I would like to implement Q-learning in a game. Here is the board: It's a 2 player game. At each turn, each player can put a pawn on a line of their choice. They can't choose the column. The right ...
Dunno's user avatar
  • 103
0 votes
0 answers
130 views

Implement 4D convolution as matrix-matrix multiplication - paper is confusing!

I am confused by this paper https://arxiv.org/pdf/1410.0759.pdf which displays on page 4 how to model a 3D convolution (input has more than 1 channel and filter has more than one output). In this ...
binaryBigInt's user avatar
0 votes
1 answer
215 views

Are there any guidelines on how to map the state space to integers in the case tabular RL algorithms?

Let's say that you want to solve a problem with a tabular reinforcement learning algorithm, for example, Q-learning. You can represent the value function $Q(s, a)$ as a $|\mathcal{S}|\times |\mathcal{...
nbro's user avatar
  • 39.5k
0 votes
1 answer
73 views

What does it mean by "lazy mean" here?

Consider the following paragraph, taken from 3.4: Named Tensors of the textbook named Deep Learning with PyTorch by Eli Stevens et al., regarding the calculation of the mean for RGB channels of an RGB ...
hanugm's user avatar
  • 3,612
2 votes
1 answer
540 views

How to code an $\epsilon$-soft policy for on-policy Monte Carlo control?

I was trying to code the on-policy Monte Carlo control method. The initial policy chosen needs to be an $\epsilon$-soft policy. Can someone tell me how to code an $\epsilon$-soft policy? I know how to ...
A Q's user avatar
  • 23
3 votes
1 answer
2k views

How do graph neural networks adapt to different number of nodes and connections of different graphs?

I have recently been studying GNN, and the fundamental idea seems to be the aggregation and transfer of information from a node's neighborhood to update the node's internal state. However, there are ...
tangolin's user avatar
  • 153
3 votes
0 answers
46 views

Are there standardized forms of the Turing Test?

Most computer science instructors will tell you that the Turing Test is more a theoretical or conceptual thought experiment than an actual exam that someone (or something!) can formally sit and ...
Robert Columbia's user avatar
1 vote
0 answers
85 views

Different equations for Yolov3 in courses/ articles and Darknet GitHub code?

I am confused by the equations for bounding boxes I find online. Some articles say that box_width = anchor_width * exp(residual_value_of_box_width)) and the ...
shikhar97gupta's user avatar
0 votes
0 answers
44 views

Why isn't my perceptron having the expected costs?

I want to implement a single perceptron for linear regression using the following formulas: The input data for the first case is one column (x(392, 1); y(392, 1)) ...
Rim Sleimi's user avatar
1 vote
2 answers
438 views

In mini-batch gradient descent, do we pass each input in the batch individually or all inputs at the same time through the layer?

In the stochastic gradient descent algorithm, the weight update happens for every training sample. In the mini-batch gradient descent algorithm, the weight update happens for every batch of training ...
hanugm's user avatar
  • 3,612
0 votes
1 answer
280 views

Why using negative integers (as dimensions?) in tensor shapes rather than natural numbers?

Consider the following paragraph from A.1 MULTI-MNIST AND CLEVR of A IMPLEMENTATION DETAILS from the research paper titled ...
hanugm's user avatar
  • 3,612
0 votes
1 answer
137 views

Does an increase in the number of epochs lead to complete breakdown?

Recently, I ran a code on my system that involves deep neural networks. The number of epochs provided by the designers are 301. I tried to increase the number of epochs to 501. To my shock, the model ...
hanugm's user avatar
  • 3,612
0 votes
1 answer
310 views

Which of the following two implementations of a Least Squares classifier in Python is correct?

I am trying to solve a classification problem by implementing the Least Squares algorithm in Python. To solve this problem, I am implementing the linear algebra formula to train the classifier, which ...
User9123's user avatar
1 vote
1 answer
41 views

What is the high-level algorithm followed by contemporary packages for the calculation of gradient?

Most of the neural network models in contemporary deep learning packages are trained based on gradients. Let $f: \mathbb{R}^m \rightarrow \mathbb{R}^n$ be a function for which we want to find a ...
hanugm's user avatar
  • 3,612
1 vote
2 answers
28 views

Why not undefined expression is different from numerical underflow?

Consider an architecture or programming language that uses $n$ bits for storing a floating point number in a particular format. Then each and every floating point number it can store should be in a ...
hanugm's user avatar
  • 3,612
2 votes
1 answer
85 views

How is this statement from a TensorFlow implementation of a certain KL-divergence formula related to the corresponding formula?

I am trying to understand a certain KL-divergence formula (which can be found on page 6 of the paper Evidential Deep Learning to Quantify Classification Uncertainty) and found a TensorFlow ...
Baka's user avatar
  • 33
0 votes
1 answer
88 views

What are the (key) purposes of unsqueezing operation on tensors?

The unsqeeze operation is used in several deep learning algorithms. However, I only found this operation in the code/implementation of the algorithms presented in the papers, which do not mention it. ...
hanugm's user avatar
  • 3,612
0 votes
1 answer
723 views

How to calculate the gradient penalty proposed in "Improved Training of Wasserstein GANs"?

The research paper titled Improved Training of Wasserstein GANs proposed a gradient penalty in order to avoid undesired behavior due to weight clipping of the discriminator. We now propose an ...
hanugm's user avatar
  • 3,612
0 votes
1 answer
2k views

Why people always say the Transformer is parallelizable while the self-attention layer still depends on outputs of all time steps to calculate?

When compared to an RNN seq-to-seq model, people always say the Transformer is parallelizable. In the original Attention Is All You Need paper, it also said that Recurrent models typically factor ...
palazzo train's user avatar
1 vote
0 answers
537 views

Is there any gain by lazy initialization of weights, biases and number of input channels for a convolution operation?

The basic layers for performing convolution operations 1,2,3 in PyTorch are ...
hanugm's user avatar
  • 3,612
3 votes
1 answer
1k views

What exactly is an XPU?

I know about CPU, GPU and TPU. But, it is the first time for me to read about XPU from PyTorch documentation about MODULE. ...
hanugm's user avatar
  • 3,612
0 votes
2 answers
92 views

What are the applications in which the precision of the neural network's weights is unimportant?

While reading about Module in PyTorch, I came across a new data type called half datatype. half() method when calls on a Module ...
hanugm's user avatar
  • 3,612
1 vote
0 answers
23 views

Is it possible to use (infinite cardinal) random variables during implementation?

Random variables can be broadly classified into three types: random variables whose range is finite, random variable whose range is countably infinite and random variables whose range is uncountable. ...
hanugm's user avatar
  • 3,612
2 votes
0 answers
42 views

How to implement a (3 + 2)-dimensional convolutional layer where the 2d space is "internal"?

I am trying to train a CNN to learn 5D (kind of) data. The data is structured as follows. It has three spatial dimensions [x, y, z], but it also has two "...
play's user avatar
  • 23
2 votes
1 answer
56 views

In practice, are perceptrons typically implemented as objects?

I'm fairly new to ANNs. I know the general structure, the math, and the algorithms behind them. I figured the logical next step on my journey to fully understanding them would to be implement one ...
lumpychum's user avatar
1 vote
1 answer
431 views

Why is the logarithm of the standard deviation used in this implementation of proximal policy optimization?

I am currently writing my bachelor thesis, which is an implementation of proximal policy optimization. Sometimes, I hit a wall because of the gaps in my mathematical knowledge. However, implementing ...
R.Sar's user avatar
  • 13
1 vote
0 answers
193 views

Off-policy Bellman Operators: Writing Operator and Weight Update Function for a 2-State System

I am studying for RL on my own and was trying to solve this question I came across. Write an operator function $T(w, \pi, \mu, l, g)$ that takes weights $w$, a target policy $\pi$, a behaviour policy ...
Kamran Janjua's user avatar
0 votes
1 answer
423 views

Why does my neural network to solve the XOR problem always output 0.5?

I'm trying to create a neural network to simulate an XOR gate. Here's my dataset: ...
user45708's user avatar
1 vote
3 answers
156 views

How can I model any structure for a neural network?

Hello I am currently doing research on the effect of altering a neural network's structure. Particularly I am investigating what affect would putting a random DAG (directed acyclic graph) in the ...
Rami Hoteit's user avatar
4 votes
2 answers
598 views

How is the AI in 3d games implemented?

A few days ago, I started looking a bit more into AI and learning about the way it works, and it is very interesting, but I can't find a clear answer on how the artificial intelligence is implemented ...
Irkl1_'s user avatar
  • 43
0 votes
1 answer
738 views

What to do with a GAN that trained well but got worse over time?

I am training a WGAN-GP network based on the following paper, though I am using a different dataset. Now, for the first ~ 60-70 epochs, my network trained really well, which I could see in the loss ...
Anonymous5638's user avatar
7 votes
2 answers
2k views

How is this Pytorch expression equivalent to the KL divergence?

I found the following PyTorch code (from this link) -0.5 * torch.sum(1 + sigma - mu.pow(2) - sigma.exp()) where mu is the mean ...
user8714896's user avatar
1 vote
1 answer
378 views

Is there a convention on the order of multiplication of the weights with the inputs in neural nets?

Is there a convention on how the input data and the weights are multiplied? The input data can be anything, including the result from the previous layers. There are two options: Option 1: $$\begin{...
Pluxyy's user avatar
  • 85
4 votes
1 answer
187 views

How should I implement the state transition when it is a Gaussian distribution?

I am reading this paper Anxiety, Avoidance and Sequential Evaluation and is confused about the implementation of a specific lab study. Namely, the authors model what is called the Balloon task using a ...
dezdichado's user avatar
1 vote
0 answers
268 views

Can someone explain to me this implementation of Tile Coding using Hash Tables?

The code below is adapted from this implementation. ...
Bazman's user avatar
  • 111
1 vote
1 answer
256 views

How do you pass the image from one convolutional layer to another in a CNN?

I am currently trying to write a CNN from scratch, but I don't understand how to feed the information from a max-pooling layer to the next convolutional layer. Specifically, I don't know what to do ...
Sean Mabli's user avatar
5 votes
1 answer
1k views

Wasserstein GAN: Implemention of Critic Loss Correct?

The WGAN paper concretely proposes Algorithm 1 (cf. page 8). Now, they also state what their loss for the critic and the generator is. When implementing the critic loss (so lines 5 and 6 of Algorithm ...
Anonymous5638's user avatar
1 vote
0 answers
19 views

Would it make sense to share the layers (except the last one) of the neural networks in Double DQN?

Context: Double Q-learning was introduced to prevent the maximization bias from q-learning. Instead of learning a single Q-network, we can learn two (or in general $K > 1$) and our Q-estimate would ...
kaiwenw's user avatar
  • 151
6 votes
1 answer
3k views

How are continuous actions sampled (or generated) from the policy network in PPO?

I am trying to understand and reproduce the Proximal Policy Optimization (PPO) algorithm in detail. One thing that I find missing in the paper introducing the algorithm is how exactly actions $a_t$ ...
Daniel B.'s user avatar
  • 815
1 vote
1 answer
422 views

Is average pooling equivalent to a strided convolution with a specific constant kernel?

It seems to me that average pooling can be replaced by a strided convolution with a constant kernel. For instance, a 3x3 pooling would be equivalent to a strided convolution (of stride $3$) with a $3 \...
1110101001's user avatar
2 votes
1 answer
1k views

How does the implementation of the VAE's objective function equate to ELBO?

For a lot of VAE implementations I've seen in code, it's not really obvious to me how it equates to ELBO. $$L(X)=H(Q)-H(Q:P(X,Z))=\sum_ZQ(Z)logP(Z,X)-\sum_ZQ(Z)log(Q(Z))$$ The above is the definition ...
user8714896's user avatar
1 vote
0 answers
62 views

How am I supposed to code equation 4.57 from the book "Machine Learning: An Algorithmic Perspective"?

Consider the equation 4.57 (p. 108) from section 4.6 of the Book Machine Learning: An Algorithmic Perspective, where the derivative of the softmax function is explained $$\delta_o(\kappa) = (y_\kappa -...
NewToCoding's user avatar
1 vote
0 answers
39 views

What is the difference between step_model and train_model in the OpenAI implementation of the A2C algorithm?

I'm struggling a little with understanding the OpenAI implementation of A2C in the baselines (version 2.9.0) package. From my understanding, one ...
sjokkopudd's user avatar
3 votes
2 answers
2k views

What are some alternatives to "Papers with Code"?

There are lots of research papers available that are worth reading. We can read papers easily, but the associated code (not necessarily the official one developed by the authors of the paper) is often ...
Posi2's user avatar
  • 358
1 vote
1 answer
589 views

What are the variables that need to be saved and loaded, so that a DQN model starts where it left off?

TensorFlow allows users to save the weights and the model architecture, however, that will be insufficient unless the values of certain other variables are also stored. For instance, in DQN, if $\...
desert_ranger's user avatar