Newest 'implementation' Questions

0

votes

1answer

18 views

How to calculate the gradient penalty?

The research paper titled Improved Training of Wasserstein GANs proposed gradient penalty in-order to avoid undesired behavior due to weight clipping of discriminator. We now propose an alternative ...

asked yesterday

hanugm

1,3294 silver badges17 bronze badges

0

votes

0answers

9 views

Why people always say the Transformer is parallelizable while the self-attention layer still depends on outputs of all time steps to calculate?

When compared to an RNN seq-to-seq model, people always say the Transformer is parallelizable. In the original Attention Is All You Need paper, it also said that Recurrent models typically factor ...

transformer attention implementation

asked 2 days ago

palazzo train

1011 bronze badge

1

vote

0answers

11 views

Is there any gain by lazy initialization of weights, biases and number of input channels for a convolution operation?

The basic layers for performing convolution operation in PyTorch are ...

convolutional-neural-networks pytorch implementation

asked Jul 22 at 3:58

hanugm

1,3294 silver badges17 bronze badges

1

vote

0answers

14 views

Is it possible to use (infinite cardinal) random variables during implementation?

Random variables can be classified into three types: random variables whose range is finite, random variable whose range is countably infinite and random variables whose range is uncountable. Random ...

implementation random-variable

asked Jul 11 at 23:55

hanugm

1,3294 silver badges17 bronze badges

2

votes

0answers

37 views

How to implement a (3 + 2)-dimensional convolutional layer where the 2d space is “internal”?

I am trying to train a CNN to learn 5D (kind of) data. The data is structured as follows. It has three spatial dimensions [x, y, z], but it also has two "...

convolutional-neural-networks implementation convolution convolutional-layers residual-networks

asked May 26 at 2:01

play

212 bronze badges

1

vote

1answer

46 views

In practice, are perceptrons typically implemented as objects?

I'm fairly new to ANNs. I know the general structure, the math, and the algorithms behind them. I figured the logical next step on my journey to fully understanding them would to be implement one ...

neural-networks implementation artificial-neuron perceptron

asked May 10 at 20:45

lumpychum

132 bronze badges

1

vote

1answer

46 views

Why is the logarithm of the standard deviation used in this implementation of proximal policy optimization?

I am currently writing my bachelor thesis, which is an implementation of proximal policy optimization. Sometimes, I hit a wall because of the gaps in my mathematical knowledge. However, implementing ...

reinforcement-learning implementation proximal-policy-optimization normal-distribution

asked Apr 11 at 7:52

R.Sar

132 bronze badges

1

vote

3answers

65 views

How can I model any structure for a neural network?

Hello I am currently doing research on the effect of altering a neural network's structure. Particularly I am investigating what affect would putting a random DAG (directed acyclic graph) in the ...

neural-networks tensorflow keras pytorch implementation

asked Mar 17 at 13:03

Rami Hoteit

111 bronze badge

4

votes

2answers

119 views

How is the AI in 3d games implemented?

A few days ago, I started looking a bit more into AI and learning about the way it works, and it is very interesting, but I can't find a clear answer on how the artificial intelligence is implemented ...

reinforcement-learning game-ai implementation gaming algorithm-request

asked Mar 15 at 21:04

Irkl1_

433 bronze badges

0

votes

1answer

39 views

What to do with a GAN that trained well but got worse over time?

I am training a WGAN-GP network based on the following paper, though I am using a different dataset. Now, for the first ~ 60-70 epochs, my network trained really well, which I could see in the loss ...

papers generative-adversarial-networks hyperparameter-optimization implementation wasserstein-gan

asked Mar 15 at 18:54

Anonymous5638

1714 bronze badges

1

vote

1answer

107 views

Is there a convention on the order of multiplication of the weights with the inputs in neural nets?

Is there a convention on how the input data and the weights are multiplied? The input data can be anything, including the result from the previous layers. There are two options: Option 1: $$\begin{...

neural-networks implementation weights

asked Feb 7 at 10:19

Pluxyy

653 bronze badges

4

votes

1answer

63 views

How should I implement the state transition when it is a Gaussian distribution?

I am reading this paper Anxiety, Avoidance and Sequential Evaluation and is confused about the implementation of a specific lab study. Namely, the authors model what is called the Balloon task using a ...

reinforcement-learning markov-decision-process implementation temporal-difference-methods transition-model

asked Jan 25 at 22:53

dezdichado

1834 bronze badges

0

votes

0answers

25 views

Is the dimensionality of the query and key tensors in this implementation of the transformer's self-attention correct?

Looking at this tutorial matmul_qk = tf.matmul(q, k, transpose_b=True) # (..., seq_len_q, seq_len_k) is the output of two tensors multiplied of shape: ...

neural-networks tensorflow transformer implementation attention

asked Jan 22 at 3:03

Jacob B

1754 bronze badges

1

vote

0answers

73 views

Can someone explain to me this implementation of Tile Coding using Hash Tables?

The code below is adapted from this implementation. ...

reinforcement-learning python implementation function-approximation tile-coding

asked Jan 20 at 17:28

Bazman

1112 bronze badges

1

vote

1answer

54 views

How do you pass the image from one convolutional layer to another in a CNN?

I am currently trying to write a CNN from scratch, but I don't understand how to feed the information from a max-pooling layer to the next convolutional layer. Specifically, I don't know what to do ...

convolutional-neural-networks implementation convolutional-layers pooling-layers

asked Jan 10 at 0:54

Sean Mabli

112 bronze badges

5

votes

0answers

138 views

Wasserstein GAN: Implemention of Critic Loss Correct?

The WGAN paper concretely proposes Algorithm 1 (cf. page 8). Now, they also state what their loss for the critic and the generator is. When implementing the critic loss (so lines 5 and 6 of Algorithm ...

papers generative-adversarial-networks implementation wasserstein-gan

asked Dec 27 '20 at 21:34

Anonymous5638

1714 bronze badges

1

vote

0answers

17 views

Would it make sense to share the layers (except the last one) of the neural networks in Double DQN?

Context: Double Q-learning was introduced to prevent the maximization bias from q-learning. Instead of learning a single Q-network, we can learn two (or in general $K > 1$) and our Q-estimate would ...

reinforcement-learning dqn implementation double-dqn

asked Dec 12 '20 at 6:05

kaiwenw

1213 bronze badges

2

votes

1answer

130 views

How are continuous actions sampled (or generated) from the policy network in PPO?

I am trying to understand and reproduce the Proximal Policy Optimization (PPO) algorithm in detail. One thing that I find missing in the paper introducing the algorithm is how exactly actions $a_t$ ...

reinforcement-learning implementation proximal-policy-optimization continuous-action-spaces

asked Dec 12 '20 at 1:42

Daniel B.

6571 gold badge2 silver badges11 bronze badges

1

vote

1answer

32 views

Is average pooling equivalent to a strided convolution with a specific constant kernel?

It seems to me that average pooling can be replaced by a strided convolution with a constant kernel. For instance, a 3x3 pooling would be equivalent to a strided convolution (of stride $3$) with a $3 \...

neural-networks convolutional-neural-networks implementation convolution pooling

asked Nov 14 '20 at 5:57

1110101001

1133 bronze badges

1

vote

0answers

43 views

How am I supposed to code equation 4.57 from the book “Machine Learning: An Algorithmic Perspective”?

Consider the equation 4.57 (p. 108) from section 4.6 of the Book Machine Learning: An Algorithmic Perspective, where the derivative of the softmax function is explained $$\delta_o(\kappa) = (y_\kappa -...

machine-learning python backpropagation implementation softmax

asked Nov 3 '20 at 9:01

NewToCoding

112 bronze badges

1

vote

0answers

31 views

What is the difference between step_model and train_model in the OpenAI implementation of the A2C algorithm?

I'm struggling a little with understanding the OpenAI implementation of A2C in the baselines (version 2.9.0) package. From my understanding, one ...

reinforcement-learning open-ai implementation advantage-actor-critic

asked Oct 24 '20 at 9:47

sjokkopudd

111 bronze badge

2

votes

2answers

419 views

What are some alternatives to “Papers with Code”?

There are lots of research papers available that are worth reading. We can read papers easily, but the associated code (not necessarily the official one developed by the authors of the paper) is often ...

papers resource-request implementation

asked Oct 8 '20 at 5:14

Posi2

3382 silver badges15 bronze badges

1

vote

1answer

94 views

What are the variables that need to be saved and loaded, so that a DQN model starts where it left off?

TensorFlow allows users to save the weights and the model architecture, however, that will be insufficient unless the values of certain other variables are also stored. For instance, in DQN, if $\...

reinforcement-learning dqn deep-rl implementation

asked Aug 23 '20 at 18:27

desert_ranger

2571 silver badge9 bronze badges

3

votes

1answer

49 views

How can I create an embedding layer to convert words to a vector space from scratch?

For an upcoming project, I am trying to build a neural network for classifying text from scratch, without the use of libraries. This requires an embedding layer, or a way to convert words to some ...

neural-networks implementation word-embedding

asked Jul 9 '20 at 15:04

JumpJehovah

433 bronze badges

1

vote

0answers

24 views

What is the score used to visualize attention in this paper?

I'm reading this paper Global-Locally Self-Attentive Dialogue State Tracker and follow through the implementation published in GLAD. I was wondering if someone can clarify what variable or score is ...

natural-language-processing papers implementation dialogue-systems

asked Jul 7 '20 at 9:22

greedsin

1111 bronze badge

1

vote

1answer

250 views

What is a RAM state in the gym's breakout-ram environment?

I have encountered the gym environment and decided to create AI that plays breakout. Here is the link: https://gym.openai.com/envs/Breakout-ram-v0/. The documentation says that the state is ...

reinforcement-learning deep-rl implementation gym

asked Jun 1 '20 at 9:23

dato nefaridze

6421 silver badge9 bronze badges

1

vote

1answer

67 views

Is there a general file type associated with AI projects?

This is a general question. Is there a general file type associated with AI projects? Photoshop = .psd Excel = csv Artificial Intelligence = ?

open-ai implementation

asked May 26 '20 at 5:12

user8273608

111 bronze badge

3

votes

1answer

281 views

What made your DDPG implementation on your environment work?

I am working on scheduling problem that has inherent randomness. The dimensions of action and state spaces are 1 and 5 respectively. I am using DDPG, but it seems extremely unstable, and so far it ...

reinforcement-learning implementation ddpg hyper-parameters

asked May 23 '20 at 4:39

Schach21

2321 silver badge6 bronze badges

1

vote

1answer

50 views

Is there a good and easy paper to code policy gradient algorithms (REINFORCE) from scratch?

I am interested in learning about policy gradient algorithms and REINFORCE. Can you suggest a good and easy paper that I can use to code them from scratch?

reinforcement-learning reference-request policy-gradients implementation reinforce

asked Apr 19 '20 at 1:34

jgauth

1419 bronze badges

2

votes

1answer

51 views

How can I implement policy evaluation when reward is tied to an action outcome?

I'm following Stanford reinforcement learning videos on youtube. One of the assignments asks to write code for policy evaluation for Gym's FrozenLake-v0 environment. In the course (and books I have ...

reinforcement-learning math implementation gym policy-evaluation

asked Apr 13 '20 at 13:01

Argod

232 bronze badges

3

votes

3answers

264 views

Why can the Bellman equation be turned into an update rule?

In chapter 4.1 of Sutton's book, the Bellman equation is turned into an update rule by simply changing the indices of it. How is it mathematically justified? I didn't quite get the initiation of why ...

reinforcement-learning implementation convergence bellman-equations policy-evaluation

asked Apr 10 '20 at 22:07

Saeid Ghafouri

1334 bronze badges

3

votes

0answers

698 views

How much time does it take to train DQN on Atari environment?

I am trying to build a DQN model for the Atari Pong game, but I am not sure whether the model is learning at all. I am using the architecture described in the paper Playing Atari with Deep ...

reinforcement-learning dqn deep-rl implementation computational-complexity

asked Apr 1 '20 at 9:23

Ach113

1712 bronze badges

2

votes

0answers

42 views

How can I get to a final output of shape $224 \times 224$, without FC layers, from a tensor of specific shape, in OpenPose?

I am approaching the implementation of the OpenPose algorithm for realtime human body pose estimation. According to the official paper OpenPose: Realtime Multi-Person 2D Pose Estimation using Part ...

neural-networks deep-learning papers implementation pose-estimation

asked Mar 21 '20 at 10:55

Chal.lo

411 bronze badge

1

vote

1answer

104 views

Why isn't the implementation of my policy evaluation for a simple MDP converging?

I am trying to code out a policy evaluation algorithm to find the $V^\pi(s)$ for all states. The following diagram below shows the MDP. In this case i let p = q = 0.5. the rewards for each states ...

reinforcement-learning markov-decision-process implementation convergence policy-evaluation

asked Mar 14 '20 at 2:41

calveeen

1,1814 silver badges14 bronze badges

2

votes

0answers

476 views

How can I implement derivative of softmax function for matrices in Python? [closed]

I have trouble understanding how to implement derivative of softmax function. Here is what I tried: ...

deep-learning python backpropagation implementation derivative

asked Jan 25 '20 at 9:30

Ach113

1712 bronze badges

3

votes

1answer

106 views

How difficult is this sound classification?

I want a microphone to pick up sounds around me (let's say beyond a 3 foot radius), but ignore sounds made at my desk, such as the rustling of paper, clicking a mouse and typing, my hands brushing up ...

classification training implementation

asked Dec 31 '19 at 16:57

donut

311 bronze badge

1

vote

0answers

60 views

Why are the current means and the old ones the same in this implementation of Elastic Weight Consolidation?

I'm trying to re-implement Elastic Weight Consolidation (EWC) as outlined in this paper. As a reference, I am also using this Github repository (another implementation). My model/idea is pretty ...

neural-networks pytorch implementation incremental-learning elastic-weight-consolidation

asked Dec 21 '19 at 18:56

Martin

1113 bronze badges

2

votes

1answer

70 views

From an implementation point of view, what are the main differences between an RNN and a CNN?

I understand that in general an RNN is good for time series data and a CNN image data, and have noticed many blogs explaining the fundamental differences in the models. As a beginner in machine ...

deep-learning convolutional-neural-networks comparison recurrent-neural-networks implementation

asked Nov 27 '19 at 23:08

nilsinelabore

2312 silver badges10 bronze badges

2

votes

0answers

221 views

Are there any general tips for troubleshooting a VAE when apparently it is not learning?

I am trying to train a VAE for anomaly detection. I chose one architecture from this Github repository and I adjusted the input and output to match what I need. In my case, the input (and hence the ...

generative-model autoencoders implementation variational-autoencoder

asked Nov 15 '19 at 3:08

Bill

211 bronze badge

5

votes

1answer

176 views

Context-based gap-fill face posture-mapper GAN

These images are handmade, not auto-generated like they will be in production. Apologies for inaccuracies in the graph overlay. I am trying to build an AI like that displayed in the diagram: when ...

ai-design generative-adversarial-networks generative-model implementation image-generation

asked Nov 13 '19 at 19:35

Geza Kerecsenyi

516 bronze badges

13

votes

4answers

8k views

What does AI software look like, and how is it different from other software?

What does AI software look like? What is the major difference between AI software and other software?

comparison implementation

asked Nov 12 '19 at 6:17

Harold Ed

2533 silver badges9 bronze badges

3

votes

0answers

132 views

How can I increase the speed and performance of my implementation of an AI for Reversi?

I made an AI for Reversi, aka Othello (8×8), like Alpha Zero, using this book. This book is written in Japanese. The source code of the AI I implemented can be found in this Github repository. There ...

tensorflow game-ai keras implementation

asked Nov 11 '19 at 10:46

TKTK-ST

312 bronze badges

3

votes

0answers

236 views

How is the rollout from the MCTS implemented in both of the AlphaGo Zero and the AlphaZero algorithms?

In a vanilla Monte Carlo tree search (MCTS) implementation, the rollout is usually implemented following a uniform random policy, that is, it takes random actions until the game is finished and only ...

monte-carlo-tree-search alphazero implementation alphago-zero

asked Nov 3 '19 at 0:40

ihavenoidea

2151 silver badge8 bronze badges

4

votes

1answer

169 views

How to implement SVM algorithm from scratch in a programming language?

I'm a computer scientist who's studying support vector machines (SVMs) in a machine learning course. I have some understanding of how SVMs are designed, thanks to 16. Learning: Support Vector Machines ...

machine-learning implementation support-vector-machine

asked Aug 23 '19 at 9:31

Marco

1413 bronze badges

3

votes

0answers

226 views

Understanding log probabilities of actions in the PPO objective

I'm trying to implement the Proximal Policy Optimization (PPO) algorithm (code here), but I am confused about certain concepts. What is the correct way to implement log probability of a policy (...

python objective-functions implementation proximal-policy-optimization

asked Jul 25 '19 at 14:12

Chintan Trivedi

1512 bronze badges

3

votes

2answers

123 views

Are feature maps merged or are they passed on as they are?

I am unsure about the following parts of the architecture and mechanics of convolution layers in CNNs. Possibly, this is implementation-dependent though. First question: Say I have 2 convolution ...

neural-networks deep-learning convolutional-neural-networks definitions implementation

asked Jul 19 '19 at 23:27

lo tolmencre

1975 bronze badges

3

votes

1answer

141 views

How does Monte Carlo Tree Search UCT exploitation value change based on perspective?

In this blog toward the end, the author writes the following: For the sake of my question, let’s assume that a terminal state gives a reward of +1 for a win and -1 for a loss. When the author says “...

monte-carlo-tree-search implementation

asked May 2 '19 at 1:23

Hanzy

4592 silver badges9 bronze badges

4

votes

2answers

272 views

How is parallelism implemented in RL algorithms like PPO?

There are multiple ways to implement parallelism in reinforcement learning. One is to use parallel workers running in their own environments to collect data in parallel, instead of using replay memory ...

reinforcement-learning actor-critic-methods implementation proximal-policy-optimization

asked Apr 30 '19 at 1:15

alex vdk

411 bronze badge

1

vote

1answer

673 views

How should I implement the backward pass through a flatten layer of a CNN?

I am making a NN library without any other external NN library, so I am implementing all layers, including the flatten layer, and algorithms (forward and backward pass) from scratch. I know the ...

neural-networks convolutional-neural-networks backpropagation implementation

asked Apr 4 '19 at 15:26

Clement Hui

1,7005 silver badges19 bronze badges

4

votes

1answer

1k views

How can the convolution operation be implemented as a matrix multiplication?

How can the convolution operation used by CNNs be implemented as a matrix-vector multiplication? We often think of the convolution operation in CNNs as a kernel that slides across the input. However, ...

machine-learning deep-learning convolutional-neural-networks implementation convolution

asked Mar 12 '19 at 13:57

nbro♦

28.1k6 gold badges56 silver badges122 bronze badges

Questions tagged [implementation]

Related Tags