Questions tagged [implementation]

For questions about implementation of software/algorithms related to Artificial Intelligence.

34 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5 votes
0 answers
503 views

Wasserstein GAN: Implemention of Critic Loss Correct?

The WGAN paper concretely proposes Algorithm 1 (cf. page 8). Now, they also state what their loss for the critic and the generator is. When implementing the critic loss (so lines 5 and 6 of Algorithm ...
user avatar
5 votes
0 answers
459 views

How is the rollout from the MCTS implemented in both of the AlphaGo Zero and the AlphaZero algorithms?

In the vanilla Monte Carlo tree search (MCTS) implementation, the rollout is usually implemented following a uniform random policy, that is, it takes random actions until the game is finished and only ...
user avatar
4 votes
1 answer
2k views

How to deal with different actions for different states of the environment?

I'm new to this AI/Machine Learning and was playing around with OpenAI Gym a bit. When looking through the environments, I came across the Blackjack-v0 environment, ...
user avatar
3 votes
1 answer
65 views

How can I create an embedding layer to convert words to a vector space from scratch?

For an upcoming project, I am trying to build a neural network for classifying text from scratch, without the use of libraries. This requires an embedding layer, or a way to convert words to some ...
user avatar
3 votes
0 answers
1k views

How much time does it take to train DQN on Atari environment?

I am trying to build a DQN model for the Atari Pong game, but I am not sure whether the model is learning at all. I am using the architecture described in the paper Playing Atari with Deep ...
user avatar
  • 151
3 votes
1 answer
147 views

How difficult is this sound classification?

I want a microphone to pick up sounds around me (let's say beyond a 3 foot radius), but ignore sounds made at my desk, such as the rustling of paper, clicking a mouse and typing, my hands brushing up ...
user avatar
  • 31
3 votes
0 answers
414 views

Understanding log probabilities of actions in the PPO objective

I'm trying to implement the Proximal Policy Optimization (PPO) algorithm (code here), but I am confused about certain concepts. What is the correct way to implement log probability of a policy (...
user avatar
2 votes
0 answers
25 views

Are there standardized forms of the Turing Test?

Most computer science instructors will tell you that the Turing Test is more a theoretical or conceptual thought experiment than an actual exam that someone (or something!) can formally sit and ...
user avatar
2 votes
0 answers
39 views

How to implement a (3 + 2)-dimensional convolutional layer where the 2d space is "internal"?

I am trying to train a CNN to learn 5D (kind of) data. The data is structured as follows. It has three spatial dimensions [x, y, z], but it also has two "...
user avatar
  • 21
2 votes
0 answers
54 views

How can I get to a final output of shape $224 \times 224$, without FC layers, from a tensor of specific shape, in OpenPose?

I am approaching the implementation of the OpenPose algorithm for realtime human body pose estimation. According to the official paper OpenPose: Realtime Multi-Person 2D Pose Estimation using Part ...
user avatar
  • 51
2 votes
0 answers
64 views

What is the correct dimension of mu/logvar and z in the VAE?

I'm having a problem to understand the needed dimensions of a VAE, especially for mu, logvar and ...
user avatar
  • 21
2 votes
0 answers
334 views

Are there any general tips for troubleshooting a VAE when apparently it is not learning?

I am trying to train a VAE for anomaly detection. I chose one architecture from this Github repository and I adjusted the input and output to match what I need. In my case, the input (and hence the ...
user avatar
  • 21
1 vote
0 answers
50 views

Different equations for Yolov3 in courses/ articles and Darknet GitHub code?

I am confused by the equations for bounding boxes I find online. Some articles say that box_width = anchor_width * exp(residual_value_of_box_width)) and the ...
user avatar
1 vote
0 answers
167 views

Is there any gain by lazy initialization of weights, biases and number of input channels for a convolution operation?

The basic layers for performing convolution operations 1,2,3 in PyTorch are ...
user avatar
  • 3,099
1 vote
0 answers
16 views

Is it possible to use (infinite cardinal) random variables during implementation?

Random variables can be broadly classified into three types: random variables whose range is finite, random variable whose range is countably infinite and random variables whose range is uncountable. ...
user avatar
  • 3,099
1 vote
0 answers
113 views

Off-policy Bellman Operators: Writing Operator and Weight Update Function for a 2-State System

I am studying for RL on my own and was trying to solve this question I came across. Write an operator function $T(w, \pi, \mu, l, g)$ that takes weights $w$, a target policy $\pi$, a behaviour policy ...
user avatar
1 vote
3 answers
92 views

How can I model any structure for a neural network?

Hello I am currently doing research on the effect of altering a neural network's structure. Particularly I am investigating what affect would putting a random DAG (directed acyclic graph) in the ...
user avatar
1 vote
0 answers
146 views

Can someone explain to me this implementation of Tile Coding using Hash Tables?

The code below is adapted from this implementation. ...
user avatar
  • 111
1 vote
0 answers
17 views

Would it make sense to share the layers (except the last one) of the neural networks in Double DQN?

Context: Double Q-learning was introduced to prevent the maximization bias from q-learning. Instead of learning a single Q-network, we can learn two (or in general $K > 1$) and our Q-estimate would ...
user avatar
  • 131
1 vote
0 answers
53 views

How am I supposed to code equation 4.57 from the book "Machine Learning: An Algorithmic Perspective"?

Consider the equation 4.57 (p. 108) from section 4.6 of the Book Machine Learning: An Algorithmic Perspective, where the derivative of the softmax function is explained $$\delta_o(\kappa) = (y_\kappa -...
user avatar
1 vote
0 answers
33 views

What is the difference between step_model and train_model in the OpenAI implementation of the A2C algorithm?

I'm struggling a little with understanding the OpenAI implementation of A2C in the baselines (version 2.9.0) package. From my understanding, one ...
user avatar
1 vote
0 answers
24 views

What is the score used to visualize attention in this paper?

I'm reading this paper Global-Locally Self-Attentive Dialogue State Tracker and follow through the implementation published in GLAD. I was wondering if someone can clarify what variable or score is ...
user avatar
  • 111
1 vote
0 answers
69 views

Why are the current means and the old ones the same in this implementation of Elastic Weight Consolidation?

I'm trying to re-implement Elastic Weight Consolidation (EWC) as outlined in this paper. As a reference, I am also using this Github repository (another implementation). My model/idea is pretty ...
user avatar
  • 111
0 votes
0 answers
3 views

How to Extract Specific Information From Web Pages Containing Job Postings?

I am currently working on my first research project, which is to implement machine learning based web scraping in the e-recruitement environment. Currently, I am building the dataset that needs to be ...
user avatar
0 votes
0 answers
24 views

Implement 4D convolution as matrix-matrix multiplication - paper is confusing!

I am confused by this paper https://arxiv.org/pdf/1410.0759.pdf which displays on page 4 how to model a 3D convolution (input has more than 1 channel and filter has more than one output). In this ...
user avatar
0 votes
0 answers
123 views

Why is my Advantage Actor Critic agent learning and then forgetting (i.e. the loss drops to zero)?

I am writing my own version of the A2C algorithm in PyTorch, and for the most part my algorithm is learning the simple environments (like CartPole-v1). Unfortunately during the training process, the ...
user avatar
  • 46
0 votes
0 answers
22 views

Which kind of RNNs are mostly used in applications: hidden-hidden or actual output-hidden?

I came across two types of RNN while reading the chapter titled Sequence Modeling: Recurrent and Recursive Nets of the textbook named Deep Learning by Ian Goodfellow et al. First type: Recurrent ...
user avatar
  • 3,099
0 votes
0 answers
31 views

Why isn't my perceptron having the expected costs?

I want to implement a single perceptron for linear regression using the following formulas: The input data for the first case is one column (x(392, 1); y(392, 1)) ...
user avatar
0 votes
0 answers
39 views

Is this a good implementation of this LSTM architecture?

I had been looking at some OCR problems and came across this presentation. I implemented it. In the presentation, there is the LSTM-Stack (diagram and algorithm, slide 32): Here is a visualization of ...
user avatar
0 votes
1 answer
51 views

Why using negative integers (as dimensions?) in tensor shapes rather than natural numbers?

Consider the following paragraph from A.1 MULTI-MNIST AND CLEVR of A IMPLEMENTATION DETAILS from the research paper titled ...
user avatar
  • 3,099
0 votes
1 answer
148 views

Which of the following two implementations of a Least Squares classifier in Python is correct?

I am trying to solve a classification problem by implementing the Least Squares algorithm in Python. To solve this problem, I am implementing the linear algebra formula to train the classifier, which ...
user avatar
0 votes
0 answers
129 views

DDPG or PPO don't work well with my custom non gym environment

I have a project to control a robot with right and left wheel speeds, and my step time is not constant. Because my outputs are continuous (right wheel speed, left wheel speed, and time step), I try to ...
user avatar
  • 3
0 votes
1 answer
69 views

What are the applications in which the precision of the neural network's weights is unimportant?

While reading about Module in PyTorch, I came across a new data type called half datatype. half() method when calls on a Module ...
user avatar
  • 3,099
0 votes
0 answers
115 views

Using AI to guess a mathematical pattern of certain polynomials in four variables: practical challenge

I'd like to use machine learning to guess a mathematical pattern: the input are certain polynomials in four variables $q_1,q_2,q_3,q_4$, the output can be zero or one. Allowed polynomials are such ...
user avatar
  • 101