Questions tagged [implementation]

For questions about implementation of software/algorithms related to Artificial Intelligence.

Filter by
Sorted by
Tagged with
1
vote
1answer
39 views

In practice, are perceptrons typically implemented as objects?

I'm fairly new to ANNs. I know the general structure, the math, and the algorithms behind them. I figured the logical next step on my journey to fully understanding them would to be implement one ...
2
votes
2answers
359 views

What are some alternatives to “Papers with Code”?

There are lots of research papers available that are worth reading. We can read papers easily, but the associated code (not necessarily the official one developed by the authors of the paper) is often ...
4
votes
1answer
158 views

How to implement SVM algorithm from scratch in a programming language?

I'm a computer scientist who's studying support vector machines (SVMs) in a machine learning course. I have some understanding of how SVMs are designed, thanks to 16. Learning: Support Vector Machines ...
1
vote
1answer
33 views

Why is the logarithm of the standard deviation used in this implementation of proximal policy optimization?

I am currently writing my bachelor thesis, which is an implementation of proximal policy optimization. Sometimes, I hit a wall because of the gaps in my mathematical knowledge. However, implementing ...
3
votes
1answer
47 views

How can I create an embedding layer to convert words to a vector space from scratch?

For an upcoming project, I am trying to build a neural network for classifying text from scratch, without the use of libraries. This requires an embedding layer, or a way to convert words to some ...
13
votes
4answers
7k views

What does AI software look like, and how is it different from other software?

What does AI software look like? What is the major difference between AI software and other software?
4
votes
2answers
104 views

How is the AI in 3d games implemented?

A few days ago, I started looking a bit more into AI and learning about the way it works, and it is very interesting, but I can't find a clear answer on how the artificial intelligence is implemented ...
0
votes
1answer
28 views

What to do with a GAN that trained well but got worse over time?

I am training a WGAN-GP network based on the following paper, though I am using a different dataset. Now, for the first ~ 60-70 epochs, my network trained really well, which I could see in the loss ...
4
votes
1answer
2k views

How to deal with different actions for different states of the environment?

I'm new to this AI/Machine Learning and was playing around with OpenAI Gym a bit. When looking through the environments, I came across the Blackjack-v0 environment, ...
4
votes
1answer
61 views

How should I implement the state transition when it is a Gaussian distribution?

I am reading this paper Anxiety, Avoidance and Sequential Evaluation and is confused about the implementation of a specific lab study. Namely, the authors model what is called the Balloon task using a ...
1
vote
1answer
106 views

Is there a convention on the order of multiplication of the weights with the inputs in neural nets?

Is there a convention on how the input data and the weights are multiplied? The input data can be anything, including the result from the previous layers. There are two options: Option 1: $$\begin{...
5
votes
2answers
5k views

How fast is TensorFlow compared to self written neural nets? [closed]

I made my first neural net in C++ without any libraries. It was a net to recognize numbers from the MNIST dataset. In a 784 - 784 - 10 net with sigmoid function and 5 epochs with every 60000 samples, ...
3
votes
1answer
99 views

How difficult is this sound classification?

I want a microphone to pick up sounds around me (let's say beyond a 3 foot radius), but ignore sounds made at my desk, such as the rustling of paper, clicking a mouse and typing, my hands brushing up ...
5
votes
0answers
75 views

Wasserstein GAN: Implemention of Critic Loss Correct?

The WGAN paper concretely proposes Algorithm 1 (cf. page 8). Now, they also state what their loss for the critic and the generator is. When implementing the critic loss (so lines 5 and 6 of Algorithm ...
0
votes
0answers
23 views

Is the dimensionality of the query and key tensors in this implementation of the transformer's self-attention correct?

Looking at this tutorial matmul_qk = tf.matmul(q, k, transpose_b=True) # (..., seq_len_q, seq_len_k) is the output of two tensors multiplied of shape: ...
1
vote
0answers
64 views

Can someone explain to me this implementation of Tile Coding using Hash Tables?

The code below is adapted from this implementation. ...
12
votes
2answers
7k views

How to implement an “unknown” class in multi-class classification with neural networks?

For example, I need to detect classes for MNIST data. But I want to have not 10 classes for digits, but also I want to have 11th class "not a digit", so that any letter, any other type of ...
3
votes
1answer
650 views

How can I read the .weights file that stores the weights of the pre-trained YOLO in Keras? [closed]

I would like to use the pre-trained weights of YOLO (tiny version, v1) in Keras, which are given in a file with extension .weights (here is an example). How can I ...
1
vote
1answer
53 views

How do you pass the image from one convolutional layer to another in a CNN?

I am currently trying to write a CNN from scratch, but I don't understand how to feed the information from a max-pooling layer to the next convolutional layer. Specifically, I don't know what to do ...
4
votes
2answers
96 views

How can I implement back-propagation for medium-sized neural networks?

I've been wanting to make my own Neural Network in Python, in order to better understand how it works. I've been following this series of videos as a sort of guide, but it seems the backpropagation ...
2
votes
1answer
74 views

How are continuous actions sampled (or generated) from the policy network in PPO?

I am trying to understand and reproduce the Proximal Policy Optimization (PPO) algorithm in detail. One thing that I find missing in the paper introducing the algorithm is how exactly actions $a_t$ ...
1
vote
1answer
641 views

How should I implement the backward pass through a flatten layer of a CNN?

I am making a NN library without any other external NN library, so I am implementing all layers, including the flatten layer, and algorithms (forward and backward pass) from scratch. I know the ...
1
vote
0answers
16 views

Would it make sense to share the layers (except the last one) of the neural networks in Double DQN?

Context: Double Q-learning was introduced to prevent the maximization bias from q-learning. Instead of learning a single Q-network, we can learn two (or in general $K > 1$) and our Q-estimate would ...
3
votes
3answers
224 views

Why can the Bellman equation be turned into an update rule?

In chapter 4.1 of Sutton's book, the Bellman equation is turned into an update rule by simply changing the indices of it. How is it mathematically justified? I didn't quite get the initiation of why ...
7
votes
4answers
537 views

Is it possible to implement Asimov's Three Laws of Robotics?

Would it be possible to put Asimov's three Laws of Robotics into an AI? The three laws are: A robot (or, more accurately, an AI) cannot harm a human being, or through inaction allow a human being to ...
1
vote
1answer
30 views

Is average pooling equivalent to a strided convolution with a specific constant kernel?

It seems to me that average pooling can be replaced by a strided convolution with a constant kernel. For instance, a 3x3 pooling would be equivalent to a strided convolution (of stride $3$) with a $3 \...
1
vote
2answers
585 views

What does “reimplementations of deep learning algorithms which replicate performance from the papers” mean?

In the OpenAI's Machine Learning Fellow position, it is written We look for candidates with one or more of the following credentials: ... Open-source reimplementations of deep learning algorithms ...
8
votes
1answer
2k views

What are other ways of handling invalid actions in scenarios where all rewards are either 0 (best reward) or negative?

I created an OpenAI Gym environment, and I would like to check the performance of the agent from OpenAI Baselines DQN approach on it. In my environment, the best possible outcome for the agent is 0 - ...
4
votes
1answer
760 views

How can the convolution operation be implemented as a matrix multiplication?

How can the convolution operation used by CNNs be implemented as a matrix-vector multiplication? We often think of the convolution operation in CNNs as a kernel that slides across the input. However, ...
1
vote
0answers
31 views

What is the difference between step_model and train_model in the OpenAI implementation of the A2C algorithm?

I'm struggling a little with understanding the OpenAI implementation of A2C in the baselines (version 2.9.0) package. From my understanding, one ...
1
vote
0answers
42 views

How am I supposed to code equation 4.57 from the book “Machine Learning: An Algorithmic Perspective”?

Consider the equation 4.57 (p. 108) from section 4.6 of the Book Machine Learning: An Algorithmic Perspective, where the derivative of the softmax function is explained $$\delta_o(\kappa) = (y_\kappa -...
0
votes
1answer
497 views

Why is this PyTorch implementation of the actor-critic algorithm inconsistent with the mathematical formulas?

This PyTorch implementation of the actor-critic algorithm calculates the losses like so: ...
3
votes
1answer
2k views

How to deal with the terminal state in SARSA in a multi-agent setting?

I'm training a SARSA agent to update a Q function, but I'm confused about how you handle the final state. In this case, when the game ends and there is no $S'$. For example, the agent performed an ...
6
votes
2answers
877 views

How should I handle action selection in the terminal state when implementing SARSA?

I recently started learning about reinforcement learning. Currently, I am trying to implement the SARSA algorithm. However, I do not know how to deal with $Q(s', a')$, when $s'$ is the terminal state. ...
1
vote
1answer
74 views

What are the variables that need to be saved and loaded, so that a DQN model starts where it left off?

TensorFlow allows users to save the weights and the model architecture, however, that will be insufficient unless the values of certain other variables are also stored. For instance, in DQN, if $\...
1
vote
0answers
24 views

What is the score used to visualize attention in this paper?

I'm reading this paper Global-Locally Self-Attentive Dialogue State Tracker and follow through the implementation published in GLAD. I was wondering if someone can clarify what variable or score is ...
2
votes
1answer
70 views

From an implementation point of view, what are the main differences between an RNN and a CNN?

I understand that in general an RNN is good for time series data and a CNN image data, and have noticed many blogs explaining the fundamental differences in the models. As a beginner in machine ...
2
votes
1answer
2k views

Why do we use a last-in-first-out queue in depth-first search?

Why do we use a last-in-first-out (LIFO) queue in the depth-first search algorithm? In the breadth-first search algorithm, we use a first-in-first-out (FIFO) queue, so I am confused.
1
vote
1answer
228 views

What is a RAM state in the gym's breakout-ram environment?

I have encountered the gym environment and decided to create AI that plays breakout. Here is the link: https://gym.openai.com/envs/Breakout-ram-v0/. The documentation says that the state is ...
2
votes
1answer
238 views

What made your DDPG implementation on your environment work?

I am working on scheduling problem that has inherent randomness. The dimensions of action and state spaces are 1 and 5 respectively. I am using DDPG, but it seems extremely unstable, and so far it ...
1
vote
1answer
59 views

Is there a general file type associated with AI projects?

This is a general question. Is there a general file type associated with AI projects? Photoshop = .psd Excel = csv Artificial Intelligence = ?
3
votes
0answers
193 views

Understanding log probabilities of actions in the PPO objective

I'm trying to implement the Proximal Policy Optimization (PPO) algorithm (code here), but I am confused about certain concepts. What is the correct way to implement log probability of a policy (...
1
vote
1answer
48 views

Is there a good and easy paper to code policy gradient algorithms (REINFORCE) from scratch?

I am interested in learning about policy gradient algorithms and REINFORCE. Can you suggest a good and easy paper that I can use to code them from scratch?
1
vote
1answer
97 views

Why isn't the implementation of my policy evaluation for a simple MDP converging?

I am trying to code out a policy evaluation algorithm to find the $V^\pi(s)$ for all states. The following diagram below shows the MDP. In this case i let p = q = 0.5. the rewards for each states ...
2
votes
1answer
51 views

How can I implement policy evaluation when reward is tied to an action outcome?

I'm following Stanford reinforcement learning videos on youtube. One of the assignments asks to write code for policy evaluation for Gym's FrozenLake-v0 environment. In the course (and books I have ...
3
votes
0answers
584 views

How much time does it take to train DQN on Atari environment?

I am trying to build a DQN model for the Atari Pong game, but I am not sure whether the model is learning at all. I am using the architecture described in the paper Playing Atari with Deep ...
2
votes
0answers
40 views

How can I get to a final output of shape $224 \times 224$, without FC layers, from a tensor of specific shape, in OpenPose?

I am approaching the implementation of the OpenPose algorithm for realtime human body pose estimation. According to the official paper OpenPose: Realtime Multi-Person 2D Pose Estimation using Part ...
10
votes
4answers
767 views

Are we technically able to make, in hardware, arbitrarily large neural networks with current technology?

If neurons and synapses can be implemented using transistors, what prevents us from creating arbitrarily large neural networks using the same methods with which GPUs are made? In essence, we have ...
4
votes
2answers
215 views

How is parallelism implemented in RL algorithms like PPO?

There are multiple ways to implement parallelism in reinforcement learning. One is to use parallel workers running in their own environments to collect data in parallel, instead of using replay memory ...
2
votes
0answers
460 views

How can I implement derivative of softmax function for matrices in Python? [closed]

I have trouble understanding how to implement derivative of softmax function. Here is what I tried: ...