Questions tagged [implementation]

For questions about implementation of software/algorithms related to Artificial Intelligence.

Filter by
Sorted by
Tagged with
6 votes
1 answer
4k views

How can the convolution operation be implemented as a matrix multiplication?

How can the convolution operation used by CNNs be implemented as a matrix-vector multiplication? We often think of the convolution operation in CNNs as a kernel that slides across the input. However, ...
user avatar
  • 33.8k
17 votes
2 answers
10k views

How to implement an "unknown" class in multi-class classification with neural networks?

For example, I need to detect classes for MNIST data. But I want to have not 10 classes for digits, but also I want to have 11th class "not a digit", so that any letter, any other type of ...
user avatar
6 votes
2 answers
591 views

How is this Pytorch expression equivalent to the KL divergence?

I found the following PyTorch code (from this link) -0.5 * torch.sum(1 + sigma - mu.pow(2) - sigma.exp()) where mu is the mean ...
user avatar
6 votes
4 answers
614 views

Is it possible to implement Asimov's Three Laws of Robotics?

Would it be possible to put Asimov's three Laws of Robotics into an AI? The three laws are: A robot (or, more accurately, an AI) cannot harm a human being, or through inaction allow a human being to ...
user avatar
  • 1,706
8 votes
2 answers
1k views

How should I handle action selection in the terminal state when implementing SARSA?

I recently started learning about reinforcement learning. Currently, I am trying to implement the SARSA algorithm. However, I do not know how to deal with $Q(s', a')$, when $s'$ is the terminal state. ...
user avatar
13 votes
4 answers
9k views

What does AI software look like, and how is it different from other software?

What does AI software look like? What is the major difference between AI software and other software?
user avatar
  • 233
9 votes
5 answers
308 views

Why isn't ethics more integrated into current AI systems?

I am a PhD student in computer science, and currently creating a state of the art overview in applications done in Machine Ethics (a multidisciplinary field combining philosophy and AI, that looks at ...
user avatar
  • 99
8 votes
1 answer
2k views

What are other ways of handling invalid actions in scenarios where all rewards are either 0 (best reward) or negative?

I created an OpenAI Gym environment, and I would like to check the performance of the agent from OpenAI Baselines DQN approach on it. In my environment, the best possible outcome for the agent is 0 - ...
user avatar
2 votes
2 answers
98 views

Which of these two numerical methods for z-score normalisation is preferable, in multivariate linear regression?

In the exercise Exercise 3: Multivariate Linear Regression, by Andrew Ng, the author suggests to "scale both types of inputs by their standard deviations and set their means to zero". $$x_{n e w}=\...
user avatar
  • 196
4 votes
3 answers
471 views

Why can the Bellman equation be turned into an update rule?

In chapter 4.1 of Sutton's book, the Bellman equation is turned into an update rule by simply changing the indices of it. How is it mathematically justified? I didn't quite get the initiation of why ...
user avatar
3 votes
0 answers
414 views

Understanding log probabilities of actions in the PPO objective

I'm trying to implement the Proximal Policy Optimization (PPO) algorithm (code here), but I am confused about certain concepts. What is the correct way to implement log probability of a policy (...
user avatar
3 votes
1 answer
2k views

How to deal with the terminal state in SARSA in a multi-agent setting?

I'm training a SARSA agent to update a Q function, but I'm confused about how you handle the final state. In this case, when the game ends and there is no $S'$. For example, the agent performed an ...
user avatar
  • 393
1 vote
2 answers
1k views

How should I implement the backward pass through a flatten layer of a CNN?

I am making a NN library without any other external NN library, so I am implementing all layers, including the flatten layer, and algorithms (forward and backward pass) from scratch. I know the ...
user avatar
  • 1,715
0 votes
1 answer
793 views

Why is this PyTorch implementation of the actor-critic algorithm inconsistent with the mathematical formulas?

This PyTorch implementation of the actor-critic algorithm calculates the losses like so: ...
user avatar
  • 669
0 votes
1 answer
42 views

What are the (key) purposes of unsqueezing operation on tensors?

The unsqeeze operation is used in several deep learning algorithms. However, I only found this operation in the code/implementation of the algorithms presented in the papers, which do not mention it. ...
user avatar
  • 3,099