Questions tagged [implementation]
For questions about implementation of software/algorithms related to Artificial Intelligence.
15
questions
6
votes
1
answer
4k
views
How can the convolution operation be implemented as a matrix multiplication?
How can the convolution operation used by CNNs be implemented as a matrix-vector multiplication? We often think of the convolution operation in CNNs as a kernel that slides across the input. However, ...
- 33.8k
17
votes
2
answers
10k
views
How to implement an "unknown" class in multi-class classification with neural networks?
For example, I need to detect classes for MNIST data. But I want to have not 10 classes for digits, but also I want to have 11th class "not a digit", so that any letter, any other type of ...
6
votes
2
answers
591
views
How is this Pytorch expression equivalent to the KL divergence?
I found the following PyTorch code (from this link)
-0.5 * torch.sum(1 + sigma - mu.pow(2) - sigma.exp())
where mu is the mean ...
- 597
6
votes
4
answers
614
views
Is it possible to implement Asimov's Three Laws of Robotics?
Would it be possible to put Asimov's three Laws of Robotics into an AI?
The three laws are:
A robot (or, more accurately, an AI) cannot harm a human being, or through inaction allow a human being to ...
- 1,706
8
votes
2
answers
1k
views
How should I handle action selection in the terminal state when implementing SARSA?
I recently started learning about reinforcement learning. Currently, I am trying to implement the SARSA algorithm. However, I do not know how to deal with $Q(s', a')$, when $s'$ is the terminal state. ...
- 522
13
votes
4
answers
9k
views
What does AI software look like, and how is it different from other software?
What does AI software look like? What is the major difference between AI software and other software?
- 233
9
votes
5
answers
308
views
Why isn't ethics more integrated into current AI systems?
I am a PhD student in computer science, and currently creating a state of the art overview in applications done in Machine Ethics (a multidisciplinary field combining philosophy and AI, that looks at ...
- 99
8
votes
1
answer
2k
views
What are other ways of handling invalid actions in scenarios where all rewards are either 0 (best reward) or negative?
I created an OpenAI Gym environment, and I would like to check the performance of the agent from OpenAI Baselines DQN approach on it.
In my environment, the best possible outcome for the agent is 0 - ...
- 263
2
votes
2
answers
98
views
Which of these two numerical methods for z-score normalisation is preferable, in multivariate linear regression?
In the exercise Exercise 3: Multivariate Linear Regression, by Andrew Ng, the author suggests to "scale both types of inputs by their standard deviations and set their means to zero".
$$x_{n e w}=\...
- 196
4
votes
3
answers
471
views
Why can the Bellman equation be turned into an update rule?
In chapter 4.1 of Sutton's book, the Bellman equation is turned into an update rule by simply changing the indices of it. How is it mathematically justified? I didn't quite get the initiation of why ...
- 113
3
votes
0
answers
414
views
Understanding log probabilities of actions in the PPO objective
I'm trying to implement the Proximal Policy Optimization (PPO) algorithm (code here), but I am confused about certain concepts.
What is the correct way to implement log probability of a policy (...
- 161
3
votes
1
answer
2k
views
How to deal with the terminal state in SARSA in a multi-agent setting?
I'm training a SARSA agent to update a Q function, but I'm confused about how you handle the final state. In this case, when the game ends and there is no $S'$.
For example, the agent performed an ...
- 393
1
vote
2
answers
1k
views
How should I implement the backward pass through a flatten layer of a CNN?
I am making a NN library without any other external NN library, so I am implementing all layers, including the flatten layer, and algorithms (forward and backward pass) from scratch. I know the ...
- 1,715
0
votes
1
answer
793
views
Why is this PyTorch implementation of the actor-critic algorithm inconsistent with the mathematical formulas?
This PyTorch implementation of the actor-critic algorithm calculates the losses like so:
...
- 669
0
votes
1
answer
42
views
What are the (key) purposes of unsqueezing operation on tensors?
The unsqeeze operation is used in several deep learning algorithms. However, I only found this operation in the code/implementation of the algorithms presented in the papers, which do not mention it.
...
- 3,099