Questions tagged [implementation]

For questions about implementation of software/algorithms related to Artificial Intelligence.

Filter by
Sorted by
Tagged with
4
votes
1answer
739 views

How can the convolution operation be implemented as a matrix multiplication?

How can the convolution operation used by CNNs be implemented as a matrix-vector multiplication? We often think of the convolution operation in CNNs as a kernel that slides across the input. However, ...
12
votes
2answers
7k views

How to implement an “unknown” class in multi-class classification with neural networks?

For example, I need to detect classes for MNIST data. But I want to have not 10 classes for digits, but also I want to have 11th class "not a digit", so that any letter, any other type of ...
7
votes
4answers
536 views

Is it possible to implement Asimov's Three Laws of Robotics?

Would it be possible to put Asimov's three Laws of Robotics into an AI? The three laws are: A robot (or, more accurately, an AI) cannot harm a human being, or through inaction allow a human being to ...
6
votes
2answers
863 views

How should I handle action selection in the terminal state when implementing SARSA?

I recently started learning about reinforcement learning. Currently, I am trying to implement the SARSA algorithm. However, I do not know how to deal with $Q(s', a')$, when $s'$ is the terminal state. ...
13
votes
4answers
7k views

What does AI software look like, and how is it different from other software?

What does AI software look like? What is the major difference between AI software and other software?
9
votes
5answers
291 views

Why isn't ethics more integrated into current AI systems?

I am a PhD student in computer science, and currently creating a state of the art overview in applications done in Machine Ethics (a multidisciplinary field combining philosophy and AI, that looks at ...
8
votes
1answer
2k views

What are other ways of handling invalid actions in scenarios where all rewards are either 0 (best reward) or negative?

I created an OpenAI Gym environment, and I would like to check the performance of the agent from OpenAI Baselines DQN approach on it. In my environment, the best possible outcome for the agent is 0 - ...
2
votes
2answers
91 views

Which of these two numerical methods for z-score normalisation is preferable, in multivariate linear regression?

In the exercise Exercise 3: Multivariate Linear Regression, by Andrew Ng, the author suggests to "scale both types of inputs by their standard deviations and set their means to zero". $$x_{n e w}=\...
3
votes
0answers
185 views

Understanding log probabilities of actions in the PPO objective

I'm trying to implement the Proximal Policy Optimization (PPO) algorithm (code here), but I am confused about certain concepts. What is the correct way to implement log probability of a policy (...
3
votes
1answer
2k views

How to deal with the terminal state in SARSA in a multi-agent setting?

I'm training a SARSA agent to update a Q function, but I'm confused about how you handle the final state. In this case, when the game ends and there is no $S'$. For example, the agent performed an ...
3
votes
3answers
218 views

Why can the Bellman equation be turned into an update rule?

In chapter 4.1 of Sutton's book, the Bellman equation is turned into an update rule by simply changing the indices of it. How is it mathematically justified? I didn't quite get the initiation of why ...
1
vote
1answer
625 views

How should I implement the backward pass through a flatten layer of a CNN?

I am making a NN library without any other external NN library, so I am implementing all layers, including the flatten layer, and algorithms (forward and backward pass) from scratch. I know the ...
0
votes
1answer
474 views

Why is this PyTorch implementation of the actor-critic algorithm inconsistent with the mathematical formulas?

This PyTorch implementation of the actor-critic algorithm calculates the losses like so: ...