# All Questions

6,580 questions
Filter by
Sorted by
Tagged with
6 views

### Fast R-CNN region proposal method

I read so many articles and the Fast RCNN paper, but I'm still confused on how the region proposal method works in Fast RCNN, as you can see in the image below they say they used a proposal method but ...
128 views

### Are Q-learning and SARSA the same when action selection is greedy?

I'm currently studying reinforcement learning and I'm having difficulties with question 6.12 in Sutton and Barto's book. Suppose action selection is greedy. Is Q-learning then exactly the same ...
33 views

### How do you find the homography matrix given 4 points in both images?

I want to understand the process of finding a homography matrix given 4 points in both images. I am able to do that in python OpenCV, but I wonder how it works behind the scenes. Suppose I have ...
14 views

### Trying to Train Cards Game with RL

I am trying to train a card game called Callbreak I tried inputs like all the opponents discarded cards all hands everything a human can see and calculate with "common sense" I fed it to the Agent but ...
24 views

### What's the meaning of the Jaccard decay and the Jaccard recall? [closed]

I know the meaning of the Jaccard index, but when it comes to Jaccard decay and Jaccard recall I cannot see the difference or the meaning of it.
12 views

### How are AI's strategies discovered from chess and starcraft games?

Chess and starcraft games have been played by AI and sometimes won. Artificial intelligence had for these games a complex set of rules, database and decision-making tools to decide which actions it ...
21 views

### What do the authors of this paper mean by the bias term in this picture of a neural network implementation?

I am reading a paper implementing a deep deterministic policy gradient algorithm for portfolio management. My question is about a specific neural network implementation they depict in this picture (...
46 views

### Why can't DQN be used for self-driving cars?

Why can't DQN be used for self-driving cars? Why can't DQN and similar RL algorithms be used for self-driving cars? The reason why I am curious is that it successfully plays go and other multistate ...
16 views

### Is the high dimensionality of input vectors a problem for a radial basis function neural network?

I have a dataset A of videos. I've extracted the feature vector of each video (with a convolutional neural network, via transfer learning) creating a dataset B. Now, every vector of the dataset B has ...
21 views

### What are some papers on face comparison in videos?

I have the following problem. Given two random frames in a video of some people, assume I can detect all faces in both frames. I would like to create a classifier that takes in pairs of faces out of ...
120 views

### How can blackjack be formulated as a Markov decision process?

I am reading sutton barton's reinforcement learning textbook and have come across the finite Markov decision process (MDP) example of the blackjack game (Example 5.1). Isn't the environment ...
28 views

### Which activation functions should I use for polynomial regression?

I am a beginner in machine learning and neural networks. I have only used neural networks for classification problems. My aim is to modify it so that it can work for polynomial regression as well. In ...
50 views

### Should Monte Carlo tree search be able to consistently beat me in the connect four game?

I've implemented the Monte Carlo tree search (MCTS) algorithm for a connect four game I've built. The MCTS agent beats a random choice agent 90-100% of the time, but I’m still able to beat it pretty ...
47 views

### Is there 1-dimensional reinforcement learning?

From what I can find, reinforcement algorithms work on a grid or 2-dimensional environment. How would I set up the problem for an approximate solution when I have a 1-dimensional signal from a light ...
26 views

### How to approach learning(deeper understanding) about AI? [closed]

I already took an undergrad class about AI. We used AIMA as a reference and CS 188 course materials from UC Berkeley. My issue with learning is that we took a very shallow approach to learning about ...
76 views

### What are the differences between artificial neural networks and other function approximators?

Modern artificial neural networks use a lot more functions than just the classic sigmoid, to the point I'm having a hard time really seeing what classifies something as a "neural network" over other ...
14 views

### 3d representation of a regression with two independent variables one of them is categorical and another is continuous

I have hopefully a fundamental question of Do I understand things right. (Thank you in advance and sorry for my English which might be not so good) 1-Preambula 1: I know that if we have 2 independent ...
39 views

### Simplification of expected reward under the limit in continuous tasks

I was reading the average reward setting for continuous tasks from rich sutton's book (page 202, 2nd edition). There he perform a simplification over the expected reward under the limit approaching to ...
86 views

### How long should the state-dependent baseline for policy gradient methods be trained at each iteration?

How long should the state-dependent baseline be trained at each iteration? Or what baseline loss should we target at each iteration for use with policy gradient methods? I'm using this equation to ...
82 views

### Is AI already being used in the drug industry to combat the COVID-19?

We all have heard about how beneficial AI can be in health. There are plenty of papers and research about confronting against diseases, like cancer. However in 2020 with COVID-19 be one of the most ...
6 views

### How to feed decimal_y_values for training where last layer of model has 10 neurons with softmax activation and loss is Earth Mover Loss [migrated]

What should be the format/data types of y-labels for training if the actual y-labels cab be any decimal number between 0-9 (4.1,8.5 etc) and the last output layer is defined as: ...
55 views

### How to implement AI strategy for Mastermind

I'm looking to implement a AI for the turn-based game Mastermind in Node.JS, using Google's Tensorflow library. Basically the AI needs to predict the 4D input for the optimal 2D output ...
16 views

### Are there any general guidelines for dealing with imbalanced data through upsampling or downsampling?

Are there any general guidelines for dealing with imbalanced data through upsampling/downsampling? This Google developer guide suggests performing downsampling with upweighting, but for the most ...
30 views

### My CNN model performs bad on new (self-created) pictures, what are possible reasons?

I wanted to train a model that recognizes sign language. I have found a dataset for this and was able to create a model that would get 94% accuracy on the test set. I have trained models before and my ...
12 views

### What is the right way to set the dimension of the word representation in SkipGram word2vec?

I know that in word2vec each word has two word representations; one for the center word and one for the context word. What is the right way to set the dimension of the center word using SkipGram, ...
33 views

### Can we combine Off-Policy with On-Policy Algorithms?

On-Policy Algorithms like PPO directly maximize the performance objective or an approximation of it. They tend to be quite stable and reliable but are often sample inefficient. Off-Policy Algorithms ...
36 views

### How should I deal with variable batch size in A3C?

I am fairly new to reinforcement learning (RL) and deep RL. I have been trying to create my first agent (using A3C) that selects an optimal path with the reward being some associated completion time (...
16 views

### How do I determine the genomes to use for crossover in NEAT?

If I have the fitness of each genome, how do I determine which genome will crossover with which, and so on, so that I get a new population? Unfortunately, I can't find anything about it in the ...
17 views

### Which NLP model to use to handle long context?

I'm trying to process product data for an e-commerce platform. The goal is to understand products' size. Just to show you some examples on how messy product dimension description is: ...
7 views

### What is the advantages of using FPGA for deep learning- computer vision task? [migrated]

I noticed some developers prefer to use FPGA like XILINX for their deep learning applications, why they prefer to use FPGA instead of GPU?! Is there any reason for that? I suppose GPUs are very ...
10 views

### How to create vector representation of roadmap like scans

What would be the best way to create a vector representation of roadmap like scans? The goal I am trying to achieve is illustrated below. The left side represents the source image, the right side the ...
26 views

### Are activation functions applied to feature maps?

If I have a convolutional neural network, and I convolve my input tensor with a kernel, the output is a feature map. Is an activation function then applied to this feature map? If its an image that ...
19 views

### How does heuristic work with multiple agents?

I have a question for heuristic search with multiple agents. I know how heuristic search works with one agent (ex. one Pacman) but I don't really understand it with multiple agents. Let's say we have ...
43 views

### How does the AlphaGo Zero policy decide what move to execute?

I was going through the AlphaGo Zero paper and I was trying to understand everything, but I just can't figure out this one formula:  \pi(a \mid s_0) = \frac{N(s_0, a)^{\frac{1}{\tau}}}{\sum_b N(s_0,...
76 views

### Are these two definitions of the state-action value function equivalent?

I have been reading the Sutton and Barto textbook and going through David Silvers UCL lecture videos on YouTube and have a question on the equivalence of two forms of the state-action value function ...
17 views

### An Encoder-Decoder based CNN to predict a tensor of points

So I have with me a data of rendered 2D images of a 3D object and along with that, I have the image projection coordinates (X, Y) of all the voxels that are in the ...
51 views

### Why can't pure KG embedding methods discover multi-hop relations paths?

According to Reinforcement Knowledge Graph Reasoning for Explainable Recommendation pure KG embedding methods lack the ability to discover multi-hop relational paths. Why is it so?
18 views

### Designing a chatbot personal project with zero coding experience, using an existing platform

My girlfriend has a masters degree in linguistics and would like to create an AI chatbot personal project to show potential employers her linguistics skills since she is struggling to find a job. ...
18 views

### Creating Text Features using word2vec

My task is to classify some texts. I have used word2vec to represent text words and I pass them to an LSTM as input. Taking into account that texts do not contain the same number of words, is it a ...
19 views

### Which word representation should I use in word2vec?

I know that in word2vec each word has two word representations; one for center word and one for context word. In NLP, when I want to set the words of a text as an input of an LSTM/RNN using word2vec, ...
46 views

...
12 views

### What is 3D face recognition? and how we can check liveness of a face image?

Actually what is mean by 3D face recognition? In normal cases we are extracting face encoding s from a 2D image,right? Is 3D face recognition is used for liveness detection? how its possible?
15 views

### How can raw data from a motion sensor (like an IMU) reduced to the main points of the data

How can I reduce the caputured movement data of a person in a way, that I have filtered the main features of the movement. Or how can I detect pattern/ main features in that data set? I captured some ...
43 views

### What does it mean to parameterise a policy in policy gradient methods?

Can you explain policy gradient methods and what it means for the policy to be parameterised? I am reading Sutton and Barto book on reinforcement learning and didn't understand well what it is, can ...
32 views

### Is there any research on models that provide uncertainty estimation?

Is there any research on machine learning models that provide uncertainty estimation? If I train a denoising autoencoder on words and put through a noised word, I'd like it to return a certainty that ...
31 views

### book suggestion for reinforcement learning [closed]

currently I am reading Sutton and Barto book on reinforcement learning, it doesn't go in depth, and hence I don't understand most of the concepts.Currently I am on policy gradient methods, he doesn't ...
37 views

### Q table not converging for an arbitrary experiment

This is an experiment in order to understand the working of Q table and Q learning. I have the states as states = [0,1,2,3] I have an arbitrary value for each ...
24 views

### State-of-the-art algorithms not working on a custom RL environment

I'm trying to train a RL agent on a custom, highly stochastic environment (MDP). In order to do so I'm using existing implementations of state-of-the-art RL algorithms as provided by Stable Baselines. ...