# All Questions

6,603 questions
Filter by
Sorted by
Tagged with
10 views

### Is the distribution of state-action pairs from sample based planning accurate for small experience sets?

From the David Silver lectures - based on Sutton and Barto - he talks about using sample based planning to use our model to take a sample of a state and then use model-free planning, such as monte ...
44 views

### What is the effect of using pooling layers in CNNs?

I know how pooling works, and what effect it has on the input dimensions - but I'm not sure why it's done in the first place. It'd be great if someone could provide some intuition behind it - while ...
17 views

### What is the intuition behind the Xavier initialization for deep neural networks?

The aim of weight initialization is to prevent layer activation outputs from exploding or vanishing during the course of a forward pass through a deep neural network I am really having trouble ...
11 views

### What is an auto-associator?

What is an auto-associator, and how does it work? How can we design an auto-associator for a given pattern? I couldn't find a clear explanation for this anywhere on the internet. Here's an example of ...
23 views

### Does anyone know of a model for comparing the eyes of people in two images to see if they match?

There’s a lot of talk of undercover cops intentionally starting violence in otherwise peaceful protests. The evidence, primarily, are images like this. https://images.app.goo.gl/4n3o2EXwFzMQfsKq6 ...
25 views

### Why is learning $s'$ from $s,a$ a kernel density estimation problem but learning $r$ from $s,a$ is just regression?

In David Silver's 8th lecture he talks about model learning and says that learning $r$ from $s,a$ is a regression problem whereas learning $s'$ from $s,a$ is a kernel density estimation. His ...
18 views

### Why is this GAN not converging?

This GAN being trained with CelebA dataset doesn't seem to mode collapse, discriminator is not really over confident, and yet the quality is stuck on these rough Picasso-like generator images. Using ...
10 views

### Visualisation for Features to Predict Timeseries Data

I have a course assignment to use an LSTM to predict the movement directions of stock prices. One of the things I am asked to do is provide a visualization to compare the predictive powers of a set of ...
15 views

### Deep learning techniques with time-fixed, time-dependent and imaging data

I have a question about the use of deep learning techniques with time-fixed features and images (setting 1) and time-dependent features (setting 2). (I am pretty new to the deep learning world so ...
281 views

### Is there any good reference for double deep Q-learning?

I am new in reinforcement learning, but I already know deep Q-learning and Q-learning. Now, I want to learn about double deep Q-learning. Do you know any good references for double deep Q-learning? ...
27 views

### Is there a way of deriving a loss function given the neural network and training data?

There is some sort of art to using the right loss function. However, I was wondering if there is a way to derive the loss function if I gave you a neural network model (the weights) as well as the ...
15 views

### VAE generates bad images. due to unbalanced loss functions? [closed]

I'm training a variational autoencoder on CelebA dataset using TensorFlow.keras The problem I'm facing is that the generated images are not diverse enough and look kinda bad. example: What I think:...
13 views

### Incorporating domain knowledge into recurrent network

I am currently trying to solve a classification task with a recurrent artificial neural network (RNN). Situation There are up to 350 inputs (X) mapped on one categorical output (y)(13 differnt ...
23 views

### How do you prove that minimax algorithm outputs a subgame-perfect Nash equilibrium?

At every node, MAX would always move to maximise the minimum payoff while MIN choose to minimise the maximum payoff, hence there is nash equilibrium. By using backwards induction, at every node, MAX ...
17 views

### How to effectively crossover mathematical curves?

I'm trying to optimize some reflective properties of curves of the form: $a_1x^n+a_2x^{n-1}+a_3x^{n-2} + ... + a_n + b_1y^n+b_2y^{n-1}+b_3y^{n-2} + ... + b_n = 0$ which is basically the curve that ...
22 views

### What are finite horizon look-ahead policies in reinforcement learning?

I was reading the paper How to Combine Tree-Search Methods in Reinforcement Learning published in AAAI Conference 2019. It starts with the sentence Finite-horizon lookahead policies are abundantly ...
37 views

### How should I decay $\epsilon$ in Q-learning?

How should I decay the $\epsilon$ in Q-learning? Currently, I am decaying epsilon as follows. I initialize $\epsilon$ to be 1, then, after every episode, I multiply it by some $C$ (let it be $0.999$)...
8 views

### Keras: how to comiple a model with a custom loss and evaluate the model with another loss? [migrated]

I'm using Keras to do some experiments. As we said in the subject, is it possible to model.compile() with a custom loss and, then, use another loss with model.evaluate()?
8 views

### Flatten in simple feedforward networks [migrated]

I am working on the CIFAR10 dataset and came across this example in Keras, using data augmentation: https://keras.io/examples/cifar10_cnn/ The example uses CNN. I want to implement just a simple ...
32 views

### Understanding the role of the target network in this DQN algorithm

I've found online this interesting algorithm: From what I understand reading this algorithm, I can't figure out why I should "perform the opposite action" and consequently storing that second ...
11 views

### Which one is more important in case of different loss optimization algorithms, Speed or the Route?

We have different kinds of algorithms to optimize the loss like AdaGrad, SGD + Momentum, etc. Some are more commonly used than others. In some algorithms, they usually range out before they converge, ...
8 views

### Training Conditional DCGAN with GAN-CLS loss

I am trying to implement conditional GAN using GAN-CLS loss as described in paper: https://arxiv.org/abs/1605.05396 So, while training discriminator, I should I have three batches of data: [...
23 views

### Simplifying Log Loss

I am reading through a paper (https://www.mitpressjournals.org/doi/pdf/10.1162/0891201053630273) where they describe logloss as a ranking function and can be simplified to the margin of the training ...
19 views

### How could a NN be trained to output a cyclic (e.g. hue) number?

I was thinking about training a neural network to colourize images. The input would be the luminosity/value for each pixel, and the output would be a hue and/or saturation. Training data would be ...
24 views

### Why do we use a delay when feeding our input data to the echo state network?

I'm new to working with neural networks and have recently began implementing neural networks for time series forecasting in some of my work. I've been particularly using Echo State Networks and have ...
8 views

### Will GPU optimized model run on TPU? [migrated]

There is a project which contains models in DLC format (Snapdragon Neural Processing Engine - SNPE) which I guess are optimized for the Qualcomm Snapdragon 820 chipset (see second link). The project ...
20 views

### Is there a classification task with multiple attribute regression?

I'm trying to look for a task that predicts a discrete label first (classification), and then predicts the multiple continuous attributes of the predicted class. I found some papers about multi-output ...
108 views

### Is this proof of $\epsilon$-greedy policy improvement correct?

The text book being referred to, in this question is "Reinforcement Learning: An introduction" by Richard Sutton and Andrew Barto (second edition, 2018). For your convenience, I have enclosed the ...
45 views

### How and when should we update the Q-target in deep Q-learning?

I have recently watched David silver's course, and started implementing the deep Q-learning algorithm. I thought I should make a switch between the Q-target and Q-current directly (meaning, every ...
113 views

### Is there any good source for when the pole actually starts all the way at the bottom, in the cartpole problem?

There are a lot of examples of balancing a pole (see image below) using reinforcement learning, but I find that almost all examples start close to the upright position. Is there any good source (or ...
35 views

### Using a model-based method to build an accurate day trading environment model

There are several different angles we can classify Reinforcement Learning methods from. We can distinguish three main aspects : Value-based and policy-based On-policy and off-policy Model-free and ...
18 views

### Can you find another reason for sample inefficiency of model-free on-policy Deep Reinforcement Learning?

The following mindmap gives an overview of multiple reasons for sample inefficiency. The list is definitely not complete. Can you see another reason not mentioned so far? Some related links: ...
18 views

### Why do we also need to normalize the action's values on continuous action spaces?

I was reading here tips & tricks for training in DRL and I noticed the following: always normalize your observation space when you can, i.e., when you know the boundaries normalize your ...
31 views

### Is it possible to create a fair machine learning system?

Started thinking about fairness of machine learning models recently. Wiki page for Fairness_(machine_learning) defines fairness as: In machine learning, a given algorithm is said to be fair, or to ...
14 views

### Low accuracy during training for text summarization

I am trying to implement an extractive text summarization model. I am using keras and tensorflow. I have used bert sentence embeddings and the embeddings are fed into an LSTM layer and then to a Dense ...
14 views

### Error while implementing SVM using cvxopt in python [closed]

I'm trying to implement SVM from scratch using cvxopt.But I'm not getting desirable results.To debug I compared my implementation's support vectors with sklearn's support vectors using ...
11 views

### Is the number of layers in the simple RNN fixed or is it random? [closed]

When I search for RNN, I find LSTM most of the times. Before I go on o for reading more about LSTM, I want to explore the vanilla RNN. I want to know an explanation of working with mathematical ...
43 views

### Is there a general file type associated with AI projects?

This is a general question. Is there a general file type associated with AI projects? Photoshop = .psd Excel = csv Artificial Intelligence = ?
35 views

### Can weighted importance sampling be applied to off-policy evaluation for continuous state space MDPs?

Can weighted importance sampling (WIS) and importance sampling (IS) be applied to off-policy evaluation for continuous state spaces MDPs? Given that I have trajectories of $(s_t,a_t)$ pairs and the ...
19 views

### Regularization to enforce feature learning

Is there any research into ways to enforce feature selection across classes by network structure? Given the number of parameters in NN, even convnets are prone to over fitting. I'm curious if there ...
19 views

### Why is the accuracy of my model very low on a separate dataset from the training and test datasets?

I am working on stock price prediction project, I am using the support vector regression (SVR) model for it. As I am splitting my data into train and test, I am getting high accuracy while predicting ...
17 views

### Banding artifacts in CNN

I was working on a CNN for HDR image generation from LDR images. I used an encoder-decoder architecture and merged the input with the decoder output. However I'm getting some banding artifacts in the ...
14 views

### How to use one-hot encoding for multiple columns (multi-class) with varying number of labels in each class?

I am a beginner in TensorFlow as well as in AI. I am basically from Pharma background and learning AI from scratch. I have data with 5038 input (Float64) and 826 output (Categorical - Multi Labels in ...
11 views

### How are the conditional probabilities for the visible and hidden units computed, in a RBM with multinomial units?

When using multinomial units for visible units of an RBM as described in section 13.1 of this training guide, how are the conditional probabilities for the visible and hidden units computed? Is it the ...
10 views

### How to make a multivariate forecasting if one of features becomes known for the future with some confidence level, e.g. weather forecast data

Let's assume that we make forecasting of another metric partially based on forecasts of the weather forecast, e.g. of temperature, pressure, then we can potentially obtain those forecasts from one of ...
8 views

### Where are some good online covid 19 genetic datasets? [closed]

I want to carry out some research with covid19 genetic datasets that include people infected by corona virus and people that aren't infected by it. Does anybody know any good ones? My main purpose is ...
64 views

### How do I convert an MDP with the reward function in the form $R(s,a,s')$ to and an MDP with a reward function in the form $R(s,a)$?

The AIMA book has an exercise about showing that an MDP with rewards of the form $r(s,a,s')$ can be converted to an MDP with rewards $r(s,a)$, and to an MDP with rewards $r(s)$ with equivalent optimal ...
24 views

### Why is the hypothesis function $h_{\theta}(x)$ equivalent to $E[y | x; \theta]$ in generalised linear models?

Reading through the CS229 lecture notes on generalised linear models, I came across the idea that a linear regression problem can be modelled as a Gaussian distribution, which is a form of the ...