# How to train a reinforcement learning agent from raw pixels?

How would you train a reinforcement learning agent from raw pixels?

For example, if you have 3 stacked images to sense motion, then how would you pass them to neural networks to output Q-learning values?

If you pass that batch output, it would be a batch of values, so from here it is impossible to deduce which ones are the true Q-values for that state.

Currently, I am watching a YouTuber: Machine Learning with Phil, and he did it very differently. On the 13th minute, he defined a network that outputs a batch of values rather than Q-values for 6 states. In short, he outputs a matrix rather than a vector.

A Convolutional Neural Network (CNN) structure is a standard neural network architecture when working with 2D pixel input in reinforcement learning, and it is the technique used in the original DQN paper (see paragraphs 1 & 3 of Section 4.1 of https://arxiv.org/abs/1312.5602). CNNs typically take 3-dimensional input, where the first two dimensions are height and width of your images, and the third is rgb color. The technique in the paper was to convert each RGB frame (or image) to greyscale format (so it has only 1 color channel/dimension instead of 3) and instead use the rgb_color dimension as a frames dimension that is indexed by each stacked frame.