enter image description here

This is how they describe their infrastructure in https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf. I want to implement the game of Atari Breakout.

import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F

class DQN(nn.Module):
    def __init__(self, height, width):
        super(DQN, self).__init__()

        self.height = height
        self.width = width

        self.conv1 = nn.Conv2d(in_channels=4, out_channels=16, kernel_size=8, stride=4)
        self.conv2 = nn.Conv2d(in_channels=6, out_channels=32, kernel_size=4, stride=2)

        self.fc = nn.Linear(in_features=????, out_features=256)
        self.out = nn.Linear(in_features=256, out_features=4)

    def forward(self, state):

        # (1) Hidden Conv. Layer
        self.layer1 = F.relu(self.conv1(state))

        #(2) Hidden Conv. Layer
        self.layer2 = F.relu(self.conv2(self.layer1))

        #(3) Hidden Linear Layer
        self.layer3 = self.fc(self.layer2)

        #(4) Output
        actions = self.out(self.layer3)

        return actions

I will probably instantiate my policy network and my target network the following way :

policy_net = DQN(envmanager.get_height(), envmanager.get_width()).to(device)
target_net = DQN(envmanager.get_height(), envmanager.get_width()).to(device)

I am very new in the world of Reinforcement Learning. I would like to implement their infrastructure in the DQN(), but I think I am wrong in several places. Am I good here? If not, how can I fix it so that it reflect the infrastructure from the above picture?


I know that the formula to calculate the output size is equal to


where $O$ is the output height/length, $W$ is the input height/length, $K$ is the filter size, $P$ is the padding, and $S$ is the stride.

I obtained forself.fc = nn.Linear(in_features=????, out_features=256) that in_features must be equal to $32*9*9$



You must log in to answer this question.

Browse other questions tagged .