0
$\begingroup$

I am following this book and I am trying to visualize the network. This part seems tricky to me and I am trying to get my head around it by visualizing it:

import numpy as np
import nnfs
from nnfs.datasets import spiral_data

nnfs.init()


class Layer_Dense:

    # Layer initialization
    def __init__(self, n_inputs, n_neurons):
        # Initialize weights and biases
        self.weights = 0.01 * np.random.randn(n_inputs, n_neurons)
        self.biases = np.zeros((1, n_neurons))

    # Forward pass
    def forward(self, inputs):
        # Calculate output values from inputs, weights and biases
        self.output = np.dot(inputs, self.weights) + self.biases

# ReLU activation

class Activation_Relu():
   
   # forward pass
   def forward(self, inputs):
     # calculate output values from inputs
     self.output = np.maximum(0,inputs)
create dense layer with 2 input features and 3 output values
dense1 = Layer_Dense(2, 3)

# create ReLU activation which will be used with Dense layer
activation1 = Activation_Relu()

# create second dense layer with 3 input features from the previous layer and 3 output values
dense2 = Layer_Dense(3,3)
# create dataset
X, y = spiral_data(samples = 100, classes = 3)

dense1.forward(X)
activation1.forward(dense1.output)
dense2.forward(activation1.output)

My input data X is an array of 300 rows and 2 columns, meaning each of my 300 inputs will have 2 values that describe it.

The Layer_Dense class is initialized with parameters (2, 3) meaning that there are 2 inputs and 3 neurons.

At the moment my variables look like this:

X.shape # (300, 2) 
x[:5]
# [[ 0.        ,  0.        ],
# [ 0.00279763,  0.00970586],
# [-0.01122664,  0.01679536],
# [ 0.02998079,  0.0044075 ],
# [-0.01222386,  0.03851056]]
dense1.weights.shape
# (2, 3)

dense1.weights 
# [[0.00862166, 0.00508044, 0.00461094],
# [0.00965116, 0.00796512, 0.00558731]])

dense1.biases
# [[0., 0., 0.]]

dense1.output.shape
(300, 3)
print(dense1.output[:5])

# [[0.0000000e+00 0.0000000e+00 0.0000000e+00]
# [8.0659374e-05 4.3710388e-05 6.5012209e-05]
# [1.5923499e-04 6.9124777e-05 1.0470775e-04]
# [2.3033096e-04 1.9152602e-04 2.7749798e-04]
# [1.9318146e-04 3.1980115e-04 4.5189835e-04]]

Does this configuration make my network look like so:

Where each of 300 inputs has 2 features

enter image description here

Or like so:

enter image description here

Do I understand this correctly:

  1. There are 300 inputs with 2 features each
  2. Each input is connected to 3 neurons in the first layer, since it is connected to 3 neurons there are 3 weights
  3. Why the shape of weights is (2, 3) instead of (300, 3) since there are 300 inputs with 2 features each, each feature connected to 3 neurons

I have used this to draw networks.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.