题目
题目链接
自定义Dense层（Custom Dense Layer）是一种常用的神经网络层，其计算公式为：
\[y = \sigma(Wx) + b
\]
其中，\(W\)是权重矩阵，\(x\)是输入，\(b\)是偏置，\(\sigma\)是激活函数。
本质是全连接层，通过矩阵乘法和偏置实现线性变换，再通过激活函数实现非线性变换。
标准代码如下
class Dense(Layer):def __init__(self, n_units, input_shape=None):self.layer_input = Noneself.input_shape = input_shapeself.n_units = n_unitsself.trainable = Trueself.W = Noneself.w0 = Nonedef initialize(self, optimizer):limit = 1 / math.sqrt(self.input_shape[0])self.W = np.random.uniform(-limit, limit, (self.input_shape[0], self.n_units))self.w0 = np.zeros((1, self.n_units))self.W_opt = copy.copy(optimizer)self.w0_opt = copy.copy(optimizer)def parameters(self):return np.prod(self.W.shape) + np.prod(self.w0.shape)def forward_pass(self, X, training=True):self.layer_input = Xreturn X.dot(self.W) + self.w0def backward_pass(self, accum_grad):W = self.Wif self.trainable:grad_w = self.layer_input.T.dot(accum_grad)grad_w0 = np.sum(accum_grad, axis=0, keepdims=True)self.W = self.W_opt.update(self.W, grad_w)self.w0 = self.w0_opt.update(self.w0, grad_w0)accum_grad = accum_grad.dot(W.T)return accum_graddef output_shape(self):return (self.n_units, )
