网站建设网站服务,wordpress edit lock,郑州超凡装饰公司怎么样,公司注册查询系统BatchNormlization BatchNormlization的编码流程#xff1a;
init阶段初始化 C i n C_in Cin大小的scale向量和shift向量#xff0c;同时初始化相同大小的滑动均值向量和滑动标准差向量#xff1b;forward时沿着非channel维度计算均值、有偏方差依据得到均值和有偏方差进…BatchNormlization BatchNormlization的编码流程
init阶段初始化 C i n C_in Cin大小的scale向量和shift向量同时初始化相同大小的滑动均值向量和滑动标准差向量forward时沿着非channel维度计算均值、有偏方差依据得到均值和有偏方差进行归一化对归一化的结果进行缩放和平移
代码 代码如下
class BN(nn.Module):def __init__(self,C_in):super(BN,self).__init__()self.scalenn.Parameter(torch.ones(C_in).view(1,-1,1,1))self.shiftnn.Parameter(torch.zeros(C_in).view(1,-1,1,1))self.momentum0.9self.register_buffer(running_mean,torch.zeros(C_in).view(1,-1,1,1))self.register_buffer(running_var,torch.zeros(C_in).view(1,-1,1,1))self.eps1e-9def forward(self,x):if self.training:N,C,H,Wx.shapemeanx.mean(dim[0,2,3],keepdimTrue)varx.var(dim[0,2,3],keepdimTrue,unbiasedFalse)x(x-mean)/torch.sqrt(varself.eps)self.running_meanself.momentum*self.running_mean(1-self.momentum)*meanself.running_varself.momentum*self.running_var(1-self.momentum)*varelse:x(x-self.running_mean)/torch.sqrt(self.running_varself.eps)return xif __name____main__:inputtorch.rand(10,3,5,5)modelBN(3)resmodel(input)print(cool)
