公式
<span class="equation-text" contenteditable="false" data-index="0" data-equation="V_{d_{w}} = \beta_1*V_{d_{w}}+(1-\beta_1)dW"><span></span><span></span></span>
<span class="equation-text" contenteditable="false" data-index="0" data-equation="\beta_1"><span></span><span></span></span>一般为0.9
<span class="equation-text" contenteditable="false" data-index="0" data-equation="V_{d_{b}} = \beta_1*V_{d_{b}}+(1-\beta_1)db"><span></span><span></span></span>
<span class="equation-text" contenteditable="false" data-index="0" data-equation="S_{d_{w}} = \beta_2*S_{d_{w}}+(1-\beta_2)dW^2"><span></span><span></span></span>
<span class="equation-text" contenteditable="false" data-index="0" data-equation="\beta_2"><span></span><span></span></span>一般为0.999
<span class="equation-text" contenteditable="false" data-index="0" data-equation="S_{d_{b}} = \beta*S_{d_{b}}+(1-\beta)db^2"><span></span><span></span></span>
偏差修正
<span class="equation-text" contenteditable="false" data-index="0" data-equation="V_{dw}^{correct} = \frac{V_{dw}}{1-\beta_1^t}"><span></span><span></span></span>
<span class="equation-text" contenteditable="false" data-index="0" data-equation="V_{db}^{correct} = \frac{V_{db}}{1-\beta_1^t}"><span></span><span></span></span>
<span class="equation-text" contenteditable="false" data-index="0" data-equation="S_{dw}^{correct} = \frac{S_{dw}}{1-\beta_2^t}"><span></span><span></span></span>
<span class="equation-text" contenteditable="false" data-index="0" data-equation="S_{db}^{correct} = \frac{S_{db}}{1-\beta_2^t}"><span></span><span></span></span>
梯度下降
<span class="equation-text" contenteditable="false" data-index="0" data-equation="W = W-\alpha*\frac{V_{d_w}^{correct}}{\sqrt{S_{dw}^{correct}}+\epsilon}"><span></span><span></span></span>
<span class="equation-text" data-index="0" data-equation="b = b-\alpha*\frac{V_{d_b}^{correct}}{\sqrt{S_{db}^{correct}}+\epsilon}" contenteditable="false"><span></span><span></span></span>
