How to update weights in backpropagation algorithm (a numerical example)?

Thu, 11 Apr 2019 17:02:04 +0000

Assume we have the following neural network and all activation functions are $f(z)=z$. If the weights are initialized with the values you see in table below, what will be new updated weights after one step if learning rate, $\alpha = 0.05$?

Assume the input values are [$i_1$,$i_2$] = [2,3] and target value $out = 1$.

Hint:
$w_{new} = w_{old} - \alpha \frac{\partial E}{\partial w}$

$E_{\text {total}}=\sum \frac{1}{2}(\text {target}-\text {output})^{2}$

Updating weights in backpropagation algorithm
Weights	Initialization	New weights after one step
$w1$	0.11	?
$w2$	0.21	?
$w3$	0.12	?
$w4$	0.08	?
$w5$	0.14	?
$w6$	0.15	?

https://i.imgur.com/v0RMeOQ.png

How to update weights using gradient decent algorithm?

Thu, 28 Mar 2019 17:17:39 +0000

For the below neural network, imagine we are going to use the backpropagation algorithm to update weights. If the Bias (b) in this problem is always 0 (ignore bias when you solve the problem), and we have a dataset with only one record of $x=2$ and the target value of $y=5$ as you can see in the following table, and activation function is defined as $f(z) = z$

feature (x)	Target (y)
2	5

1) Define the cost function, $J(w)$, based on the error in backpropagation algorithm: $J(w) = E = \frac{1}{2}(predicted - target)^2$, and draw it

2) Initialize the weight by $w=3$, and calculate the error

3) Calculate updated weights using the gradient decent algorithm after three updates if we have the following values for learning rate ($\alpha$)

$\alpha$ = 1
$\alpha$ = 0.1
$\alpha$ = 0.5

Hint: $w_{new} = w_{old} - \alpha \frac{\partial E}{\partial w}$

https://i.imgur.com/uohFS6l.png

Ask Ghassem - Recent questions tagged gradient-decent

How to update weights in backpropagation algorithm (a numerical example)?

How to update weights using gradient decent algorithm?