Add regularisation to all weights other than bias.
Add regularisation to all weights other than bias.