Skip to content

Wrong derivative of leaky-relu activation #15

@sjh11556

Description

@sjh11556

According to mod_activation.F90 file, It seems like derivative function of leaky_relu is same as RELU.

  pure function leaky_relu_prime(x, alpha) result(res)
    ! First derivative of the REctified Linear Unit (RELU) activation function.
    real(rk), intent(in) :: x(:)
    real(rk), intent(in) :: alpha
    real(rk) :: res(size(x))
    where (0.3 * x > 0)
      res = 1
    elsewhere
      res = 0
    end where
  end function leaky_relu_prime

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions