Skip to content

Loss explosion issue in ConvnLSTMnArchitecture #59

@frMitola

Description

@frMitola

Hello, I was following the tutorials and using "clf_cl_n" (ConvnLSTMnArchitecture) of the model zoo.
I noticed that while using this model, sometimes the loss just explode suddenly to very high numbers.
I analyzed the code of ConvnLSTMnArchitecture in tsgm/models/architectures/zoo.py and I noticed that LSTM layer has as activation function "relu", while usually it requests a saturating non-linearity, as it is a recurrent layer (otherwise with linear activation of the ReLU it can explode!).
I modified locally the implementation, simply deleting the "relu" in LSTM and seems to work fine after some trials!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions