1.23.52

Welcome back to a new semester and a new country! After about two months of silence and vacation, it's a new starting point for academic research and life. In the future, the DeepMuon will be more powerful as well as its publishments. Let's see what was updated at the beginning of the semester:

Issue Solved:

#6 : Large dataset acceleration template enabled.
#8 : Better double precision training architecture.

New Features:

• Large dataset multithread loading module enabled, accelerating data loading speed.
• Improved code quality, making it easier to read and modify.
• Training data plotting function enabled, automatically plots all data records during the training, making evaluating and analyzing models easier.
• More rich information in Logging files makes reproducing models easier and simpler.

Future Work:

• #7 Complete tutorials
• #9 Large model acceleration plan
• Interdisciplinary deep learning large model
• Empower the research of next-generation AI+Science

1.23.32 edition

Issue solved

#5 : New website of DeepMuon built

New features:

• Neural network hyperparameter searching (NNHS)
• More flexible configuration system
• More accurate and detailed tutorials

Future works:

• Model Explanation Visualization

1.23.23 edtion

Issues solved:

#4 : Support Video Swin-Transformer
#3 : Support discriminative tasks, including LSTM-type and normal type
#2 : TRIDENT optimization task completed

Some Features:

• Parallel computing algorithms:
  • Data Distributed Parallel
  • Fully Sharded Data Parallel
• Tasks available:
  • Regression tasks, mainly focused on the loss optimization
  • Discrimination tasks, mainly focused on the metrics improvement
• Optimize configuration:
  • Gradient accumulation
  • Gradient clip
  • Mixed precision training
• Model interpretation:
  • Attribution analysis:
    • Guided GradCAM, mainly for CNNs
    • Integrated Gradient
    • Layer Conductance
    • Neuron Conductance
  • Neuron flow:
    • Trace data flow within the model
• Customizable:
  • Model
  • Loss
  • Dataset
  • Evaluation metric
  • Interpreter
  • Training configuration
  • Anything you want

Future:

• NNI auto ML
• Visualization of Neuron flow
• Visualization of attribution
• Segmentation tasks
• Site of DeepMuon