Add exploratory data analysis, more data preprocessing and features, and more models

[schance995]

Here's @0mWh's and my solution so far. We plan to attempt all 3 UnitaryHack challenges. There are several changes, we look forward to any questions and feedback.

Notebooks

• QRNG_ Classification_Main_UnitaryHack windowed.ipynb for models with more training data from data/QRNG_ Classification_Main_UnitaryHack windowed_preprocessed_df_1717557318.csv.zst (generated in same notebook)
• QRNG_ Classification_Main_UnitaryHack.ipynb for more models, preprocessing, and exploratory data analysis.
• process_logical_reduction.ipynb for distribution analysis and statistical testing

Changes so far

• Use sliding window of 100 bits to generate more training data
  • Any subsequence of 100 bits is also generated by the same quantum computer
• Compare results against classical PRNGs
  • Impossible to classify classical PRNGs unless noise is added
• Exploratory data analysis
  • Check frequencies of bitstrings
    • Each label has a set of unique bitstrings, so it should be possible to them apart
  • Mann-Whitney U test to tell distributions apart
    • We can tell quantum computer 4 apart from the rest, but 1,2,3 are quite similar
  • Use PCA, tSNE, UMAP to determine clustering of bitstrings and features
    • The bitstrings themselves are not informative
    • Need some computed features
    • Features become more informative with larger bitstrings
• Add more features
  • Import errors need to be fixed by hand, only a few files
  • Steve Nang's NIST Library - scipy.zeros removed stevenang/randomness_testsuite#21
  • spkit entropy measures - blackmanharris import needs to be changed Nikeshbajaj/spkit#11
• Make features usable for ML
  • remove NaNs
  • remove features with identical values
  • mean and 0-1 normalize features
  • avoid test/train leakage
  • under/oversample
• Train/test models
  • add threads to speed up computation
  • more sklearn models
    • Naive Bayes, K-means

Best performance so far

We got 67% on one of our models, but we caution against further interpretation until we implement more robust model testing. A limitation is that we don't have a held-out test set for a fair comparison against other project submissions.

Next steps

• Model tuning
  • balanced class weights
  • k-fold cross validation
  • hyperparameter sweep
• Generalized quantum circuit
  • Qiskit Quantum Volume Circuit

[0mWh]

currently sitting at 75-77% accuracy