diff --git a/machine-learning-das/notebooks/8-SetTransformer-PointCloud.ipynb b/machine-learning-das/notebooks/8-SetTransformer-PointCloud.ipynb
index 8be7be6..22b7347 100644
--- a/machine-learning-das/notebooks/8-SetTransformer-PointCloud.ipynb
+++ b/machine-learning-das/notebooks/8-SetTransformer-PointCloud.ipynb
@@ -1,730 +1,749 @@
{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {
- "colab_type": "text",
- "id": "view-in-github"
- },
- "source": [
- "![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\")
"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "BsonEqBekjyy"
- },
- "source": [
- "# Jet Tagging with Permutation Invariance\n",
- "\n",
- "In this notebook we will see an implementation of the Transformer architecture for sets applied to the jet tagging task. For *sets* it is meant here a point cloud, i.e. a set of nodes without edges. We will instead use Multi-Head Attention to learn which nodes (or particles) have strong pair-wise interaction.\n",
- "\n",
- "The architecture was introduced by [J. Lee at al. (ICML 2019)](https://arxiv.org/abs/1810.00825) -- specifically designed to model interactions among elements in the input set without pre-defined edges. The model consists of an encoder and a decoder, both of which rely on attention mechanisms, as in the original Transformer implementation [by Vaswani](https://arxiv.org/abs/1706.03762). The main difference is that positional encoding is removed plus some other low level adaptions.\n",
- "\n",
- "We will use tensorflow for this implementation.\n",
- "\n",
- "Before you start, choose GPU as a hardware accelerator for this notebook. To do this first go to Edit -> Notebook Settings -> Choose GPU as a hardware accelerator."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "A7OS3w5WRSCj"
- },
- "outputs": [],
- "source": [
- "import tensorflow as tf\n",
- "from tensorflow import keras\n",
- "from tensorflow.keras import layers\n",
- "import h5py\n",
- "import numpy as np\n",
- "\n",
- "#checking if we have GPUs\n",
- "print(\"Num GPUs Available: \", len(tf.config.list_physical_devices('GPU')))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "qwekVVRzneqU"
- },
- "source": [
- "## Dataset exploration"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "BHQGiyC4Pr4R"
- },
- "outputs": [],
- "source": [
- "! curl https://cernbox.cern.ch/s/6Ec5pGFEpFWeH6S/download -o Data-MLtutorial.tar.gz\n",
- "! tar -xvzf Data-MLtutorial.tar.gz \n",
- "! ls Data-MLtutorial/JetDataset/\n",
- "! rm Data-MLtutorial.tar.gz "
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "J9ZLcoKpPteG"
- },
- "outputs": [],
- "source": [
- "# let's open the file\n",
- "data_dir = 'Data-MLtutorial/JetDataset/'\n",
- "fileIN = data_dir+'jetImage_7_100p_30000_40000.h5'\n",
- "f = h5py.File(fileIN)\n",
- "# and see what it contains\n",
- "print(list(f.keys()))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "Ktx1VjNoOu4c"
- },
- "source": [
- "* 'jetImage' ,' jetImageECAL' and 'jetImageHCAL' contains the image representation of the jets . We will not use them today but build our point cloud from the other information.\n",
- "* 'jetConstituentList' is the list of particles cointained in the jet. For each particle, a list of relevant quantities is stored. This is the dataset we will consider in this notebook.\n",
- "* 'particleFeatureNames' is the list of the names corresponding to the quantities contained in 'jetConstituentList'\n",
- "* 'jets' is the list of jets with the high-level jet features stored. We will only use jet ID from it, indecies [-6:-1]\n",
- "* 'jetFeatureNames' is the list of the names corresponding to the quantities contained in 'jets'. These quantities are build using physics knowledge and correspond to high-level infromation and features per graph (as opposed to per node)\n",
- "\n",
- "The first 100 highest transverse momentum $p_T$ particles are considered for each jet.\n",
- "\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "Re7oXWWmPxz9"
- },
- "outputs": [],
- "source": [
- "target_onehot = np.array([])\n",
- "jetList = np.array([])\n",
- "jetImages = np.array([])\n",
- "features_names = dict()\n",
- "datafiles = ['jetImage_7_100p_0_10000.h5',\n",
- " 'jetImage_7_100p_10000_20000.h5',\n",
- " 'jetImage_7_100p_30000_40000.h5',\n",
- " 'jetImage_7_100p_40000_50000.h5',\n",
- " 'jetImage_7_100p_50000_60000.h5'\n",
- " ]\n",
- "for i_f,fileIN in enumerate(datafiles):\n",
- " print(\"Appending %s\" %fileIN)\n",
- " f = h5py.File(data_dir + fileIN)\n",
- " jetList_file = np.array(f.get(\"jetConstituentList\"))\n",
- " target_file = np.array(f.get('jets')[0:,-6:-1])\n",
- " jetImages_file = np.array(f.get('jetImage'))\n",
- " jetList = np.concatenate([jetList, jetList_file], axis=0) if jetList.size else jetList_file\n",
- " target_onehot = np.concatenate([target_onehot, target_file], axis=0) if target_onehot.size else target_file\n",
- " jetImages = np.concatenate([jetImages, jetImages_file], axis=0) if jetImages.size else jetImages_file\n",
- " del jetList_file, target_file, jetImages_file\n",
- " #save particles/nodes features names and their indecies in a dictionary\n",
- " if i_f==0:\n",
- " for feat_idx,feat_name in enumerate(list(f['particleFeatureNames'])[:-1]):\n",
- " features_names[feat_name.decode(\"utf-8\").replace('j1_','')] = feat_idx\n",
- " f.close()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "7kQnL9vkP4rK"
- },
- "source": [
- "The ground truth is incorporated in the ['j_g', 'j_q', 'j_w', 'j_z', 'j_t] vector of boolean, taking the form\n",
- "* [1, 0, 0, 0, 0] for gluons\n",
- "* [0, 1, 0, 0, 0] for quarks\n",
- "* [0, 0, 1, 0, 0] for W\n",
- "* [0, 0, 0, 1, 0] for Z \n",
- "* [0, 0, 0, 0, 1] for top quarks\n",
- "\n",
- "This is what is called 'one-hot' encoding of a descrete label (typical of ground truth for classification problems). These labels are the 'target' for our classification tasks. Let's convert it back to single-column encoding :\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "84NSj2W7P477"
- },
- "outputs": [],
- "source": [
- "print(\"Labels for the first five entries in the dataset, one-hot encoded:\")\n",
- "for i in range(5):\n",
- " print(target_onehot[i])\n",
- "print(target_onehot.shape)\n",
- "target = np.argmax(target_onehot, axis=1)\n",
- "print(target.shape)\n",
- "print(\"Labels for the first five entries in the dataset, single column encoded:\")\n",
- "for i in range(0,5):\n",
- " print(target[i])"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "mqsd_aP__RIi"
- },
- "source": [
- "Now our lables correspond to :\n",
- "* 0 for gluons\n",
- "* 1 for quarks\n",
- "* 2 for W\n",
- "* 3 for Z \n",
- "* 4 for top quarks\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "hyP15oxhP5ek"
- },
- "outputs": [],
- "source": [
- "num_classes = len(np.unique(target))\n",
- "label_names= [\"gluon\", \"quark\", \"W\", \"Z\", \"top\"]"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "Ik-6OX0LMJW7"
- },
- "source": [
- "Now let's inspect our data. Each jet is a point cloud/graph with 100 particles/nodes, each of which has 16 features. We have a double-index dataset: (jet index, particle index). The list is cut at 100 constituents per jet. If less constituents are present in the jet/point cloud, the dataset is completed filling it with 0s (zero padding). Note : zero-padding is not using during the training, it is only used to store the ragged dataset.\n",
- "\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "YfHRopq0P8tW"
- },
- "outputs": [],
- "source": [
- "print('Jets shape : ',jetList.shape)\n",
- "print('Target/Labels shape : ',target.shape)\n",
- "print('Particles/Nodes features : ',list(features_names.keys()))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "VBYwH4t8MhHm"
- },
- "source": [
- "We are not interested in all features for now. For now we will only consider the same node features as were considered in the ParticleNet paper: "
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "QWtB3vTWP_QY"
- },
- "outputs": [],
- "source": [
- "features_to_consider = 'etarel,phirel,pt,e,ptrel,erel,deltaR'.split(',')\n",
- "features_idx = [features_names[name] for name in features_to_consider]\n",
- "jetList = jetList[:,:,features_idx]\n",
- "print(jetList.shape)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "-M8uvPR4mfI7"
- },
- "source": [
- "Let's define basics hyperparamters:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "2mHCuVm6ZJaY"
- },
- "outputs": [],
- "source": [
- "batch_size=128\n",
- "learning_rate=0.0001\n",
- "epochs=20"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "unpVhZNfmotI"
- },
- "source": [
- "In the original paper, multi-head attention is also applied in the decoder step to obtain a smarter pooling operation. For this excercise we will simplify the model and use instead a `Lambda` layer to apply a custom pooling function to the input tensor. In this case, the `Lambda` layer is being used to sum over the first dimension, i.e. over the elements in the output set of the previous layer, which has shape `(batch_size, n_elements, features)`. By summing over the first dimension (`axis=1`), we obtain a tensor of shape `(batch_size, features)` that represents an aggregation of each feature over the elements in the set.\n",
- "\n",
- "Here is the full model:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "O7rzMn7wRcrP"
- },
- "outputs": [],
- "source": [
- "inputs = keras.Input(shape=(100,7), name='input')\n",
- "x = layers.TimeDistributed(layers.Dense(64))(inputs)\n",
- "x = layers.TimeDistributed(layers.LeakyReLU())(x)\n",
- "x = layers.TimeDistributed(layers.Dense(64))(x)\n",
- "x = layers.TimeDistributed(layers.LeakyReLU())(x)\n",
- "x = layers.TimeDistributed(layers.Dense(64))(x)\n",
- "x = layers.TimeDistributed(layers.LeakyReLU())(x)\n",
- "x = layers.TimeDistributed(layers.Dense(64))(x)\n",
- "x = layers.TimeDistributed(layers.LeakyReLU())(x)\n",
- "x = layers.Lambda(lambda y: tf.reduce_sum(y, axis=1))(x)\n",
- "x = layers.BatchNormalization()(x)\n",
- "x = layers.Dense(64)(x)\n",
- "x = layers.LeakyReLU()(x)\n",
- "x = layers.Dense(64)(x)\n",
- "x = layers.LeakyReLU()(x)\n",
- "x = layers.Dense(16)(x)\n",
- "x = layers.LeakyReLU()(x)\n",
- "output = layers.Dense(5, dtype='float32')(x)\n",
- "model = keras.models.Model(inputs=inputs, outputs=output)\n",
- "model.summary()"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "G8NI-_bYdSAq"
- },
- "outputs": [],
- "source": [
- "model.compile(\n",
- " loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
- " optimizer=keras.optimizers.Adam(learning_rate=learning_rate),\n",
- " metrics=[keras.metrics.SparseCategoricalAccuracy()],\n",
- ")"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "DhAKhgMMcrwa"
- },
- "outputs": [],
- "source": [
- "from sklearn.model_selection import train_test_split\n",
- "X_train, X_val, y_train, y_val, y_train_onehot, y_val_onehot = train_test_split(jetList, target, target_onehot, test_size=0.1, shuffle=True)\n",
- "print(X_train.shape, X_val.shape, y_train.shape, y_val.shape)\n",
- "del jetList, target, target_onehot"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "lz7rfyeCdNF0"
- },
- "outputs": [],
- "source": [
- "history = model.fit(X_train, y_train, batch_size=batch_size, epochs=epochs, validation_split=0.1)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "Egcr8vMhp-2v"
- },
- "source": [
- "We can now plot the validation and training loss evolution over the epochs:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "sjTOMuzAqGEr"
- },
- "outputs": [],
- "source": [
- "import matplotlib.pyplot as plt\n",
- "fig,axes = plt.subplots(2)\n",
- "\n",
- "axes[0].plot(history.history[\"sparse_categorical_accuracy\"])\n",
- "axes[0].plot(history.history[\"val_sparse_categorical_accuracy\"])\n",
- "axes[0].set_title(\"Accuracy\")\n",
- "axes[0].legend([\"Training\", \"Validation\"])\n",
- "\n",
- "axes[1].plot(history.history[\"loss\"])\n",
- "axes[1].plot(history.history[\"val_loss\"])\n",
- "axes[1].legend([\"Training\", \"Validation\"])\n",
- "axes[1].set_title(\"Loss\")\n",
- "\n",
- "fig.show()\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "CwrPPStDrS4J"
- },
- "source": [
- "Now we finally evaluate the performance by plotting the ROC curves for the different classes:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "JKM0yYFfecJh"
- },
- "outputs": [],
- "source": [
- "import pandas as pd\n",
- "import matplotlib.pyplot as plt\n",
- "%matplotlib inline\n",
- "from sklearn.metrics import roc_curve, auc\n",
- "predict_val = tf.nn.softmax(model.predict(X_val))\n",
- "df = pd.DataFrame()\n",
- "fpr = {}\n",
- "tpr = {}\n",
- "auc1 = {}\n",
- "\n",
- "plt.figure()\n",
- "for i, label in enumerate(label_names):\n",
- "\n",
- " df[label] = y_val_onehot[:,i]\n",
- " df[label + '_pred'] = predict_val[:,i]\n",
- "\n",
- " fpr[label], tpr[label], threshold = roc_curve(df[label],df[label+'_pred'])\n",
- "\n",
- " auc1[label] = auc(fpr[label], tpr[label])\n",
- "\n",
- " plt.plot(tpr[label],fpr[label],label='%s tagger, auc = %.1f%%'%(label,auc1[label]*100.))\n",
- "plt.semilogy()\n",
- "plt.xlabel(\"sig. efficiency\")\n",
- "plt.ylabel(\"bkg. mistag rate\")\n",
- "plt.ylim(0.000001,1)\n",
- "plt.grid(True)\n",
- "plt.legend(loc='lower right')\n",
- "plt.show()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "IzxPDanrrYZB"
- },
- "source": [
- "As you can see the performance are not as good for other models we have trained on the same dataset. As mentioned at the beginning of the notebook training a transformer might be tricky. You can try the optional excercise below to improve the performance and surpass the other models."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Multi Head Attention recap\n",
- "\n",
- "Assume we have $n$ query vectors (corresponding to the $n$ elements in the set) each with dimension $d_q : Q \\in \\mathbb{R}^{n\\times d_q}$. In the jet tagging dataset $n=100$ and $d_q=7$.\n",
- "\n",
- "An attention function $\\mathrm{Att}(Q,K,V)$ is a function that maps queries $Q$ to outputs using $n_v$ key-value pairs $K \\in \\mathbb{R}^{n_v \\times d_q}, V \\in \\mathbb{R}^{n_v\\times d_v}$:\n",
- "\n",
- "$$\n",
- "\\mathrm{Att}(Q,K,V;\\omega) = \\omega(QK^{T})V.\n",
- "$$\n",
- "\n",
- "The pairwise dot product $QT^\\mathrm{T} \\in \\mathbb{R}^{n\\times n_v}$ measures how similar each pair of query and key vectors is, with weights computed with an activation function $\\omega$. The output $\\omega(QK^{T})V$ is a weighted sum of $V$ where a value gets more weight if its corresponding key has larger dot product with the query.\n",
- "\n",
- "Instead of computing a single attention function, the **multi-head attention** method first projects $Q, K, V$ onto $h$ different $d^M_q,d^M_q,d^M_v$-dimensional vectors, respectively. An attention function $\\mathrm{Att}(\\cdot; \\omega_j)$ is applied to each of these $h$ projections. The output is a linear transformation of the concatenation of all attention outputs:\n",
- "\n",
- "$$\n",
- "\\mathrm{Multihead}(Q, K, V ; \\lambda, \\omega) = \\mathrm{concat}(O_1,..., O_h)W^O\n",
- "$$\n",
- "\n",
- "$$\n",
- "O_j = \\mathrm{Att}(QW^Q_j, KW^K_j, VW^V_j ; \\omega_j )\n",
- "$$\n",
- "\n",
- "In other words, the model tells you what is the score of a particle in the set knowing its interaction with the other particles in the set given all features but in a way that the features are attended separately.\n",
- "\n",
- "Note that $\\mathrm{Multihead}(\\cdot, \\cdot, \\cdot; \\lambda)$ has learnable parameters $\\lambda =$ {$W^Q_j, W^K_j, W^V_j$}$_{j=1,...,h}$ where $W^Q_j, W^K_j \\in \\mathbb{R}^{d_q\\times d^M_q}, W^V_j \\in \\mathbb{R}^{d_v\\times d^M_v}, W^O \\in \\mathbb{R}^{hd^M_v\\times d}$. A typical choice for the dimension hyperparameters is $d^M_q = d_q /h, d^M_v = d_v /h, d = d_q$. For the Set Transformer we set $d_q = d_v = d$ and $d^M_q = d^M_v = d/h$. A scaled softmax $\\omega_j (\\cdot) = \\mathrm{softmax}(\\cdot/\\sqrt{d})$ is used.\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Building the Set Transformer\n",
- "\n",
- "We will implement a simplified version of the [original Set Transformer architecture](https://arxiv.org/abs/1810.00825). The reason is because Transformers are typically computationally and data hungry. As an optional excercise at the end of the notebook you can try to implement the full model and test it on a simpler problem like the MNIST dataset classification (or on a larger jet class dataset).\n",
- "\n",
- "The architecture is based on the block called `MAB` (= Multihead Attention Block) which implements the following:\n",
- "\n",
- "$$\n",
- "\\mathrm{MAB}(X, Y) = \\mathrm{LayerNorm}(H + \\mathrm{rFF}(H))\n",
- "$$\n",
- "\n",
- "$$\n",
- "H = \\mathrm{LayerNorm}(X + \\mathrm{Multihead}(X, X, X ; ω))\n",
- "$$\n",
- "\n",
- "where $X \\in \\mathbb{R}^{n\\times d}$ is the input set and $\\mathrm{rFF}$ is any feedforward layer. Since $Q=K=V=X$, the MAB takes a set and performs *self-attention* between the elements in the set, resulting in a set of equal size. Since the output of MAB contains information about pairwise interactions among the elements in the input set $X$, we can stack multiple MABs to encode higher order interactions. This stack is the *encoder* part of the transformer. \n",
- "\n",
- "The `LayerNorm` normalizes the activations of a layer across the last dimension (feature dimension) of the input tensor. Specifically, it centers and scales each feature dimension independently by subtracting the mean and dividing by the standard deviation, which are computed over the corresponding feature dimension of the input tensor. As for `BatchNormalization` it has learnable $\\gamma$ (scaling) and $\\beta$ (shifting) parameters. The difference with respect to `BatchNormalization` is that the normalization is performed indipendently per each instance in the batch. `LayerNorm` leads to improved stability when you expect instances of different sizes (or different zero padding degree as in the jet tagging case)."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "RORSIHwVRPx4"
- },
- "outputs": [],
- "source": [
- "class SABTransformerBlock(tf.keras.layers.Layer):\n",
- " def __init__(self, num_heads, hidden_units, mlp_hidden_units=128, dropout_rate=0.1, **kwargs):\n",
- " super(SABTransformerBlock, self).__init__(**kwargs)\n",
- " self.num_heads = num_heads\n",
- " self.hidden_units = hidden_units\n",
- " self.mlp_hidden_units = mlp_hidden_units\n",
- " self.dropout_rate = dropout_rate\n",
- "\n",
- " def build(self, input_shape):\n",
- " self.attention = tf.keras.layers.MultiHeadAttention(num_heads=self.num_heads, \n",
- " key_dim=self.hidden_units//self.num_heads)\n",
- " self.feedforward = tf.keras.Sequential([\n",
- " layers.Dense(units=self.mlp_hidden_units, activation=\"relu\"),\n",
- " # Dropout(rate=self.dropout_rate),\n",
- " layers.Dense(units=input_shape[-1])\n",
- " ])\n",
- " self.layer_norm1 = layers.LayerNormalization(epsilon=1e-6)\n",
- " self.layer_norm2 = layers.LayerNormalization(epsilon=1e-6)\n",
- " self.dropout1 = layers.Dropout(rate=self.dropout_rate)\n",
- " self.dropout2 = layers.Dropout(rate=self.dropout_rate)\n",
- " super(SABTransformerBlock, self).build(input_shape)\n",
- " \n",
- " def call(self, inputs, mask=None):\n",
- " attention_output = self.attention(inputs, inputs, attention_mask=mask)[0]\n",
- " # attention_output = self.dropout1(attention_output)\n",
- " attention_output = self.layer_norm1(inputs + attention_output)\n",
- " feedforward_output = self.feedforward(attention_output)\n",
- " # feedforward_output = self.dropout2(feedforward_output)\n",
- " block_output = self.layer_norm2(attention_output + feedforward_output)\n",
- " return block_output\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "inputs = keras.Input(shape=(100,7), name='input')\n",
- "x = layers.TimeDistributed(layers.Dense(64))(inputs)\n",
- "x = SABTransformerBlock(num_heads=8, hidden_units=64)(x)\n",
- "x = SABTransformerBlock(num_heads=8, hidden_units=64)(x)\n",
- "x = SABTransformerBlock(num_heads=8, hidden_units=64)(x)\n",
- "x = layers.Lambda(lambda y: tf.reduce_sum(y, axis=1))(x)\n",
- "x = layers.BatchNormalization()(x)\n",
- "x = layers.Dense(64)(x)\n",
- "x = layers.LeakyReLU()(x)\n",
- "x = layers.Dense(64)(x)\n",
- "x = layers.LeakyReLU()(x)\n",
- "x = layers.Dense(16)(x)\n",
- "x = layers.LeakyReLU()(x)\n",
- "output = layers.Dense(5, dtype='float32')(x)\n",
- "model_st = keras.models.Model(inputs=inputs, outputs=output)\n",
- "model.summary()"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "model.compile(\n",
- " loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
- " optimizer=keras.optimizers.Adam(learning_rate=learning_rate),\n",
- " metrics=[keras.metrics.SparseCategoricalAccuracy()],\n",
- ")"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "history = model.fit(X_train, y_train, batch_size=batch_size, epochs=epochs, validation_split=0.1)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import matplotlib.pyplot as plt\n",
- "fig,axes = plt.subplots(2)\n",
- "\n",
- "axes[0].plot(history.history[\"sparse_categorical_accuracy\"])\n",
- "axes[0].plot(history.history[\"val_sparse_categorical_accuracy\"])\n",
- "axes[0].set_title(\"Accuracy\")\n",
- "axes[0].legend([\"Training\", \"Validation\"])\n",
- "\n",
- "axes[1].plot(history.history[\"loss\"])\n",
- "axes[1].plot(history.history[\"val_loss\"])\n",
- "axes[1].legend([\"Training\", \"Validation\"])\n",
- "axes[1].set_title(\"Loss\")\n",
- "\n",
- "fig.show()\n",
- "\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Now we finally evaluate the performance by plotting the ROC curves for the different classes:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import pandas as pd\n",
- "import matplotlib.pyplot as plt\n",
- "%matplotlib inline\n",
- "from sklearn.metrics import roc_curve, auc\n",
- "predict_val = tf.nn.softmax(model.predict(X_val))\n",
- "df = pd.DataFrame()\n",
- "fpr = {}\n",
- "tpr = {}\n",
- "auc1 = {}\n",
- "\n",
- "plt.figure()\n",
- "for i, label in enumerate(label_names):\n",
- "\n",
- " df[label] = y_val_onehot[:,i]\n",
- " df[label + '_pred'] = predict_val[:,i]\n",
- "\n",
- " fpr[label], tpr[label], threshold = roc_curve(df[label],df[label+'_pred'])\n",
- "\n",
- " auc1[label] = auc(fpr[label], tpr[label])\n",
- "\n",
- " plt.plot(tpr[label],fpr[label],label='%s tagger, auc = %.1f%%'%(label,auc1[label]*100.))\n",
- "plt.semilogy()\n",
- "plt.xlabel(\"sig. efficiency\")\n",
- "plt.ylabel(\"bkg. mistag rate\")\n",
- "plt.ylim(0.000001,1)\n",
- "plt.grid(True)\n",
- "plt.legend(loc='lower right')\n",
- "plt.show()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Optional Excercise\n",
- "\n",
- "The original paper also use MH mechanism in the decoder step (while we used a simple sum over the latent space nodes). If you would like to try it out the `Lambda` layer should be replaced with the `PoolingByMultiHeadAttention` block below.\n",
- "\n",
- "Consider also the fact that it might be hard to train a Transformer architecture of this kind over the rather small dataset used here. Check out [this other dataset](https://events.mcs.cmu.edu/us-cms-2023/) for increased statistics or [this notebook](https://github.com/DLii-Research/tf-settransformer/blob/master/examples/mnist_pointcloud.ipynb) for a simpler task.\n",
- "\n",
- "Below is the starting point for a smarter decoder:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "id": "9qHj1_Y7ZU-R"
- },
- "outputs": [],
- "source": [
- "class PoolingByMultiHeadAttention(tf.keras.layers.Layer):\n",
- " def __init__(self, num_heads, hidden_units, mlp_hidden_units=128, num_seeds=1, **kwargs):\n",
- " super(PoolingByMultiHeadAttention, self).__init__(**kwargs)\n",
- " self.num_heads = num_heads\n",
- " self.hidden_units = hidden_units\n",
- " self.mlp_hidden_units = mlp_hidden_units\n",
- " self.num_seeds = num_seeds\n",
- " \n",
- " def build(self, input_shape):\n",
- " \n",
- " self.attention = tf.keras.layers.MultiHeadAttention(num_heads=self.num_heads, \n",
- " key_dim=self.hidden_units)\n",
- " \n",
- " self.seed_vectors = self.add_weight(\n",
- " shape=(1, self.num_seeds, self.hidden_units),\n",
- " initializer=\"random_normal\",\n",
- " trainable=True,\n",
- " name=\"Seeds\")\n",
- "\n",
- " self.feedforward = tf.keras.Sequential([\n",
- " layers.Dense(units=self.mlp_hidden_units, activation=\"relu\"),\n",
- " layers.Dense(units=self.hidden_units)\n",
- " ])\n",
- " self.layer_norm1 = layers.LayerNormalization(epsilon=1e-6)\n",
- " self.layer_norm2 = layers.LayerNormalization(epsilon=1e-6)\n",
- " super(PoolingByMultiHeadAttention, self).build(input_shape)\n",
- "\n",
- " def call(self, inputs, training=None):\n",
- " a = tf.expand_dims(self.seed_vectors, axis=0)\n",
- " seeds = tf.tile(self.seed_vectors, [tf.shape(inputs)[0], 1, 1])\n",
- " attention_output = self.attention(seeds, inputs)[0]\n",
- " attention_output = self.layer_norm1(seeds + attention_output)\n",
- " feedforward_output = self.feedforward(attention_output)\n",
- " block_output = self.layer_norm2(attention_output + feedforward_output)\n",
- " return block_output"
- ]
- }
- ],
- "metadata": {
- "accelerator": "GPU",
- "colab": {
- "authorship_tag": "ABX9TyPn4xtio5MeIQMG/e23naQt",
- "include_colab_link": true,
- "provenance": []
- },
- "gpuClass": "standard",
- "kernelspec": {
- "display_name": "Python 3",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.16"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "colab_type": "text",
+ "id": "view-in-github"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "BsonEqBekjyy"
+ },
+ "source": [
+ "# Jet Tagging with Permutation Invariance\n",
+ "\n",
+ "In this notebook we will see an implementation of the Transformer architecture for sets applied to the jet tagging task. For *sets* it is meant here a point cloud, i.e. a set of nodes without edges. We will instead use Multi-Head Attention to learn which nodes (or particles) have strong pair-wise interaction.\n",
+ "\n",
+ "The architecture was introduced by [J. Lee at al. (ICML 2019)](https://arxiv.org/abs/1810.00825) -- specifically designed to model interactions among elements in the input set without pre-defined edges. The model consists of an encoder and a decoder, both of which rely on attention mechanisms, as in the original Transformer implementation [by Vaswani](https://arxiv.org/abs/1706.03762). The main difference is that positional encoding is removed plus some other low level adaptions.\n",
+ "\n",
+ "We will use tensorflow for this implementation.\n",
+ "\n",
+ "Before you start, choose GPU as a hardware accelerator for this notebook. To do this first go to Edit -> Notebook Settings -> Choose GPU as a hardware accelerator."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "A7OS3w5WRSCj"
+ },
+ "outputs": [],
+ "source": [
+ "import tensorflow as tf\n",
+ "from tensorflow import keras\n",
+ "from tensorflow.keras import layers\n",
+ "import h5py\n",
+ "import numpy as np\n",
+ "\n",
+ "# checking if we have GPUs\n",
+ "print(\"Num GPUs Available: \", len(tf.config.list_physical_devices(\"GPU\")))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "qwekVVRzneqU"
+ },
+ "source": [
+ "## Dataset exploration"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "BHQGiyC4Pr4R"
+ },
+ "outputs": [],
+ "source": [
+ "! curl https://cernbox.cern.ch/s/6Ec5pGFEpFWeH6S/download -o Data-MLtutorial.tar.gz\n",
+ "! tar -xvzf Data-MLtutorial.tar.gz\n",
+ "! ls Data-MLtutorial/JetDataset/\n",
+ "! rm Data-MLtutorial.tar.gz"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "J9ZLcoKpPteG"
+ },
+ "outputs": [],
+ "source": [
+ "# let's open the file\n",
+ "data_dir = \"Data-MLtutorial/JetDataset/\"\n",
+ "fileIN = data_dir + \"jetImage_7_100p_30000_40000.h5\"\n",
+ "f = h5py.File(fileIN)\n",
+ "# and see what it contains\n",
+ "print(list(f.keys()))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Ktx1VjNoOu4c"
+ },
+ "source": [
+ "* 'jetImage' ,' jetImageECAL' and 'jetImageHCAL' contains the image representation of the jets . We will not use them today but build our point cloud from the other information.\n",
+ "* 'jetConstituentList' is the list of particles cointained in the jet. For each particle, a list of relevant quantities is stored. This is the dataset we will consider in this notebook.\n",
+ "* 'particleFeatureNames' is the list of the names corresponding to the quantities contained in 'jetConstituentList'\n",
+ "* 'jets' is the list of jets with the high-level jet features stored. We will only use jet ID from it, indecies [-6:-1]\n",
+ "* 'jetFeatureNames' is the list of the names corresponding to the quantities contained in 'jets'. These quantities are build using physics knowledge and correspond to high-level infromation and features per graph (as opposed to per node)\n",
+ "\n",
+ "The first 100 highest transverse momentum $p_T$ particles are considered for each jet.\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Re7oXWWmPxz9"
+ },
+ "outputs": [],
+ "source": [
+ "target_onehot = np.array([])\n",
+ "jetList = np.array([])\n",
+ "jetImages = np.array([])\n",
+ "features_names = dict()\n",
+ "datafiles = [\n",
+ " \"jetImage_7_100p_0_10000.h5\",\n",
+ " \"jetImage_7_100p_10000_20000.h5\",\n",
+ " \"jetImage_7_100p_30000_40000.h5\",\n",
+ " \"jetImage_7_100p_40000_50000.h5\",\n",
+ " \"jetImage_7_100p_50000_60000.h5\",\n",
+ "]\n",
+ "for i_f, fileIN in enumerate(datafiles):\n",
+ " print(\"Appending %s\" % fileIN)\n",
+ " f = h5py.File(data_dir + fileIN)\n",
+ " jetList_file = np.array(f.get(\"jetConstituentList\"))\n",
+ " target_file = np.array(f.get(\"jets\")[0:, -6:-1])\n",
+ " jetImages_file = np.array(f.get(\"jetImage\"))\n",
+ " jetList = np.concatenate([jetList, jetList_file], axis=0) if jetList.size else jetList_file\n",
+ " target_onehot = (\n",
+ " np.concatenate([target_onehot, target_file], axis=0) if target_onehot.size else target_file\n",
+ " )\n",
+ " jetImages = (\n",
+ " np.concatenate([jetImages, jetImages_file], axis=0) if jetImages.size else jetImages_file\n",
+ " )\n",
+ " del jetList_file, target_file, jetImages_file\n",
+ " # save particles/nodes features names and their indecies in a dictionary\n",
+ " if i_f == 0:\n",
+ " for feat_idx, feat_name in enumerate(list(f[\"particleFeatureNames\"])[:-1]):\n",
+ " features_names[feat_name.decode(\"utf-8\").replace(\"j1_\", \"\")] = feat_idx\n",
+ " f.close()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "7kQnL9vkP4rK"
+ },
+ "source": [
+ "The ground truth is incorporated in the ['j_g', 'j_q', 'j_w', 'j_z', 'j_t] vector of boolean, taking the form\n",
+ "* [1, 0, 0, 0, 0] for gluons\n",
+ "* [0, 1, 0, 0, 0] for quarks\n",
+ "* [0, 0, 1, 0, 0] for W\n",
+ "* [0, 0, 0, 1, 0] for Z \n",
+ "* [0, 0, 0, 0, 1] for top quarks\n",
+ "\n",
+ "This is what is called 'one-hot' encoding of a descrete label (typical of ground truth for classification problems). These labels are the 'target' for our classification tasks. Let's convert it back to single-column encoding :\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "84NSj2W7P477"
+ },
+ "outputs": [],
+ "source": [
+ "print(\"Labels for the first five entries in the dataset, one-hot encoded:\")\n",
+ "for i in range(5):\n",
+ " print(target_onehot[i])\n",
+ "print(target_onehot.shape)\n",
+ "target = np.argmax(target_onehot, axis=1)\n",
+ "print(target.shape)\n",
+ "print(\"Labels for the first five entries in the dataset, single column encoded:\")\n",
+ "for i in range(0, 5):\n",
+ " print(target[i])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "mqsd_aP__RIi"
+ },
+ "source": [
+ "Now our lables correspond to :\n",
+ "* 0 for gluons\n",
+ "* 1 for quarks\n",
+ "* 2 for W\n",
+ "* 3 for Z \n",
+ "* 4 for top quarks\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "hyP15oxhP5ek"
+ },
+ "outputs": [],
+ "source": [
+ "num_classes = len(np.unique(target))\n",
+ "label_names = [\"gluon\", \"quark\", \"W\", \"Z\", \"top\"]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Ik-6OX0LMJW7"
+ },
+ "source": [
+ "Now let's inspect our data. Each jet is a point cloud/graph with 100 particles/nodes, each of which has 16 features. We have a double-index dataset: (jet index, particle index). The list is cut at 100 constituents per jet. If less constituents are present in the jet/point cloud, the dataset is completed filling it with 0s (zero padding). Note : zero-padding is not using during the training, it is only used to store the ragged dataset.\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "YfHRopq0P8tW"
+ },
+ "outputs": [],
+ "source": [
+ "print(\"Jets shape : \", jetList.shape)\n",
+ "print(\"Target/Labels shape : \", target.shape)\n",
+ "print(\"Particles/Nodes features : \", list(features_names.keys()))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "VBYwH4t8MhHm"
+ },
+ "source": [
+ "We are not interested in all features for now. For now we will only consider the same node features as were considered in the ParticleNet paper: "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "QWtB3vTWP_QY"
+ },
+ "outputs": [],
+ "source": [
+ "features_to_consider = \"etarel,phirel,pt,e,ptrel,erel,deltaR\".split(\",\")\n",
+ "features_idx = [features_names[name] for name in features_to_consider]\n",
+ "jetList = jetList[:, :, features_idx]\n",
+ "print(jetList.shape)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "-M8uvPR4mfI7"
+ },
+ "source": [
+ "Let's define basics hyperparamters:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "2mHCuVm6ZJaY"
+ },
+ "outputs": [],
+ "source": [
+ "batch_size = 128\n",
+ "learning_rate = 0.0001\n",
+ "epochs = 20"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "unpVhZNfmotI"
+ },
+ "source": [
+ "In the original paper, multi-head attention is also applied in the decoder step to obtain a smarter pooling operation. For this excercise we will simplify the model and use instead a `Lambda` layer to apply a custom pooling function to the input tensor. In this case, the `Lambda` layer is being used to sum over the first dimension, i.e. over the elements in the output set of the previous layer, which has shape `(batch_size, n_elements, features)`. By summing over the first dimension (`axis=1`), we obtain a tensor of shape `(batch_size, features)` that represents an aggregation of each feature over the elements in the set.\n",
+ "\n",
+ "Here is the full model:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "O7rzMn7wRcrP"
+ },
+ "outputs": [],
+ "source": [
+ "inputs = keras.Input(shape=(100, 7), name=\"input\")\n",
+ "x = layers.TimeDistributed(layers.Dense(64))(inputs)\n",
+ "x = layers.TimeDistributed(layers.LeakyReLU())(x)\n",
+ "x = layers.TimeDistributed(layers.Dense(64))(x)\n",
+ "x = layers.TimeDistributed(layers.LeakyReLU())(x)\n",
+ "x = layers.TimeDistributed(layers.Dense(64))(x)\n",
+ "x = layers.TimeDistributed(layers.LeakyReLU())(x)\n",
+ "x = layers.TimeDistributed(layers.Dense(64))(x)\n",
+ "x = layers.TimeDistributed(layers.LeakyReLU())(x)\n",
+ "x = layers.Lambda(lambda y: tf.reduce_sum(y, axis=1))(x)\n",
+ "x = layers.BatchNormalization()(x)\n",
+ "x = layers.Dense(64)(x)\n",
+ "x = layers.LeakyReLU()(x)\n",
+ "x = layers.Dense(64)(x)\n",
+ "x = layers.LeakyReLU()(x)\n",
+ "x = layers.Dense(16)(x)\n",
+ "x = layers.LeakyReLU()(x)\n",
+ "output = layers.Dense(5, dtype=\"float32\")(x)\n",
+ "model = keras.models.Model(inputs=inputs, outputs=output)\n",
+ "model.summary()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "G8NI-_bYdSAq"
+ },
+ "outputs": [],
+ "source": [
+ "model.compile(\n",
+ " loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
+ " optimizer=keras.optimizers.Adam(learning_rate=learning_rate),\n",
+ " metrics=[keras.metrics.SparseCategoricalAccuracy()],\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "DhAKhgMMcrwa"
+ },
+ "outputs": [],
+ "source": [
+ "from sklearn.model_selection import train_test_split\n",
+ "\n",
+ "X_train, X_val, y_train, y_val, y_train_onehot, y_val_onehot = train_test_split(\n",
+ " jetList, target, target_onehot, test_size=0.1, shuffle=True\n",
+ ")\n",
+ "print(X_train.shape, X_val.shape, y_train.shape, y_val.shape)\n",
+ "del jetList, target, target_onehot"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "lz7rfyeCdNF0"
+ },
+ "outputs": [],
+ "source": [
+ "history = model.fit(X_train, y_train, batch_size=batch_size, epochs=epochs, validation_split=0.1)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Egcr8vMhp-2v"
+ },
+ "source": [
+ "We can now plot the validation and training loss evolution over the epochs:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "sjTOMuzAqGEr"
+ },
+ "outputs": [],
+ "source": [
+ "import matplotlib.pyplot as plt\n",
+ "\n",
+ "fig, axes = plt.subplots(2)\n",
+ "\n",
+ "axes[0].plot(history.history[\"sparse_categorical_accuracy\"])\n",
+ "axes[0].plot(history.history[\"val_sparse_categorical_accuracy\"])\n",
+ "axes[0].set_title(\"Accuracy\")\n",
+ "axes[0].legend([\"Training\", \"Validation\"])\n",
+ "\n",
+ "axes[1].plot(history.history[\"loss\"])\n",
+ "axes[1].plot(history.history[\"val_loss\"])\n",
+ "axes[1].legend([\"Training\", \"Validation\"])\n",
+ "axes[1].set_title(\"Loss\")\n",
+ "\n",
+ "fig.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "CwrPPStDrS4J"
+ },
+ "source": [
+ "Now we finally evaluate the performance by plotting the ROC curves for the different classes:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "JKM0yYFfecJh"
+ },
+ "outputs": [],
+ "source": [
+ "import pandas as pd\n",
+ "import matplotlib.pyplot as plt\n",
+ "\n",
+ "%matplotlib inline\n",
+ "from sklearn.metrics import roc_curve, auc\n",
+ "\n",
+ "predict_val = tf.nn.softmax(model.predict(X_val))\n",
+ "df = pd.DataFrame()\n",
+ "fpr = {}\n",
+ "tpr = {}\n",
+ "auc1 = {}\n",
+ "\n",
+ "plt.figure()\n",
+ "for i, label in enumerate(label_names):\n",
+ "\n",
+ " df[label] = y_val_onehot[:, i]\n",
+ " df[label + \"_pred\"] = predict_val[:, i]\n",
+ "\n",
+ " fpr[label], tpr[label], threshold = roc_curve(df[label], df[label + \"_pred\"])\n",
+ "\n",
+ " auc1[label] = auc(fpr[label], tpr[label])\n",
+ "\n",
+ " plt.plot(tpr[label], fpr[label], label=\"%s tagger, auc = %.1f%%\" % (label, auc1[label] * 100.0))\n",
+ "plt.semilogy()\n",
+ "plt.xlabel(\"sig. efficiency\")\n",
+ "plt.ylabel(\"bkg. mistag rate\")\n",
+ "plt.ylim(0.000001, 1)\n",
+ "plt.grid(True)\n",
+ "plt.legend(loc=\"lower right\")\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "IzxPDanrrYZB"
+ },
+ "source": [
+ "As you can see the performance are not as good for other models we have trained on the same dataset. As mentioned at the beginning of the notebook training a transformer might be tricky. You can try the optional excercise below to improve the performance and surpass the other models."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Multi Head Attention recap\n",
+ "\n",
+ "Assume we have $n$ query vectors (corresponding to the $n$ elements in the set) each with dimension $d_q : Q \\in \\mathbb{R}^{n\\times d_q}$. In the jet tagging dataset $n=100$ and $d_q=7$.\n",
+ "\n",
+ "An attention function $\\mathrm{Att}(Q,K,V)$ is a function that maps queries $Q$ to outputs using $n_v$ key-value pairs $K \\in \\mathbb{R}^{n_v \\times d_q}, V \\in \\mathbb{R}^{n_v\\times d_v}$:\n",
+ "\n",
+ "$$\n",
+ "\\mathrm{Att}(Q,K,V;\\omega) = \\omega(QK^{T})V.\n",
+ "$$\n",
+ "\n",
+ "The pairwise dot product $QT^\\mathrm{T} \\in \\mathbb{R}^{n\\times n_v}$ measures how similar each pair of query and key vectors is, with weights computed with an activation function $\\omega$. The output $\\omega(QK^{T})V$ is a weighted sum of $V$ where a value gets more weight if its corresponding key has larger dot product with the query.\n",
+ "\n",
+ "Instead of computing a single attention function, the **multi-head attention** method first projects $Q, K, V$ onto $h$ different $d^M_q,d^M_q,d^M_v$-dimensional vectors, respectively. An attention function $\\mathrm{Att}(\\cdot; \\omega_j)$ is applied to each of these $h$ projections. The output is a linear transformation of the concatenation of all attention outputs:\n",
+ "\n",
+ "$$\n",
+ "\\mathrm{Multihead}(Q, K, V ; \\lambda, \\omega) = \\mathrm{concat}(O_1,..., O_h)W^O\n",
+ "$$\n",
+ "\n",
+ "$$\n",
+ "O_j = \\mathrm{Att}(QW^Q_j, KW^K_j, VW^V_j ; \\omega_j )\n",
+ "$$\n",
+ "\n",
+ "In other words, the model tells you what is the score of a particle in the set knowing its interaction with the other particles in the set given all features but in a way that the features are attended separately.\n",
+ "\n",
+ "Note that $\\mathrm{Multihead}(\\cdot, \\cdot, \\cdot; \\lambda)$ has learnable parameters $\\lambda =$ {$W^Q_j, W^K_j, W^V_j$}$_{j=1,...,h}$ where $W^Q_j, W^K_j \\in \\mathbb{R}^{d_q\\times d^M_q}, W^V_j \\in \\mathbb{R}^{d_v\\times d^M_v}, W^O \\in \\mathbb{R}^{hd^M_v\\times d}$. A typical choice for the dimension hyperparameters is $d^M_q = d_q /h, d^M_v = d_v /h, d = d_q$. For the Set Transformer we set $d_q = d_v = d$ and $d^M_q = d^M_v = d/h$. A scaled softmax $\\omega_j (\\cdot) = \\mathrm{softmax}(\\cdot/\\sqrt{d})$ is used.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Building the Set Transformer\n",
+ "\n",
+ "We will implement a simplified version of the [original Set Transformer architecture](https://arxiv.org/abs/1810.00825). The reason is because Transformers are typically computationally and data hungry. As an optional excercise at the end of the notebook you can try to implement the full model and test it on a simpler problem like the MNIST dataset classification (or on a larger jet class dataset).\n",
+ "\n",
+ "The architecture is based on the block called `MAB` (= Multihead Attention Block) which implements the following:\n",
+ "\n",
+ "$$\n",
+ "\\mathrm{MAB}(X, Y) = \\mathrm{LayerNorm}(H + \\mathrm{rFF}(H))\n",
+ "$$\n",
+ "\n",
+ "$$\n",
+ "H = \\mathrm{LayerNorm}(X + \\mathrm{Multihead}(X, X, X ; ω))\n",
+ "$$\n",
+ "\n",
+ "where $X \\in \\mathbb{R}^{n\\times d}$ is the input set and $\\mathrm{rFF}$ is any feedforward layer. Since $Q=K=V=X$, the MAB takes a set and performs *self-attention* between the elements in the set, resulting in a set of equal size. Since the output of MAB contains information about pairwise interactions among the elements in the input set $X$, we can stack multiple MABs to encode higher order interactions. This stack is the *encoder* part of the transformer. \n",
+ "\n",
+ "The `LayerNorm` normalizes the activations of a layer across the last dimension (feature dimension) of the input tensor. Specifically, it centers and scales each feature dimension independently by subtracting the mean and dividing by the standard deviation, which are computed over the corresponding feature dimension of the input tensor. As for `BatchNormalization` it has learnable $\\gamma$ (scaling) and $\\beta$ (shifting) parameters. The difference with respect to `BatchNormalization` is that the normalization is performed indipendently per each instance in the batch. `LayerNorm` leads to improved stability when you expect instances of different sizes (or different zero padding degree as in the jet tagging case)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "RORSIHwVRPx4"
+ },
+ "outputs": [],
+ "source": [
+ "class SABTransformerBlock(tf.keras.layers.Layer):\n",
+ " def __init__(self, num_heads, hidden_units, mlp_hidden_units=128, dropout_rate=0.1, **kwargs):\n",
+ " super(SABTransformerBlock, self).__init__(**kwargs)\n",
+ " self.num_heads = num_heads\n",
+ " self.hidden_units = hidden_units\n",
+ " self.mlp_hidden_units = mlp_hidden_units\n",
+ " self.dropout_rate = dropout_rate\n",
+ "\n",
+ " def build(self, input_shape):\n",
+ " self.attention = tf.keras.layers.MultiHeadAttention(\n",
+ " num_heads=self.num_heads, key_dim=self.hidden_units // self.num_heads\n",
+ " )\n",
+ " self.feedforward = tf.keras.Sequential(\n",
+ " [\n",
+ " layers.Dense(units=self.mlp_hidden_units, activation=\"relu\"),\n",
+ " # Dropout(rate=self.dropout_rate),\n",
+ " layers.Dense(units=input_shape[-1]),\n",
+ " ]\n",
+ " )\n",
+ " self.layer_norm1 = layers.LayerNormalization(epsilon=1e-6)\n",
+ " self.layer_norm2 = layers.LayerNormalization(epsilon=1e-6)\n",
+ " self.dropout1 = layers.Dropout(rate=self.dropout_rate)\n",
+ " self.dropout2 = layers.Dropout(rate=self.dropout_rate)\n",
+ " super(SABTransformerBlock, self).build(input_shape)\n",
+ "\n",
+ " def call(self, inputs, mask=None):\n",
+ " attention_output = self.attention(inputs, inputs, attention_mask=mask)[0]\n",
+ " # attention_output = self.dropout1(attention_output)\n",
+ " attention_output = self.layer_norm1(inputs + attention_output)\n",
+ " feedforward_output = self.feedforward(attention_output)\n",
+ " # feedforward_output = self.dropout2(feedforward_output)\n",
+ " block_output = self.layer_norm2(attention_output + feedforward_output)\n",
+ " return block_output"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "inputs = keras.Input(shape=(100, 7), name=\"input\")\n",
+ "x = layers.TimeDistributed(layers.Dense(64))(inputs)\n",
+ "x = SABTransformerBlock(num_heads=8, hidden_units=64)(x)\n",
+ "x = SABTransformerBlock(num_heads=8, hidden_units=64)(x)\n",
+ "x = SABTransformerBlock(num_heads=8, hidden_units=64)(x)\n",
+ "x = layers.Lambda(lambda y: tf.reduce_sum(y, axis=1))(x)\n",
+ "x = layers.BatchNormalization()(x)\n",
+ "x = layers.Dense(64)(x)\n",
+ "x = layers.LeakyReLU()(x)\n",
+ "x = layers.Dense(64)(x)\n",
+ "x = layers.LeakyReLU()(x)\n",
+ "x = layers.Dense(16)(x)\n",
+ "x = layers.LeakyReLU()(x)\n",
+ "output = layers.Dense(5, dtype=\"float32\")(x)\n",
+ "model_st = keras.models.Model(inputs=inputs, outputs=output)\n",
+ "model.summary()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "model.compile(\n",
+ " loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
+ " optimizer=keras.optimizers.Adam(learning_rate=learning_rate),\n",
+ " metrics=[keras.metrics.SparseCategoricalAccuracy()],\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "history = model.fit(X_train, y_train, batch_size=batch_size, epochs=epochs, validation_split=0.1)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import matplotlib.pyplot as plt\n",
+ "\n",
+ "fig, axes = plt.subplots(2)\n",
+ "\n",
+ "axes[0].plot(history.history[\"sparse_categorical_accuracy\"])\n",
+ "axes[0].plot(history.history[\"val_sparse_categorical_accuracy\"])\n",
+ "axes[0].set_title(\"Accuracy\")\n",
+ "axes[0].legend([\"Training\", \"Validation\"])\n",
+ "\n",
+ "axes[1].plot(history.history[\"loss\"])\n",
+ "axes[1].plot(history.history[\"val_loss\"])\n",
+ "axes[1].legend([\"Training\", \"Validation\"])\n",
+ "axes[1].set_title(\"Loss\")\n",
+ "\n",
+ "fig.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now we finally evaluate the performance by plotting the ROC curves for the different classes:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import pandas as pd\n",
+ "import matplotlib.pyplot as plt\n",
+ "\n",
+ "%matplotlib inline\n",
+ "from sklearn.metrics import roc_curve, auc\n",
+ "\n",
+ "predict_val = tf.nn.softmax(model.predict(X_val))\n",
+ "df = pd.DataFrame()\n",
+ "fpr = {}\n",
+ "tpr = {}\n",
+ "auc1 = {}\n",
+ "\n",
+ "plt.figure()\n",
+ "for i, label in enumerate(label_names):\n",
+ "\n",
+ " df[label] = y_val_onehot[:, i]\n",
+ " df[label + \"_pred\"] = predict_val[:, i]\n",
+ "\n",
+ " fpr[label], tpr[label], threshold = roc_curve(df[label], df[label + \"_pred\"])\n",
+ "\n",
+ " auc1[label] = auc(fpr[label], tpr[label])\n",
+ "\n",
+ " plt.plot(tpr[label], fpr[label], label=\"%s tagger, auc = %.1f%%\" % (label, auc1[label] * 100.0))\n",
+ "plt.semilogy()\n",
+ "plt.xlabel(\"sig. efficiency\")\n",
+ "plt.ylabel(\"bkg. mistag rate\")\n",
+ "plt.ylim(0.000001, 1)\n",
+ "plt.grid(True)\n",
+ "plt.legend(loc=\"lower right\")\n",
+ "plt.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Optional Excercise\n",
+ "\n",
+ "The original paper also use MH mechanism in the decoder step (while we used a simple sum over the latent space nodes). If you would like to try it out the `Lambda` layer should be replaced with the `PoolingByMultiHeadAttention` block below.\n",
+ "\n",
+ "Consider also the fact that it might be hard to train a Transformer architecture of this kind over the rather small dataset used here. Check out [this other dataset](https://events.mcs.cmu.edu/us-cms-2023/) for increased statistics or [this notebook](https://github.com/DLii-Research/tf-settransformer/blob/master/examples/mnist_pointcloud.ipynb) for a simpler task.\n",
+ "\n",
+ "Below is the starting point for a smarter decoder:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "9qHj1_Y7ZU-R"
+ },
+ "outputs": [],
+ "source": [
+ "class PoolingByMultiHeadAttention(tf.keras.layers.Layer):\n",
+ " def __init__(self, num_heads, hidden_units, mlp_hidden_units=128, num_seeds=1, **kwargs):\n",
+ " super(PoolingByMultiHeadAttention, self).__init__(**kwargs)\n",
+ " self.num_heads = num_heads\n",
+ " self.hidden_units = hidden_units\n",
+ " self.mlp_hidden_units = mlp_hidden_units\n",
+ " self.num_seeds = num_seeds\n",
+ "\n",
+ " def build(self, input_shape):\n",
+ "\n",
+ " self.attention = tf.keras.layers.MultiHeadAttention(\n",
+ " num_heads=self.num_heads, key_dim=self.hidden_units\n",
+ " )\n",
+ "\n",
+ " self.seed_vectors = self.add_weight(\n",
+ " shape=(1, self.num_seeds, self.hidden_units),\n",
+ " initializer=\"random_normal\",\n",
+ " trainable=True,\n",
+ " name=\"Seeds\",\n",
+ " )\n",
+ "\n",
+ " self.feedforward = tf.keras.Sequential(\n",
+ " [\n",
+ " layers.Dense(units=self.mlp_hidden_units, activation=\"relu\"),\n",
+ " layers.Dense(units=self.hidden_units),\n",
+ " ]\n",
+ " )\n",
+ " self.layer_norm1 = layers.LayerNormalization(epsilon=1e-6)\n",
+ " self.layer_norm2 = layers.LayerNormalization(epsilon=1e-6)\n",
+ " super(PoolingByMultiHeadAttention, self).build(input_shape)\n",
+ "\n",
+ " def call(self, inputs, training=None):\n",
+ " a = tf.expand_dims(self.seed_vectors, axis=0)\n",
+ " seeds = tf.tile(self.seed_vectors, [tf.shape(inputs)[0], 1, 1])\n",
+ " attention_output = self.attention(seeds, inputs)[0]\n",
+ " attention_output = self.layer_norm1(seeds + attention_output)\n",
+ " feedforward_output = self.feedforward(attention_output)\n",
+ " block_output = self.layer_norm2(attention_output + feedforward_output)\n",
+ " return block_output"
+ ]
+ }
+ ],
+ "metadata": {
+ "accelerator": "GPU",
+ "colab": {
+ "authorship_tag": "ABX9TyPn4xtio5MeIQMG/e23naQt",
+ "include_colab_link": true,
+ "provenance": []
+ },
+ "gpuClass": "standard",
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.9.16"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
}
diff --git a/machine-learning-das/setup-libraries.ipynb b/machine-learning-das/setup-libraries.ipynb
index 814b502..e2d4982 100644
--- a/machine-learning-das/setup-libraries.ipynb
+++ b/machine-learning-das/setup-libraries.ipynb
@@ -19,678 +19,9 @@
},
{
"cell_type": "code",
- "execution_count": 1,
+ "execution_count": null,
"metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Sourcing the conda and mamba setup scripts...\n",
- "Creating the miniconda3 environment \"machine-learning-das\"...\n",
- "Retrieving notices: ...working... done\n",
- "\n",
- "\n",
- "Looking for: ['python==3.10', 'pip']\n",
- "\n",
- "\n",
- "Transaction\n",
- "\n",
- " Prefix: /home/cms.rkansal/mambaforge/envs/machine-learning-das\n",
- "\n",
- " Updating specs:\n",
- "\n",
- " - python==3.10\n",
- " - pip\n",
- "\n",
- "\n",
- " Package Version Build Channel Size\n",
- "───────────────────────────────────────────────────────────────────────────────────────\n",
- " Install:\n",
- "───────────────────────────────────────────────────────────────────────────────────────\n",
- "\n",
- " + _libgcc_mutex 0.1 conda_forge conda-forge/linux-64 Cached\n",
- " + _openmp_mutex 4.5 2_gnu conda-forge/linux-64 Cached\n",
- " + bzip2 1.0.8 hd590300_5 conda-forge/linux-64 254kB\n",
- " + ca-certificates 2023.11.17 hbcca054_0 conda-forge/linux-64 154kB\n",
- " + ld_impl_linux-64 2.40 h41732ed_0 conda-forge/linux-64 Cached\n",
- " + libffi 3.4.2 h7f98852_5 conda-forge/linux-64 Cached\n",
- " + libgcc-ng 13.2.0 h807b86a_3 conda-forge/linux-64 774kB\n",
- " + libgomp 13.2.0 h807b86a_3 conda-forge/linux-64 422kB\n",
- " + libnsl 2.0.1 hd590300_0 conda-forge/linux-64 33kB\n",
- " + libsqlite 3.44.2 h2797004_0 conda-forge/linux-64 846kB\n",
- " + libuuid 2.38.1 h0b41bf4_0 conda-forge/linux-64 Cached\n",
- " + libzlib 1.2.13 hd590300_5 conda-forge/linux-64 Cached\n",
- " + ncurses 6.4 h59595ed_2 conda-forge/linux-64 884kB\n",
- " + openssl 3.2.0 hd590300_1 conda-forge/linux-64 3MB\n",
- " + pip 23.3.2 pyhd8ed1ab_0 conda-forge/noarch 1MB\n",
- " + python 3.10.0 h543edf9_3_cpython conda-forge/linux-64 Cached\n",
- " + readline 8.2 h8228510_1 conda-forge/linux-64 Cached\n",
- " + setuptools 69.0.3 pyhd8ed1ab_0 conda-forge/noarch 471kB\n",
- " + sqlite 3.44.2 h2c6b66d_0 conda-forge/linux-64 836kB\n",
- " + tk 8.6.13 noxft_h4845f30_101 conda-forge/linux-64 3MB\n",
- " + tzdata 2023d h0c530f3_0 conda-forge/noarch 120kB\n",
- " + wheel 0.42.0 pyhd8ed1ab_0 conda-forge/noarch 58kB\n",
- " + xz 5.2.6 h166bdaf_0 conda-forge/linux-64 Cached\n",
- "\n",
- " Summary:\n",
- "\n",
- " Install: 23 packages\n",
- "\n",
- " Total download: 12MB\n",
- "\n",
- "───────────────────────────────────────────────────────────────────────────────────────\n",
- "\n",
- "\n",
- "\n",
- "Downloading and Extracting Packages\n",
- "\r\n",
- "Preparing transaction: ...working... done\n",
- "Verifying transaction: ...working... done\n",
- "Executing transaction: ...working... done\n",
- "Installing pip dependencies: ...working... Ran pip subprocess with arguments:\n",
- "['/home/cms.rkansal/mambaforge/envs/machine-learning-das/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/cms.rkansal/machine-learning-das/machine-learning-das/condaenv.5i5i1hst.requirements.txt', '--exists-action=b']\n",
- "Pip subprocess output:\n",
- "Collecting matplotlib (from -r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 1))\n",
- " Downloading matplotlib-3.8.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.8 kB)\n",
- "Collecting numpy (from -r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 2))\n",
- " Downloading numpy-1.26.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 7.1 MB/s eta 0:00:00\n",
- "Collecting uproot (from -r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 3))\n",
- " Downloading uproot-5.2.1-py3-none-any.whl.metadata (30 kB)\n",
- "Collecting h5py (from -r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 4))\n",
- " Downloading h5py-3.10.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.5 kB)\n",
- "Collecting keras (from -r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 5))\n",
- " Downloading keras-3.0.2-py3-none-any.whl.metadata (4.8 kB)\n",
- "Collecting tensorflow (from -r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading tensorflow-2.15.0.post1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.2 kB)\n",
- "Collecting scikit-learn (from -r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 7))\n",
- " Downloading scikit_learn-1.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)\n",
- "Collecting scikit-optimize (from -r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 8))\n",
- " Downloading scikit_optimize-0.9.0-py2.py3-none-any.whl (100 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.3/100.3 kB 29.2 MB/s eta 0:00:00\n",
- "Collecting pandas (from -r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 9))\n",
- " Downloading pandas-2.1.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB)\n",
- "Collecting torch (from -r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading torch-2.1.2-cp310-cp310-manylinux1_x86_64.whl.metadata (25 kB)\n",
- "Collecting ipykernel (from -r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading ipykernel-6.28.0-py3-none-any.whl.metadata (6.0 kB)\n",
- "Collecting tqdm (from -r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 12))\n",
- " Downloading tqdm-4.66.1-py3-none-any.whl.metadata (57 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.6/57.6 kB 97.9 MB/s eta 0:00:00\n",
- "Collecting jupyter (from -r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading jupyter-1.0.0-py2.py3-none-any.whl (2.7 kB)\n",
- "Collecting xgboost (from -r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 14))\n",
- " Downloading xgboost-2.0.3-py3-none-manylinux2014_x86_64.whl.metadata (2.0 kB)\n",
- "Collecting contourpy>=1.0.1 (from matplotlib->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 1))\n",
- " Downloading contourpy-1.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.8 kB)\n",
- "Collecting cycler>=0.10 (from matplotlib->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 1))\n",
- " Downloading cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB)\n",
- "Collecting fonttools>=4.22.0 (from matplotlib->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 1))\n",
- " Downloading fonttools-4.47.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (157 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 157.2/157.2 kB 180.8 MB/s eta 0:00:00\n",
- "Collecting kiwisolver>=1.3.1 (from matplotlib->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 1))\n",
- " Downloading kiwisolver-1.4.5-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (6.4 kB)\n",
- "Collecting packaging>=20.0 (from matplotlib->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 1))\n",
- " Downloading packaging-23.2-py3-none-any.whl.metadata (3.2 kB)\n",
- "Collecting pillow>=8 (from matplotlib->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 1))\n",
- " Downloading pillow-10.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.7 kB)\n",
- "Collecting pyparsing>=2.3.1 (from matplotlib->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 1))\n",
- " Downloading pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)\n",
- "Collecting python-dateutil>=2.7 (from matplotlib->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 1))\n",
- " Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 122.7 MB/s eta 0:00:00\n",
- "Collecting awkward>=2.4.6 (from uproot->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 3))\n",
- " Downloading awkward-2.5.1-py3-none-any.whl.metadata (6.9 kB)\n",
- "Collecting fsspec (from uproot->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 3))\n",
- " Downloading fsspec-2023.12.2-py3-none-any.whl.metadata (6.8 kB)\n",
- "Collecting typing-extensions>=4.1.0 (from uproot->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 3))\n",
- " Downloading typing_extensions-4.9.0-py3-none-any.whl.metadata (3.0 kB)\n",
- "Collecting absl-py (from keras->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 5))\n",
- " Downloading absl_py-2.0.0-py3-none-any.whl.metadata (2.3 kB)\n",
- "Collecting rich (from keras->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 5))\n",
- " Downloading rich-13.7.0-py3-none-any.whl.metadata (18 kB)\n",
- "Collecting namex (from keras->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 5))\n",
- " Downloading namex-0.0.7-py3-none-any.whl (5.8 kB)\n",
- "Collecting dm-tree (from keras->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 5))\n",
- " Downloading dm_tree-0.1.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (152 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 152.8/152.8 kB 196.7 MB/s eta 0:00:00\n",
- "Collecting astunparse>=1.6.0 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)\n",
- "Collecting flatbuffers>=23.5.26 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading flatbuffers-23.5.26-py2.py3-none-any.whl.metadata (850 bytes)\n",
- "Collecting gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading gast-0.5.4-py3-none-any.whl (19 kB)\n",
- "Collecting google-pasta>=0.1.1 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.5/57.5 kB 172.5 MB/s eta 0:00:00\n",
- "Collecting libclang>=13.0.0 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl.metadata (5.2 kB)\n",
- "Collecting ml-dtypes~=0.2.0 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading ml_dtypes-0.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (20 kB)\n",
- "Collecting opt-einsum>=2.3.2 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.5/65.5 kB 123.0 MB/s eta 0:00:00\n",
- "Collecting protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading protobuf-4.25.1-cp37-abi3-manylinux2014_x86_64.whl.metadata (541 bytes)\n",
- "Requirement already satisfied: setuptools in /home/cms.rkansal/mambaforge/envs/machine-learning-das/lib/python3.10/site-packages (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6)) (69.0.3)\n",
- "Collecting six>=1.12.0 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)\n",
- "Collecting termcolor>=1.1.0 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading termcolor-2.4.0-py3-none-any.whl.metadata (6.1 kB)\n",
- "Collecting wrapt<1.15,>=1.11.0 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading wrapt-1.14.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (77 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 kB 106.3 MB/s eta 0:00:00\n",
- "Collecting tensorflow-io-gcs-filesystem>=0.23.1 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading tensorflow_io_gcs_filesystem-0.35.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (14 kB)\n",
- "Collecting grpcio<2.0,>=1.24.3 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading grpcio-1.60.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB)\n",
- "Collecting tensorboard<2.16,>=2.15 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading tensorboard-2.15.1-py3-none-any.whl.metadata (1.7 kB)\n",
- "Collecting tensorflow-estimator<2.16,>=2.15.0 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading tensorflow_estimator-2.15.0-py2.py3-none-any.whl.metadata (1.3 kB)\n",
- "Collecting keras (from -r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 5))\n",
- " Downloading keras-2.15.0-py3-none-any.whl.metadata (2.4 kB)\n",
- "Collecting scipy>=1.5.0 (from scikit-learn->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 7))\n",
- " Downloading scipy-1.11.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.4/60.4 kB 115.9 MB/s eta 0:00:00\n",
- "Collecting joblib>=1.1.1 (from scikit-learn->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 7))\n",
- " Downloading joblib-1.3.2-py3-none-any.whl.metadata (5.4 kB)\n",
- "Collecting threadpoolctl>=2.0.0 (from scikit-learn->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 7))\n",
- " Downloading threadpoolctl-3.2.0-py3-none-any.whl.metadata (10.0 kB)\n",
- "Collecting pyaml>=16.9 (from scikit-optimize->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 8))\n",
- " Downloading pyaml-23.12.0-py3-none-any.whl.metadata (11 kB)\n",
- "Collecting pytz>=2020.1 (from pandas->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 9))\n",
- " Downloading pytz-2023.3.post1-py2.py3-none-any.whl.metadata (22 kB)\n",
- "Collecting tzdata>=2022.1 (from pandas->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 9))\n",
- " Downloading tzdata-2023.4-py2.py3-none-any.whl.metadata (1.4 kB)\n",
- "Collecting filelock (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading filelock-3.13.1-py3-none-any.whl.metadata (2.8 kB)\n",
- "Collecting sympy (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading sympy-1.12-py3-none-any.whl (5.7 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.7/5.7 MB 59.7 MB/s eta 0:00:00\n",
- "Collecting networkx (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading networkx-3.2.1-py3-none-any.whl.metadata (5.2 kB)\n",
- "Collecting jinja2 (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 kB 122.8 MB/s eta 0:00:00\n",
- "Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 23.7/23.7 MB 65.8 MB/s eta 0:00:00\n",
- "Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 823.6/823.6 kB 59.1 MB/s eta 0:00:00\n",
- "Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.1/14.1 MB 21.0 MB/s eta 0:00:00\n",
- "Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n",
- "Collecting nvidia-cublas-cu12==12.1.3.1 (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 410.6/410.6 MB 26.7 MB/s eta 0:00:00\n",
- "Collecting nvidia-cufft-cu12==11.0.2.54 (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.6/121.6 MB 34.0 MB/s eta 0:00:00\n",
- "Collecting nvidia-curand-cu12==10.3.2.106 (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.5/56.5 MB 30.7 MB/s eta 0:00:00\n",
- "Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.2/124.2 MB 25.4 MB/s eta 0:00:00\n",
- "Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 196.0/196.0 MB 35.0 MB/s eta 0:00:00\n",
- "Collecting nvidia-nccl-cu12==2.18.1 (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading nvidia_nccl_cu12-2.18.1-py3-none-manylinux1_x86_64.whl (209.8 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 209.8/209.8 MB 34.6 MB/s eta 0:00:00\n",
- "Collecting nvidia-nvtx-cu12==12.1.105 (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 99.1/99.1 kB 142.8 MB/s eta 0:00:00\n",
- "Collecting triton==2.1.0 (from torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading triton-2.1.0-0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.3 kB)\n",
- "Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading nvidia_nvjitlink_cu12-12.3.101-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n",
- "Collecting comm>=0.1.1 (from ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading comm-0.2.1-py3-none-any.whl.metadata (3.7 kB)\n",
- "Collecting debugpy>=1.6.5 (from ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading debugpy-1.8.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.0 kB)\n",
- "Collecting ipython>=7.23.1 (from ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading ipython-8.20.0-py3-none-any.whl.metadata (5.9 kB)\n",
- "Collecting jupyter-client>=6.1.12 (from ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading jupyter_client-8.6.0-py3-none-any.whl.metadata (8.3 kB)\n",
- "Collecting jupyter-core!=5.0.*,>=4.12 (from ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading jupyter_core-5.7.1-py3-none-any.whl.metadata (3.4 kB)\n",
- "Collecting matplotlib-inline>=0.1 (from ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading matplotlib_inline-0.1.6-py3-none-any.whl (9.4 kB)\n",
- "Collecting nest-asyncio (from ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading nest_asyncio-1.5.8-py3-none-any.whl.metadata (2.8 kB)\n",
- "Collecting psutil (from ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading psutil-5.9.7-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (21 kB)\n",
- "Collecting pyzmq>=24 (from ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading pyzmq-25.1.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.9 kB)\n",
- "Collecting tornado>=6.1 (from ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading tornado-6.4-cp38-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.5 kB)\n",
- "Collecting traitlets>=5.4.0 (from ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading traitlets-5.14.1-py3-none-any.whl.metadata (10 kB)\n",
- "Collecting notebook (from jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading notebook-7.0.6-py3-none-any.whl.metadata (10 kB)\n",
- "Collecting qtconsole (from jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading qtconsole-5.5.1-py3-none-any.whl.metadata (5.1 kB)\n",
- "Collecting jupyter-console (from jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading jupyter_console-6.6.3-py3-none-any.whl (24 kB)\n",
- "Collecting nbconvert (from jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading nbconvert-7.14.0-py3-none-any.whl.metadata (7.7 kB)\n",
- "Collecting ipywidgets (from jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading ipywidgets-8.1.1-py3-none-any.whl.metadata (2.4 kB)\n",
- "Requirement already satisfied: wheel<1.0,>=0.23.0 in /home/cms.rkansal/mambaforge/envs/machine-learning-das/lib/python3.10/site-packages (from astunparse>=1.6.0->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6)) (0.42.0)\n",
- "Collecting awkward-cpp==27 (from awkward>=2.4.6->uproot->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 3))\n",
- " Downloading awkward_cpp-27-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.2 kB)\n",
- "Collecting importlib-metadata>=4.13.0 (from awkward>=2.4.6->uproot->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 3))\n",
- " Downloading importlib_metadata-7.0.1-py3-none-any.whl.metadata (4.9 kB)\n",
- "Collecting decorator (from ipython>=7.23.1->ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading decorator-5.1.1-py3-none-any.whl (9.1 kB)\n",
- "Collecting jedi>=0.16 (from ipython>=7.23.1->ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading jedi-0.19.1-py2.py3-none-any.whl.metadata (22 kB)\n",
- "Collecting prompt-toolkit<3.1.0,>=3.0.41 (from ipython>=7.23.1->ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading prompt_toolkit-3.0.43-py3-none-any.whl.metadata (6.5 kB)\n",
- "Collecting pygments>=2.4.0 (from ipython>=7.23.1->ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading pygments-2.17.2-py3-none-any.whl.metadata (2.6 kB)\n",
- "Collecting stack-data (from ipython>=7.23.1->ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading stack_data-0.6.3-py3-none-any.whl.metadata (18 kB)\n",
- "Collecting exceptiongroup (from ipython>=7.23.1->ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading exceptiongroup-1.2.0-py3-none-any.whl.metadata (6.6 kB)\n",
- "Collecting pexpect>4.3 (from ipython>=7.23.1->ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading pexpect-4.9.0-py2.py3-none-any.whl.metadata (2.5 kB)\n",
- "Collecting platformdirs>=2.5 (from jupyter-core!=5.0.*,>=4.12->ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading platformdirs-4.1.0-py3-none-any.whl.metadata (11 kB)\n",
- "Collecting PyYAML (from pyaml>=16.9->scikit-optimize->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 8))\n",
- " Downloading PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)\n",
- "Collecting google-auth<3,>=1.6.3 (from tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading google_auth-2.26.1-py2.py3-none-any.whl.metadata (4.7 kB)\n",
- "Collecting google-auth-oauthlib<2,>=0.5 (from tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading google_auth_oauthlib-1.2.0-py2.py3-none-any.whl.metadata (2.7 kB)\n",
- "Collecting markdown>=2.6.8 (from tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading Markdown-3.5.1-py3-none-any.whl.metadata (7.1 kB)\n",
- "Collecting protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 (from tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl.metadata (540 bytes)\n",
- "Collecting requests<3,>=2.21.0 (from tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)\n",
- "Collecting tensorboard-data-server<0.8.0,>=0.7.0 (from tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading tensorboard_data_server-0.7.2-py3-none-any.whl.metadata (1.1 kB)\n",
- "Collecting werkzeug>=1.0.1 (from tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading werkzeug-3.0.1-py3-none-any.whl.metadata (4.1 kB)\n",
- "Collecting widgetsnbextension~=4.0.9 (from ipywidgets->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading widgetsnbextension-4.0.9-py3-none-any.whl.metadata (1.6 kB)\n",
- "Collecting jupyterlab-widgets~=3.0.9 (from ipywidgets->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading jupyterlab_widgets-3.0.9-py3-none-any.whl.metadata (4.1 kB)\n",
- "Collecting MarkupSafe>=2.0 (from jinja2->torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)\n",
- "Collecting beautifulsoup4 (from nbconvert->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading beautifulsoup4-4.12.2-py3-none-any.whl (142 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.0/143.0 kB 111.8 MB/s eta 0:00:00\n",
- "Collecting bleach!=5.0.0 (from nbconvert->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading bleach-6.1.0-py3-none-any.whl.metadata (30 kB)\n",
- "Collecting defusedxml (from nbconvert->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)\n",
- "Collecting jupyterlab-pygments (from nbconvert->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading jupyterlab_pygments-0.3.0-py3-none-any.whl.metadata (4.4 kB)\n",
- "Collecting mistune<4,>=2.0.3 (from nbconvert->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading mistune-3.0.2-py3-none-any.whl.metadata (1.7 kB)\n",
- "Collecting nbclient>=0.5.0 (from nbconvert->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading nbclient-0.9.0-py3-none-any.whl.metadata (7.8 kB)\n",
- "Collecting nbformat>=5.7 (from nbconvert->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading nbformat-5.9.2-py3-none-any.whl.metadata (3.4 kB)\n",
- "Collecting pandocfilters>=1.4.1 (from nbconvert->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading pandocfilters-1.5.0-py2.py3-none-any.whl (8.7 kB)\n",
- "Collecting tinycss2 (from nbconvert->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading tinycss2-1.2.1-py3-none-any.whl (21 kB)\n",
- "Collecting jupyter-server<3,>=2.4.0 (from notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading jupyter_server-2.12.2-py3-none-any.whl.metadata (8.4 kB)\n",
- "Collecting jupyterlab-server<3,>=2.22.1 (from notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading jupyterlab_server-2.25.2-py3-none-any.whl.metadata (5.9 kB)\n",
- "Collecting jupyterlab<5,>=4.0.2 (from notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading jupyterlab-4.0.10-py3-none-any.whl.metadata (15 kB)\n",
- "Collecting notebook-shim<0.3,>=0.2 (from notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading notebook_shim-0.2.3-py3-none-any.whl (13 kB)\n",
- "Collecting qtpy>=2.4.0 (from qtconsole->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading QtPy-2.4.1-py3-none-any.whl.metadata (12 kB)\n",
- "Collecting mpmath>=0.19 (from sympy->torch->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 10))\n",
- " Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 44.7 MB/s eta 0:00:00\n",
- "Collecting webencodings (from bleach!=5.0.0->nbconvert->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)\n",
- "Collecting cachetools<6.0,>=2.0.0 (from google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading cachetools-5.3.2-py3-none-any.whl.metadata (5.2 kB)\n",
- "Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.3/181.3 kB 122.0 MB/s eta 0:00:00\n",
- "Collecting rsa<5,>=3.1.4 (from google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading rsa-4.9-py3-none-any.whl (34 kB)\n",
- "Collecting requests-oauthlib>=0.7.0 (from google-auth-oauthlib<2,>=0.5->tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)\n",
- "Collecting zipp>=0.5 (from importlib-metadata>=4.13.0->awkward>=2.4.6->uproot->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 3))\n",
- " Downloading zipp-3.17.0-py3-none-any.whl.metadata (3.7 kB)\n",
- "Collecting parso<0.9.0,>=0.8.3 (from jedi>=0.16->ipython>=7.23.1->ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading parso-0.8.3-py2.py3-none-any.whl (100 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.8/100.8 kB 106.4 MB/s eta 0:00:00\n",
- "Collecting anyio>=3.1.0 (from jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading anyio-4.2.0-py3-none-any.whl.metadata (4.6 kB)\n",
- "Collecting argon2-cffi (from jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading argon2_cffi-23.1.0-py3-none-any.whl.metadata (5.2 kB)\n",
- "Collecting jupyter-events>=0.9.0 (from jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading jupyter_events-0.9.0-py3-none-any.whl.metadata (5.7 kB)\n",
- "Collecting jupyter-server-terminals (from jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading jupyter_server_terminals-0.5.1-py3-none-any.whl.metadata (5.6 kB)\n",
- "Collecting overrides (from jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading overrides-7.4.0-py3-none-any.whl.metadata (5.7 kB)\n",
- "Collecting prometheus-client (from jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading prometheus_client-0.19.0-py3-none-any.whl.metadata (1.8 kB)\n",
- "Collecting send2trash>=1.8.2 (from jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading Send2Trash-1.8.2-py3-none-any.whl (18 kB)\n",
- "Collecting terminado>=0.8.3 (from jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading terminado-0.18.0-py3-none-any.whl.metadata (5.8 kB)\n",
- "Collecting websocket-client (from jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading websocket_client-1.7.0-py3-none-any.whl.metadata (7.9 kB)\n",
- "Collecting async-lru>=1.0.0 (from jupyterlab<5,>=4.0.2->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading async_lru-2.0.4-py3-none-any.whl.metadata (4.5 kB)\n",
- "Collecting jupyter-lsp>=2.0.0 (from jupyterlab<5,>=4.0.2->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading jupyter_lsp-2.2.1-py3-none-any.whl.metadata (1.8 kB)\n",
- "Collecting tomli (from jupyterlab<5,>=4.0.2->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading tomli-2.0.1-py3-none-any.whl (12 kB)\n",
- "Collecting babel>=2.10 (from jupyterlab-server<3,>=2.22.1->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading Babel-2.14.0-py3-none-any.whl.metadata (1.6 kB)\n",
- "Collecting json5>=0.9.0 (from jupyterlab-server<3,>=2.22.1->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading json5-0.9.14-py2.py3-none-any.whl.metadata (10 kB)\n",
- "Collecting jsonschema>=4.18.0 (from jupyterlab-server<3,>=2.22.1->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading jsonschema-4.20.0-py3-none-any.whl.metadata (8.1 kB)\n",
- "Collecting fastjsonschema (from nbformat>=5.7->nbconvert->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading fastjsonschema-2.19.1-py3-none-any.whl.metadata (2.1 kB)\n",
- "Collecting ptyprocess>=0.5 (from pexpect>4.3->ipython>=7.23.1->ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)\n",
- "Collecting wcwidth (from prompt-toolkit<3.1.0,>=3.0.41->ipython>=7.23.1->ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading wcwidth-0.2.13-py2.py3-none-any.whl.metadata (14 kB)\n",
- "Collecting charset-normalizer<4,>=2 (from requests<3,>=2.21.0->tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)\n",
- "Collecting idna<4,>=2.5 (from requests<3,>=2.21.0->tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading idna-3.6-py3-none-any.whl.metadata (9.9 kB)\n",
- "Collecting urllib3<3,>=1.21.1 (from requests<3,>=2.21.0->tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading urllib3-2.1.0-py3-none-any.whl.metadata (6.4 kB)\n",
- "Collecting certifi>=2017.4.17 (from requests<3,>=2.21.0->tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading certifi-2023.11.17-py3-none-any.whl.metadata (2.2 kB)\n",
- "Collecting soupsieve>1.2 (from beautifulsoup4->nbconvert->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading soupsieve-2.5-py3-none-any.whl.metadata (4.7 kB)\n",
- "Collecting executing>=1.2.0 (from stack-data->ipython>=7.23.1->ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading executing-2.0.1-py2.py3-none-any.whl.metadata (9.0 kB)\n",
- "Collecting asttokens>=2.1.0 (from stack-data->ipython>=7.23.1->ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading asttokens-2.4.1-py2.py3-none-any.whl.metadata (5.2 kB)\n",
- "Collecting pure-eval (from stack-data->ipython>=7.23.1->ipykernel->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 11))\n",
- " Downloading pure_eval-0.2.2-py3-none-any.whl (11 kB)\n",
- "Collecting sniffio>=1.1 (from anyio>=3.1.0->jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading sniffio-1.3.0-py3-none-any.whl (10 kB)\n",
- "Collecting attrs>=22.2.0 (from jsonschema>=4.18.0->jupyterlab-server<3,>=2.22.1->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading attrs-23.2.0-py3-none-any.whl.metadata (9.5 kB)\n",
- "Collecting jsonschema-specifications>=2023.03.6 (from jsonschema>=4.18.0->jupyterlab-server<3,>=2.22.1->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading jsonschema_specifications-2023.12.1-py3-none-any.whl.metadata (3.0 kB)\n",
- "Collecting referencing>=0.28.4 (from jsonschema>=4.18.0->jupyterlab-server<3,>=2.22.1->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading referencing-0.32.1-py3-none-any.whl.metadata (2.7 kB)\n",
- "Collecting rpds-py>=0.7.1 (from jsonschema>=4.18.0->jupyterlab-server<3,>=2.22.1->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading rpds_py-0.16.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.1 kB)\n",
- "Collecting python-json-logger>=2.0.4 (from jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading python_json_logger-2.0.7-py3-none-any.whl (8.1 kB)\n",
- "Collecting rfc3339-validator (from jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading rfc3339_validator-0.1.4-py2.py3-none-any.whl (3.5 kB)\n",
- "Collecting rfc3986-validator>=0.1.1 (from jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading rfc3986_validator-0.1.1-py2.py3-none-any.whl (4.2 kB)\n",
- "Collecting pyasn1<0.6.0,>=0.4.6 (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading pyasn1-0.5.1-py2.py3-none-any.whl.metadata (8.6 kB)\n",
- "Collecting oauthlib>=3.0.0 (from requests-oauthlib>=0.7.0->google-auth-oauthlib<2,>=0.5->tensorboard<2.16,>=2.15->tensorflow->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 6))\n",
- " Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 kB 123.9 MB/s eta 0:00:00\n",
- "Collecting argon2-cffi-bindings (from argon2-cffi->jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading argon2_cffi_bindings-21.2.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (86 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.2/86.2 kB 91.0 MB/s eta 0:00:00\n",
- "Collecting fqdn (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading fqdn-1.5.1-py3-none-any.whl (9.1 kB)\n",
- "Collecting isoduration (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading isoduration-20.11.0-py3-none-any.whl (11 kB)\n",
- "Collecting jsonpointer>1.13 (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading jsonpointer-2.4-py2.py3-none-any.whl.metadata (2.5 kB)\n",
- "Collecting uri-template (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading uri_template-1.3.0-py3-none-any.whl.metadata (8.8 kB)\n",
- "Collecting webcolors>=1.11 (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading webcolors-1.13-py3-none-any.whl (14 kB)\n",
- "Collecting cffi>=1.0.1 (from argon2-cffi-bindings->argon2-cffi->jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading cffi-1.16.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.5 kB)\n",
- "Collecting pycparser (from cffi>=1.0.1->argon2-cffi-bindings->argon2-cffi->jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 kB 182.1 MB/s eta 0:00:00\n",
- "Collecting arrow>=0.15.0 (from isoduration->jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading arrow-1.3.0-py3-none-any.whl.metadata (7.5 kB)\n",
- "Collecting types-python-dateutil>=2.8.10 (from arrow>=0.15.0->isoduration->jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook->jupyter->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 13))\n",
- " Downloading types_python_dateutil-2.8.19.20240106-py3-none-any.whl.metadata (1.8 kB)\n",
- "Collecting markdown-it-py>=2.2.0 (from rich->keras->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 5))\n",
- " Downloading markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)\n",
- "Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich->keras->-r /home/cms.rkansal/machine-learning-das/machine-learning-das/requirements.txt (line 5))\n",
- " Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)\n",
- "Downloading matplotlib-3.8.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.6/11.6 MB 43.1 MB/s eta 0:00:00\n",
- "Downloading numpy-1.26.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 46.7 MB/s eta 0:00:00\n",
- "Downloading uproot-5.2.1-py3-none-any.whl (343 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 343.8/343.8 kB 56.7 MB/s eta 0:00:00\n",
- "Downloading h5py-3.10.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.8 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.8/4.8 MB 57.1 MB/s eta 0:00:00\n",
- "Downloading tensorflow-2.15.0.post1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (475.2 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 475.2/475.2 MB 36.6 MB/s eta 0:00:00\n",
- "Downloading keras-2.15.0-py3-none-any.whl (1.7 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 44.6 MB/s eta 0:00:00\n",
- "Downloading scikit_learn-1.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (10.8 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.8/10.8 MB 27.5 MB/s eta 0:00:00\n",
- "Downloading pandas-2.1.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.3 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.3/12.3 MB 23.1 MB/s eta 0:00:00\n",
- "Downloading torch-2.1.2-cp310-cp310-manylinux1_x86_64.whl (670.2 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 670.2/670.2 MB 28.3 MB/s eta 0:00:00\n",
- "Downloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 731.7/731.7 MB 45.1 MB/s eta 0:00:00\n",
- "Downloading triton-2.1.0-0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (89.2 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.2/89.2 MB 23.9 MB/s eta 0:00:00\n",
- "Downloading ipykernel-6.28.0-py3-none-any.whl (114 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.1/114.1 kB 165.6 MB/s eta 0:00:00\n",
- "Downloading tqdm-4.66.1-py3-none-any.whl (78 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.3/78.3 kB 194.3 MB/s eta 0:00:00\n",
- "Downloading xgboost-2.0.3-py3-none-manylinux2014_x86_64.whl (297.1 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 297.1/297.1 MB 32.8 MB/s eta 0:00:00\n",
- "Downloading absl_py-2.0.0-py3-none-any.whl (130 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 130.2/130.2 kB 189.1 MB/s eta 0:00:00\n",
- "Downloading awkward-2.5.1-py3-none-any.whl (733 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 733.1/733.1 kB 37.9 MB/s eta 0:00:00\n",
- "Downloading awkward_cpp-27-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (706 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 706.4/706.4 kB 40.5 MB/s eta 0:00:00\n",
- "Downloading comm-0.2.1-py3-none-any.whl (7.2 kB)\n",
- "Downloading contourpy-1.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (310 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 310.7/310.7 kB 48.4 MB/s eta 0:00:00\n",
- "Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB)\n",
- "Downloading debugpy-1.8.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB 35.7 MB/s eta 0:00:00\n",
- "Downloading flatbuffers-23.5.26-py2.py3-none-any.whl (26 kB)\n",
- "Downloading fonttools-4.47.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.6/4.6 MB 40.1 MB/s eta 0:00:00\n",
- "Downloading grpcio-1.60.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.4/5.4 MB 42.2 MB/s eta 0:00:00\n",
- "Downloading ipython-8.20.0-py3-none-any.whl (809 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 809.2/809.2 kB 52.8 MB/s eta 0:00:00\n",
- "Downloading joblib-1.3.2-py3-none-any.whl (302 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 302.2/302.2 kB 76.6 MB/s eta 0:00:00\n",
- "Downloading jupyter_client-8.6.0-py3-none-any.whl (105 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 105.9/105.9 kB 203.2 MB/s eta 0:00:00\n",
- "Downloading jupyter_core-5.7.1-py3-none-any.whl (28 kB)\n",
- "Downloading kiwisolver-1.4.5-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 46.6 MB/s eta 0:00:00\n",
- "Downloading libclang-16.0.6-py2.py3-none-manylinux2010_x86_64.whl (22.9 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 22.9/22.9 MB 42.2 MB/s eta 0:00:00\n",
- "Downloading ml_dtypes-0.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 39.1 MB/s eta 0:00:00\n",
- "Downloading packaging-23.2-py3-none-any.whl (53 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.0/53.0 kB 127.7 MB/s eta 0:00:00\n",
- "Downloading pillow-10.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.4 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.4/4.4 MB 36.7 MB/s eta 0:00:00\n",
- "Downloading pyaml-23.12.0-py3-none-any.whl (23 kB)\n",
- "Downloading pyparsing-3.1.1-py3-none-any.whl (103 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 103.1/103.1 kB 89.5 MB/s eta 0:00:00\n",
- "Downloading pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 502.5/502.5 kB 34.0 MB/s eta 0:00:00\n",
- "Downloading pyzmq-25.1.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 30.3 MB/s eta 0:00:00\n",
- "Downloading scipy-1.11.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (36.4 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 36.4/36.4 MB 34.4 MB/s eta 0:00:00\n",
- "Downloading tensorboard-2.15.1-py3-none-any.whl (5.5 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.5/5.5 MB 37.7 MB/s eta 0:00:00\n",
- "Downloading protobuf-4.23.4-cp37-abi3-manylinux2014_x86_64.whl (304 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 304.5/304.5 kB 47.9 MB/s eta 0:00:00\n",
- "Downloading tensorflow_estimator-2.15.0-py2.py3-none-any.whl (441 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 442.0/442.0 kB 55.0 MB/s eta 0:00:00\n",
- "Downloading tensorflow_io_gcs_filesystem-0.35.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.2 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 42.9 MB/s eta 0:00:00\n",
- "Downloading termcolor-2.4.0-py3-none-any.whl (7.7 kB)\n",
- "Downloading threadpoolctl-3.2.0-py3-none-any.whl (15 kB)\n",
- "Downloading tornado-6.4-cp38-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (435 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 435.4/435.4 kB 55.8 MB/s eta 0:00:00\n",
- "Downloading traitlets-5.14.1-py3-none-any.whl (85 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 85.4/85.4 kB 179.2 MB/s eta 0:00:00\n",
- "Downloading typing_extensions-4.9.0-py3-none-any.whl (32 kB)\n",
- "Downloading tzdata-2023.4-py2.py3-none-any.whl (346 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 346.6/346.6 kB 54.3 MB/s eta 0:00:00\n",
- "Downloading filelock-3.13.1-py3-none-any.whl (11 kB)\n",
- "Downloading fsspec-2023.12.2-py3-none-any.whl (168 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 169.0/169.0 kB 150.6 MB/s eta 0:00:00\n",
- "Downloading ipywidgets-8.1.1-py3-none-any.whl (139 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 139.4/139.4 kB 157.2 MB/s eta 0:00:00\n",
- "Downloading nbconvert-7.14.0-py3-none-any.whl (256 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 256.4/256.4 kB 100.7 MB/s eta 0:00:00\n",
- "Downloading nest_asyncio-1.5.8-py3-none-any.whl (5.3 kB)\n",
- "Downloading networkx-3.2.1-py3-none-any.whl (1.6 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 46.1 MB/s eta 0:00:00\n",
- "Downloading notebook-7.0.6-py3-none-any.whl (4.0 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.0/4.0 MB 45.6 MB/s eta 0:00:00\n",
- "Downloading psutil-5.9.7-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (285 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 285.5/285.5 kB 79.2 MB/s eta 0:00:00\n",
- "Downloading qtconsole-5.5.1-py3-none-any.whl (123 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 123.4/123.4 kB 117.3 MB/s eta 0:00:00\n",
- "Downloading bleach-6.1.0-py3-none-any.whl (162 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 162.8/162.8 kB 145.2 MB/s eta 0:00:00\n",
- "Downloading google_auth-2.26.1-py2.py3-none-any.whl (186 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 186.4/186.4 kB 177.6 MB/s eta 0:00:00\n",
- "Downloading google_auth_oauthlib-1.2.0-py2.py3-none-any.whl (24 kB)\n",
- "Downloading importlib_metadata-7.0.1-py3-none-any.whl (23 kB)\n",
- "Downloading jedi-0.19.1-py2.py3-none-any.whl (1.6 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 48.9 MB/s eta 0:00:00\n",
- "Downloading jupyter_server-2.12.2-py3-none-any.whl (380 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 380.3/380.3 kB 53.8 MB/s eta 0:00:00\n",
- "Downloading jupyterlab-4.0.10-py3-none-any.whl (9.2 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.2/9.2 MB 45.9 MB/s eta 0:00:00\n",
- "Downloading jupyterlab_server-2.25.2-py3-none-any.whl (58 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.9/58.9 kB 156.0 MB/s eta 0:00:00\n",
- "Downloading jupyterlab_widgets-3.0.9-py3-none-any.whl (214 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 214.9/214.9 kB 149.7 MB/s eta 0:00:00\n",
- "Downloading Markdown-3.5.1-py3-none-any.whl (102 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 102.2/102.2 kB 199.6 MB/s eta 0:00:00\n",
- "Downloading MarkupSafe-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)\n",
- "Downloading mistune-3.0.2-py3-none-any.whl (47 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.0/48.0 kB 175.0 MB/s eta 0:00:00\n",
- "Downloading nbclient-0.9.0-py3-none-any.whl (24 kB)\n",
- "Downloading nbformat-5.9.2-py3-none-any.whl (77 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.6/77.6 kB 155.8 MB/s eta 0:00:00\n",
- "Downloading pexpect-4.9.0-py2.py3-none-any.whl (63 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.8/63.8 kB 121.0 MB/s eta 0:00:00\n",
- "Downloading platformdirs-4.1.0-py3-none-any.whl (17 kB)\n",
- "Downloading prompt_toolkit-3.0.43-py3-none-any.whl (386 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 386.1/386.1 kB 61.6 MB/s eta 0:00:00\n",
- "Downloading pygments-2.17.2-py3-none-any.whl (1.2 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 50.5 MB/s eta 0:00:00\n",
- "Downloading QtPy-2.4.1-py3-none-any.whl (93 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 93.5/93.5 kB 111.2 MB/s eta 0:00:00\n",
- "Downloading requests-2.31.0-py3-none-any.whl (62 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 158.3 MB/s eta 0:00:00\n",
- "Downloading tensorboard_data_server-0.7.2-py3-none-any.whl (2.4 kB)\n",
- "Downloading werkzeug-3.0.1-py3-none-any.whl (226 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 226.7/226.7 kB 165.8 MB/s eta 0:00:00\n",
- "Downloading widgetsnbextension-4.0.9-py3-none-any.whl (2.3 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.3/2.3 MB 48.9 MB/s eta 0:00:00\n",
- "Downloading exceptiongroup-1.2.0-py3-none-any.whl (16 kB)\n",
- "Downloading jupyterlab_pygments-0.3.0-py3-none-any.whl (15 kB)\n",
- "Downloading nvidia_nvjitlink_cu12-12.3.101-py3-none-manylinux1_x86_64.whl (20.5 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20.5/20.5 MB 56.5 MB/s eta 0:00:00\n",
- "Downloading PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 705.5/705.5 kB 74.6 MB/s eta 0:00:00\n",
- "Downloading stack_data-0.6.3-py3-none-any.whl (24 kB)\n",
- "Downloading anyio-4.2.0-py3-none-any.whl (85 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 85.5/85.5 kB 189.3 MB/s eta 0:00:00\n",
- "Downloading asttokens-2.4.1-py2.py3-none-any.whl (27 kB)\n",
- "Downloading async_lru-2.0.4-py3-none-any.whl (6.1 kB)\n",
- "Downloading Babel-2.14.0-py3-none-any.whl (11.0 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.0/11.0 MB 58.1 MB/s eta 0:00:00\n",
- "Downloading cachetools-5.3.2-py3-none-any.whl (9.3 kB)\n",
- "Downloading certifi-2023.11.17-py3-none-any.whl (162 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 162.5/162.5 kB 196.7 MB/s eta 0:00:00\n",
- "Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 142.1/142.1 kB 200.4 MB/s eta 0:00:00\n",
- "Downloading executing-2.0.1-py2.py3-none-any.whl (24 kB)\n",
- "Downloading idna-3.6-py3-none-any.whl (61 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.6/61.6 kB 138.0 MB/s eta 0:00:00\n",
- "Downloading json5-0.9.14-py2.py3-none-any.whl (19 kB)\n",
- "Downloading jsonschema-4.20.0-py3-none-any.whl (84 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.7/84.7 kB 151.3 MB/s eta 0:00:00\n",
- "Downloading jupyter_events-0.9.0-py3-none-any.whl (18 kB)\n",
- "Downloading jupyter_lsp-2.2.1-py3-none-any.whl (66 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.0/66.0 kB 186.5 MB/s eta 0:00:00\n",
- "Downloading soupsieve-2.5-py3-none-any.whl (36 kB)\n",
- "Downloading terminado-0.18.0-py3-none-any.whl (14 kB)\n",
- "Downloading urllib3-2.1.0-py3-none-any.whl (104 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 104.6/104.6 kB 182.9 MB/s eta 0:00:00\n",
- "Downloading zipp-3.17.0-py3-none-any.whl (7.4 kB)\n",
- "Downloading argon2_cffi-23.1.0-py3-none-any.whl (15 kB)\n",
- "Downloading fastjsonschema-2.19.1-py3-none-any.whl (23 kB)\n",
- "Downloading jupyter_server_terminals-0.5.1-py3-none-any.whl (13 kB)\n",
- "Downloading overrides-7.4.0-py3-none-any.whl (17 kB)\n",
- "Downloading prometheus_client-0.19.0-py3-none-any.whl (54 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.2/54.2 kB 130.6 MB/s eta 0:00:00\n",
- "Downloading wcwidth-0.2.13-py2.py3-none-any.whl (34 kB)\n",
- "Downloading websocket_client-1.7.0-py3-none-any.whl (58 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.5/58.5 kB 158.5 MB/s eta 0:00:00\n",
- "Downloading attrs-23.2.0-py3-none-any.whl (60 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.8/60.8 kB 184.3 MB/s eta 0:00:00\n",
- "Downloading jsonschema_specifications-2023.12.1-py3-none-any.whl (18 kB)\n",
- "Downloading pyasn1-0.5.1-py2.py3-none-any.whl (84 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.9/84.9 kB 157.9 MB/s eta 0:00:00\n",
- "Downloading referencing-0.32.1-py3-none-any.whl (26 kB)\n",
- "Downloading rpds_py-0.16.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 74.3 MB/s eta 0:00:00\n",
- "Downloading cffi-1.16.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (443 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 443.9/443.9 kB 72.3 MB/s eta 0:00:00\n",
- "Downloading jsonpointer-2.4-py2.py3-none-any.whl (7.8 kB)\n",
- "Downloading uri_template-1.3.0-py3-none-any.whl (11 kB)\n",
- "Downloading arrow-1.3.0-py3-none-any.whl (66 kB)\n",
- " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.4/66.4 kB 138.4 MB/s eta 0:00:00\n",
- "Downloading types_python_dateutil-2.8.19.20240106-py3-none-any.whl (9.7 kB)\n",
- "Installing collected packages: webencodings, wcwidth, pytz, pure-eval, ptyprocess, mpmath, libclang, json5, flatbuffers, fastjsonschema, zipp, wrapt, widgetsnbextension, websocket-client, webcolors, urllib3, uri-template, tzdata, typing-extensions, types-python-dateutil, traitlets, tqdm, tornado, tomli, tinycss2, threadpoolctl, termcolor, tensorflow-io-gcs-filesystem, tensorflow-estimator, tensorboard-data-server, sympy, soupsieve, sniffio, six, send2trash, rpds-py, rfc3986-validator, pyzmq, PyYAML, python-json-logger, pyparsing, pygments, pycparser, pyasn1, psutil, protobuf, prompt-toolkit, prometheus-client, platformdirs, pillow, pexpect, parso, pandocfilters, packaging, overrides, oauthlib, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, numpy, networkx, nest-asyncio, mistune, MarkupSafe, markdown, kiwisolver, keras, jupyterlab-widgets, jupyterlab-pygments, jsonpointer, joblib, idna, grpcio, gast, fsspec, fqdn, fonttools, filelock, executing, exceptiongroup, defusedxml, decorator, debugpy, cycler, charset-normalizer, certifi, cachetools, babel, attrs, absl-py, werkzeug, triton, terminado, scipy, rsa, rfc3339-validator, requests, referencing, qtpy, python-dateutil, pyasn1-modules, pyaml, opt-einsum, nvidia-cusparse-cu12, nvidia-cudnn-cu12, ml-dtypes, matplotlib-inline, jupyter-core, jinja2, jedi, importlib-metadata, h5py, google-pasta, contourpy, comm, cffi, bleach, beautifulsoup4, awkward-cpp, async-lru, astunparse, asttokens, anyio, xgboost, stack-data, scikit-learn, requests-oauthlib, pandas, nvidia-cusolver-cu12, matplotlib, jupyter-server-terminals, jupyter-client, jsonschema-specifications, google-auth, awkward, arrow, argon2-cffi-bindings, uproot, torch, scikit-optimize, jsonschema, isoduration, ipython, google-auth-oauthlib, argon2-cffi, tensorboard, nbformat, ipywidgets, ipykernel, tensorflow, qtconsole, nbclient, jupyter-events, jupyter-console, nbconvert, jupyter-server, notebook-shim, jupyterlab-server, jupyter-lsp, jupyterlab, notebook, jupyter\n",
- "Successfully installed MarkupSafe-2.1.3 PyYAML-6.0.1 absl-py-2.0.0 anyio-4.2.0 argon2-cffi-23.1.0 argon2-cffi-bindings-21.2.0 arrow-1.3.0 asttokens-2.4.1 astunparse-1.6.3 async-lru-2.0.4 attrs-23.2.0 awkward-2.5.1 awkward-cpp-27 babel-2.14.0 beautifulsoup4-4.12.2 bleach-6.1.0 cachetools-5.3.2 certifi-2023.11.17 cffi-1.16.0 charset-normalizer-3.3.2 comm-0.2.1 contourpy-1.2.0 cycler-0.12.1 debugpy-1.8.0 decorator-5.1.1 defusedxml-0.7.1 exceptiongroup-1.2.0 executing-2.0.1 fastjsonschema-2.19.1 filelock-3.13.1 flatbuffers-23.5.26 fonttools-4.47.0 fqdn-1.5.1 fsspec-2023.12.2 gast-0.5.4 google-auth-2.26.1 google-auth-oauthlib-1.2.0 google-pasta-0.2.0 grpcio-1.60.0 h5py-3.10.0 idna-3.6 importlib-metadata-7.0.1 ipykernel-6.28.0 ipython-8.20.0 ipywidgets-8.1.1 isoduration-20.11.0 jedi-0.19.1 jinja2-3.1.2 joblib-1.3.2 json5-0.9.14 jsonpointer-2.4 jsonschema-4.20.0 jsonschema-specifications-2023.12.1 jupyter-1.0.0 jupyter-client-8.6.0 jupyter-console-6.6.3 jupyter-core-5.7.1 jupyter-events-0.9.0 jupyter-lsp-2.2.1 jupyter-server-2.12.2 jupyter-server-terminals-0.5.1 jupyterlab-4.0.10 jupyterlab-pygments-0.3.0 jupyterlab-server-2.25.2 jupyterlab-widgets-3.0.9 keras-2.15.0 kiwisolver-1.4.5 libclang-16.0.6 markdown-3.5.1 matplotlib-3.8.2 matplotlib-inline-0.1.6 mistune-3.0.2 ml-dtypes-0.2.0 mpmath-1.3.0 nbclient-0.9.0 nbconvert-7.14.0 nbformat-5.9.2 nest-asyncio-1.5.8 networkx-3.2.1 notebook-7.0.6 notebook-shim-0.2.3 numpy-1.26.3 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.18.1 nvidia-nvjitlink-cu12-12.3.101 nvidia-nvtx-cu12-12.1.105 oauthlib-3.2.2 opt-einsum-3.3.0 overrides-7.4.0 packaging-23.2 pandas-2.1.4 pandocfilters-1.5.0 parso-0.8.3 pexpect-4.9.0 pillow-10.2.0 platformdirs-4.1.0 prometheus-client-0.19.0 prompt-toolkit-3.0.43 protobuf-4.23.4 psutil-5.9.7 ptyprocess-0.7.0 pure-eval-0.2.2 pyaml-23.12.0 pyasn1-0.5.1 pyasn1-modules-0.3.0 pycparser-2.21 pygments-2.17.2 pyparsing-3.1.1 python-dateutil-2.8.2 python-json-logger-2.0.7 pytz-2023.3.post1 pyzmq-25.1.2 qtconsole-5.5.1 qtpy-2.4.1 referencing-0.32.1 requests-2.31.0 requests-oauthlib-1.3.1 rfc3339-validator-0.1.4 rfc3986-validator-0.1.1 rpds-py-0.16.2 rsa-4.9 scikit-learn-1.3.2 scikit-optimize-0.9.0 scipy-1.11.4 send2trash-1.8.2 six-1.16.0 sniffio-1.3.0 soupsieve-2.5 stack-data-0.6.3 sympy-1.12 tensorboard-2.15.1 tensorboard-data-server-0.7.2 tensorflow-2.15.0.post1 tensorflow-estimator-2.15.0 tensorflow-io-gcs-filesystem-0.35.0 termcolor-2.4.0 terminado-0.18.0 threadpoolctl-3.2.0 tinycss2-1.2.1 tomli-2.0.1 torch-2.1.2 tornado-6.4 tqdm-4.66.1 traitlets-5.14.1 triton-2.1.0 types-python-dateutil-2.8.19.20240106 typing-extensions-4.9.0 tzdata-2023.4 uproot-5.2.1 uri-template-1.3.0 urllib3-2.1.0 wcwidth-0.2.13 webcolors-1.13 webencodings-0.5.1 websocket-client-1.7.0 werkzeug-3.0.1 widgetsnbextension-4.0.9 wrapt-1.14.1 xgboost-2.0.3 zipp-3.17.0\n",
- "\n",
- "done\n",
- "#\n",
- "# To activate this environment, use\n",
- "#\n",
- "# $ conda activate machine-learning-das\n",
- "#\n",
- "# To deactivate an active environment, use\n",
- "#\n",
- "# $ conda deactivate\n",
- "\n",
- "Activating the kernel \"machine-learning-das\"...\n",
- "Installed kernelspec machine-learning-das in /home/cms.rkansal/.local/share/jupyter/kernels/machine-learning-das\n",
- "Installed \"machine-learning-das\"!\n"
- ]
- }
- ],
+ "outputs": [],
"source": [
"%%bash\n",
"#!/bin/bash\n",
@@ -698,29 +29,22 @@
"KERNEL_NAME=\"machine-learning-das\"\n",
"\n",
"# Download mambaforge install script\n",
- "if [[ ! -f $HOME/mambaforge.sh ]]; then\n",
- " wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh -O $HOME/mambaforge.sh || { echo \"Failed to download the mambaforge installation script.\"; exit $?; }\n",
+ "if [[ ! -f $HOME/miniconda3.sh ]]; then\n",
+ " wget \"https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh\" -O $HOME/miniconda3.sh || { echo \"Failed to download the miniforge installation script.\"; exit $?; }\n",
"fi\n",
"\n",
"# Install miniconda3\n",
- "if [[ ! -d $HOME/mambaforge ]]; then\n",
- " chmod u+x $HOME/mambaforge.sh\n",
- " bash $HOME/mambaforge.sh -b -f -u -p $HOME/mambaforge\n",
+ "if [[ ! -d $HOME/miniconda3 ]]; then\n",
+ " bash $HOME/miniconda3.sh -b -f -u -p $HOME/miniconda3\n",
"fi\n",
"\n",
- "echo \"Sourcing the conda and mamba setup scripts...\"\n",
- "source $HOME/mambaforge/etc/profile.d/conda.sh\n",
- "source $HOME/mambaforge/etc/profile.d/mamba.sh\n",
+ "# echo \"Sourcing the conda and mamba setup scripts...\"\n",
+ "source $HOME/miniconda3/etc/profile.d/conda.sh\n",
"\n",
- "# # Not updating miniconda since that results in an issues with mamba \n",
- "# # https://github.com/mamba-org/mamba/issues/1775\n",
- "# echo \"Updating the miniconda3 release...\"\n",
- "# conda update -y -n base -c defaults conda 2>&1 || { echo \"Failed to properly update miniconda3.\"; exit $?; }\n",
- "\n",
- "# Create conda environment\n",
- "if [[ ! -d $HOME/mambaforge/envs/$KERNEL_NAME ]]; then\n",
+ "# # Create conda environment\n",
+ "if [[ ! -d $HOME/miniconda3/envs/$KERNEL_NAME ]]; then\n",
" echo \"Creating the miniconda3 environment \\\"$KERNEL_NAME\\\"...\"\n",
- " mamba env create -n $KERNEL_NAME -f environment.yml || { echo \"Failed to properly setup the environment \\\"$KERNEL_NAME\\\" using mamba.\"; exit $?; }\n",
+ " conda env create -n $KERNEL_NAME -f environment.yml || { echo \"Failed to properly setup the environment \\\"$KERNEL_NAME\\\" using mamba.\"; exit $?; }\n",
"fi\n",
"echo \"Activating the kernel \\\"$KERNEL_NAME\\\"...\"\n",
"conda activate $KERNEL_NAME || { echo \"Failed to activate the kernel \\\"$KERNEL_NAME\\\".\"; exit $?; }\n",
diff --git a/machine-learning-das/setup/lpc.md b/machine-learning-das/setup/lpc.md
index 1635e6a..2a546e3 100644
--- a/machine-learning-das/setup/lpc.md
+++ b/machine-learning-das/setup/lpc.md
@@ -3,16 +3,43 @@
This is possible, but with a little less user support. You will need to use the following command to open up your ssh connection:
```bash
-ssh -L localhost:8888:localhost:8888 @cmslpc-sl7.fnal.gov
+ssh -L localhost:8888:localhost:8888 @cmslpc-el9.fnal.gov
```
Replace `` with your LPC username. Then `cd` to the directory of your choice where you will clone the repository as before:
```bash
git clone https://github.com/FNALLPC/machine-learning-das
```
-In order to open Jupyter with all the appopriate libraries, you will need to have either installed a `conda` environment with Jupyter in your `nobackup` area (where you have more space) or have a CMSSW environment setup. To install and activate a `conda` environment you can do:
+## `venv`
+Firstly, to avoid filling up the quota in your home directory, you can change the location of cache directory used by `pip` by running
```bash
-wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O $HOME/nobackup/miniconda3.sh
+pip3 config set global.cache-dir ~/nobackup/.cache
+```
+Create and activate a virtual Python virtual environment by running
+```bash
+python3 -m venv venv
+source venv/bin/activate
+```
+Install the required packages in this virtual environment
+```bash
+cd machine-learning-das
+pip3 install -r requirements.txt
+```
+Open Jupyter
+```bash
+jupyter notebook --no-browser --port=8888 --ip 127.0.0.1
+```
+If everything worked, the last line of the output should be a url of the form:
+```bash
+http://127.0.0.1:8888/?token=
+```
+Copy this url into your browser.
+
+## Miniforge
+
+Alternatively, in order to open Jupyter with all the appopriate libraries, you can use a `conda` environment with Jupyter in your `nobackup` area (where you have more space). To install and activate a `conda` environment you can do:
+```bash
+wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh" -O $HOME/nobackup/miniconda3.sh
bash $HOME/nobackup/miniconda3.sh -b -f -u -p $HOME/nobackup/miniconda3
source $HOME/nobackup/miniconda3/etc/profile.d/conda.sh
conda env create -f environment.yml