R_to_python.Rmd

---
title: "R to Python for Data Analysis"
author: "Leonardo Uchoa"
date: "3/31/2020"
output: 
  pdf_document:
    toc: true
    number_sections: true
header-includes:
- \usepackage{amsbsy}
- \usepackage{amsmath}
- \usepackage{float}
- \usepackage{graphicx}
---

\newpage

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

# The tables

That's my R to python port. It's intended to make my approach to learning python faster and its mostly composed of data wrangling routine tools. Many of those are already listed in other sources.

<!-- -------Data wrangling------- -->

\begin{table}[H]
\centering
\caption{Data Wrangling}
\begin{tabular}{lr}
R & Python \\
\hline
dim & df.shape (pd) \\
stop & raise ValueError \\
str & df.dtypes / df.info (pd) \\
unique & np.unique (np) \\
sort & np.sort (np) \\
rbind & np.hstack (np) \\
summary & df.describe (pd) \\
\texttt{group\_by} & df.groupby (pd) \\
count & \verb|df.value_count| (pd) \\
table & np.bincount (np array) \\
apply & df.apply (pd) \\
if.else & df.where[case,true,false] (pd)  \\
table & pd.crosstab \\
corr & np.corrcoef (np) \\
mutate(df, c=a-b) & df.assign(c=df['a']-df['b']) (pd) \\
colSums(is.na()) & df.isnull().sum() (pd) \\
na.omit & df.dropna(axis=X) (pd) \\
*imputation* & df.fillna(df.mean()) (pd) \\
colnames() <- & df.colnames (pd) \\
\%in\% & df.isin(list_of_elements) (pd)
\end{tabular}
\end{table}

<!-- -------Plotting------- -->


\begin{table}[H]
\centering
\caption{Plotting facilities}
\begin{tabular}{lr}
base R & matplotlib \\
\hline \\
pairs & scatterplotmatrix (mlext subm) \\
heatmap & heatmap (mlext subm)\\
image & np.imshow(img,cmap = "Color") (plt)\\
\end{tabular}
\end{table}


<!-- -------Linear Algebra------- -->

\begin{table}[H]
\centering
\caption{Linear Algebra}
\begin{tabular}{lr}
R & Python \\
\hline
eigen & np.linalg.eig (np) \\
\%*\% & np.dot \\
----- & np.matmul \\
\end{tabular}
\end{table}

<!-- -------Abreviations------- -->

\begin{table}[H]
\centering
\caption{Abreviations}
\begin{tabular}{lr}
Module Abreviation & Module \\
\hline
pd & Pandas \\
np & Numpy \\
plt & matplotlib \\
---- & mlextend \\
\end{tabular}
\end{table}

\newpage 

\begin{center}
\section*{Usefull examples}
\end{center}
\addtocounter{section}{0}


These are convertions of the commands I use the most and some other ( because they're different from what I'm used to do in R) when analysing data in R.

I wrote this document using Rstudio and Rmarkdown. So in order to load python within R the thing to do was to use `reticulate`. But  I've also installed all my python packages using anaconda because it handles compatability between libraries more efficiently. In that case you can load `reticulate` and set it's path with `use_python` for it to load the packages installed via anaconda \footnote{The paths are \textbf{not} the same}.

```{r}
library(reticulate)
use_python("/home/leonardo/anaconda3/bin/python")
```

```{python}
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
```


\begin{center}
\section{Numpy Broadcasting}
\end{center}

From [numpy's manual](https://numpy.org/doc/stable/user/basics.broadcasting.html) broadcasting in python is a way to "describe how numpy treats arrays with different shapes during arithmetic operations. Subject to certain constraints, the smaller array is 'broadcast' across the larger array so that they have compatible shapes". 

 - The good thing is that it allows for internal C looping instead of running in python, which is more efficient.
 - The bad thing is that sometimes it leads to some inefficiencies

But it's different from what happens in R in the sense that's less harmfull. In we complete the smaller vector and can (and wp1 will) eventually cause some disasters. But in after some update at least we receive a warning message, so stay alert!

```{r}
a = 1:10
b = 1:5
a + b 
```

In python receive an error message instead of a warning, which is better. 

```{python}
a = np.array([1,2,3,4,5])
b = np.array([1,2])

print(a + b)

print(a * b)

# but adding a scalar to an array is cool and allowed

print(a + 100)
```

But the uncool thing still remais and some random NaNs may appear.

```{r}
B = pd.DataFrame([[1,2,3],[1,2,3]])
C = pd.DataFrame([100,200])

print(B + C)
print(C + B)
print(B + C.T)
```


Even so broadcasting in python allows us to do some cool things, like computing row/column-wise percentages. For example:

```{python}
A = pd.DataFrame([[np.random.rand() for _ in range(5)] for _ in range(5)])
rowwise_total = A.sum(axis = 0)
print(100 * A/rowwise_total)
```

As some general rules if A is (m,n) and B is (1,n) or (m,1) then `A +,-,*,/ B` will make the result be of dimension (m,n)

\begin{center}
\section{Counting per column}
\end{center}


\textbf{\underline{Source}:} StackOverflow.

```{python}

df = pd.DataFrame(np.random.randint(0, 2, (10, 4)), columns=list('abcd'))
df.apply(pd.Series.value_counts)
```

\begin{center}
\section{Multi argument iteration with zip}
\end{center}

Zip allows us to construct tuples for iterating over multiple arguments.

```{python}
players = [ "Sachin", "Sehwag", "Gambhir", "Dravid", "Raina" ] 
scores = [100, 15, 17, 28, 43 ] 

# Lets see how it constructs the tuples

print(tuple(zip(players, scores)))

# Now we just need to iterate over them

for pl, sc in zip(players, scores): 
    print ("Player :  %s     Score : %d" %(pl, sc)) 
```


\begin{center}
\section{Categorical data encoding}
\end{center}

\textbf{\underline{Source}:} Chapter 4 of Python Machine Learning [2].


For this section we're working the toy data bellow

```{python}

df = pd.DataFrame([
['green', 'M', 10.1, 'class2'],
['red', 'L', 13.5, 'class1'],
['blue', 'XL', 15.3, 'class2']])
df.columns = ['color', 'size', 'price', 'classlabel']

df
```

In both approaches bellow we use a dictionary the create the mapping identifier for the `map` method. Remember that according to [w3schools](https://www.w3schools.com/python/python_dictionaries.asp) a dictionary is 

>*A dictionary is a collection which is unordered, changeable and indexed. In Python dictionaries are written with curly brackets, and they have keys and values.*

## Encoding ordinals - create labels manually

```{python}
#create the dict mapping from ordinal to integer
size_mapping = {'XL': 3,'L': 2,'M': 1}

#use map to in the desired column get the mapped values
df['size'] = df['size'].map(size_mapping)
```

## Encoding nominals - creating labels automatically

```{python}
class_mapping = {label: idx for idx, label in enumerate(np.unique(df['classlabel']))}
```

Now what that command is doing is looping through the iterators `idx` and `label` (created by the `enumerate` function) in the unique values of the `classlabel` column and assigning both to `label` and `idx`. Let's see

```{python}
print(list(
    enumerate(np.unique(df['classlabel']))
  ))
```

So iterating through the list we get to assign "class1"/"class2" to `label` and 0/1 to `idx`\footnote{Note the inversion in `label: idx for idx, label` }. Finally the last step to map

```{python}
df['classlabel'] = df['classlabel'].map(class_mapping)
df
```

Want to get the mapping backwards? Access the `items` method in the `class_mapping` object and loop again

```{python}
inv_class_mapping = {a:b for b,a in class_mapping.items()}
df['classlabel'] = df['classlabel'].map(inv_class_mapping)
df
```


**Ps.: There's also an object in `skitlearn` module `preprocessing` that does this: `LabelEncoder`**

## Creating Dummies in Design Matrix

\underline{First way: Pandas }

Now to create dummy variables for the design matrix, there's a simple way using pandas


```{python}
df_dm = pd.get_dummies(df[['price', 'color', 'size']], drop_first = True)
df_dm
```

The `drop_first = True` is important. Otherwise we woud get another column name "color_blue" with an $(0,0,1)^{T}$ entry which we do not need because when the other 2 columns are 0 we already encode the blue color. 

\underline{Second way: sklearns' OneHotEncoder}

The OneHotEncoder function arguments are (name, transformer,
columns) tuples

```{python}

from sklearn.preprocessing import OneHotEncoder
from sklearn.compose import ColumnTransformer

#creat the object
color_ohe = OneHotEncoder(categories='auto', drop='first')
c_transf = ColumnTransformer([
('onehot', color_ohe, [0]), # respectively 'transformer_name',transformer_object,column
('nothing', 'passthrough', [1, 2]) # respectively 'action_1','action_2', columns for action_1/2
])

#note that the function input is an np array
c_transf.fit_transform(df[['color', 'size', 'price']].values).astype(float)
```

\newpage


\begin{center}
\section{Train-test Split and basic pre-process}
\end{center}

\underline{Source:} Chapter 4 of Python Machine Learning [2].

## Train-test Split

```{python}
from sklearn import datasets
from sklearn.model_selection import train_test_split

wine = datasets.load_wine()
wine.data.shape
wine.target.shape

X = wine.data
y = wine.target

X_train, X_test, y_train, y_test = train_test_split(X,y,
test_size=0.3, # split
random_state=0, #  set.seed
stratify=y) #stratification of data based on target frequencies 

```

## Basic Continuous pre-process

```{python, eval = FALSE}
from sklearn.preprocessing import MinMaxScaler
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA

stdsc = StandardScaler()
X_train_std = stdsc.fit_transform(X_train)
X_test_std = stdsc.transform(X_test)

mms = MinMaxScaler()
X_train_mms = mms.fit_transform(X_train)
X_test_mms = mms.transform(X_test)

pca = PCA(n_components=2)
X_train_pca = pca.fit_transform(X_train_std)
X_test_pca = pca.transform(X_test_std)

lda = LDA(n_components=2)
X_train_lda = lda.fit_transform(X_train_std, y_train)

```


\newpage 

\begin{center}
\section{Basic Pipeline}
\end{center}

## Cross Validation

```{python, eval = FALSE}
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression

pipe_lr = make_pipeline(StandardScaler(),
  PCA(n_components=2),
  LogisticRegression(random_state=1,
  solver='lbfgs'))

# First way

from sklearn.model_selection import StratifiedKFold

kfold = StratifiedKFold(n_splits=10).split(X_train, y_train)
scores = []

for k, (train, test) in enumerate(kfold):
  pipe_lr.fit(X_train[train], y_train[train])
  score = pipe_lr.score(X_train[test], y_train[test])
  scores.append(score)
  print('Fold: %2d, Class dist.: %s, Acc: %.3f' % (k+1,
    np.bincount(y_train[train]), score))
    
# Second and Better way

from sklearn.model_selection import cross_val_score

scores = cross_val_score(estimator=pipe_lr,
  X=X_train,y=y_train,cv=10,n_jobs=1)
  
print('CV accuracy scores: %s' % scores)
```

## Sample Size Learning Curves

One thing to note here is, according to [2]

>Note that we passed max_iter=10000 as an additional argument when instantiating the LogisticRegression object (which uses 1,000 iterations as a default) to avoid convergence issues for the smaller dataset sizes or extreme regularization parameter
values.

Which can be valuable for other algorithms too. Note also that stratified k-fold CV is the default routine.

```{python, eval = FALSE}
import matplotlib.pyplot as plt
from sklearn.model_selection import learning_curve


pipe_lr = make_pipeline(StandardScaler(),
LogisticRegression(penalty='l2',
random_state=1,
solver='lbfgs',
max_iter=10000))

train_sizes, train_scores, test_scores =\ learning_curve(estimator=pipe_lr,
X=X_train,
y=y_train,
train_sizes=np.linspace(
0.1, 1.0, 10),
cv=10,
n_jobs=1)

train_mean = np.mean(train_scores, axis=1)
train_std = np.std(train_scores, axis=1)
test_mean = np.mean(test_scores, axis=1)
test_std = np.std(test_scores, axis=1)


plt.plot(train_sizes, train_mean, color='blue', marker='o',
markersize=5, label='Training accuracy')

plt.fill_between(train_sizes,train_mean + train_std, train_mean - train_std,alpha=0.15, color='blue')

plt.plot(train_sizes, test_mean,color='green', linestyle='--', marker='s', markersize=5,label='Validation accuracy')

plt.fill_between(train_sizes,
test_mean + test_std,
test_mean - test_std,
alpha=0.15, color='green')

plt.grid()
plt.xlabel('Number of training examples')
plt.ylabel('Accuracy')
plt.legend(loc='lower right')
plt.ylim([0.8, 1.03])
plt.show()
```


## Validation Curves

Here stratified k-fold CV is the default routine.

```{python, eval = FALSE}

from sklearn.model_selection import validation_curve

param_range = [0.001, 0.01, 0.1, 1.0, 10.0, 100.0]

train_scores, test_scores = validation_curve(
estimator=pipe_lr,
X=X_train,
y=y_train,
param_name='logisticregression__C',
param_range=param_range,
cv=10)

train_mean = np.mean(train_scores, axis=1)
train_std = np.std(train_scores, axis=1)
test_mean = np.mean(test_scores, axis=1)
test_std = np.std(test_scores, axis=1)

plt.plot(param_range, train_mean,
color='blue', marker='o',
markersize=5, label='Training accuracy')
plt.fill_between(param_range, train_mean + train_std,
train_mean - train_std, alpha=0.15,
color='blue')
plt.plot(param_range, test_mean,
color='green', linestyle='--',
marker='s', markersize=5,
label='Validation accuracy')

plt.fill_between(param_range,
test_mean + test_std,
test_mean - test_std,
alpha=0.15, color='green')

plt.grid()
plt.xscale('log')
plt.legend(loc='lower right')
plt.xlabel('Parameter C')
plt.ylabel('Accuracy')
plt.ylim([0.8, 1.0])
plt.show()
```

## Tuning hyperparameters via grid search

Here we're grid searching hyperparameters for an SVM with two different kernel functions and their respective parameters\footnote{Using RandomizedSearchCV in scikit-learn, we can perform Randomized Grid Search. See [3]}. Thats is: for the linear basis function we search for the range and for the rbf we search the range and gamma.

Here we have two dictionaries: 

- `{'svc__C': param_range,'svc__kernel': ['linear']}` and

- `{'svc__C': param_range,'svc__gamma': param_range,'svc__kernel': ['rbf']}`

who compose our grids.

```{python, eval = FALSE}
from sklearn.model_selection import GridSearchCV
from sklearn.svm import SVC

pipe_svc = make_pipeline(StandardScaler(),SVC(random_state=1))

param_range = [0.0001, 0.001, 0.01, 0.1,1.0, 10.0, 100.0, 1000.0]

param_grid = [{'svc__C': param_range,'svc__kernel': ['linear']},
{'svc__C': param_range,'svc__gamma': param_range,
'svc__kernel': ['rbf']}]

gs = GridSearchCV(estimator=pipe_svc,param_grid=param_grid
scoring='accuracy',cv=10,refit=True,n_jobs=-1)

gs = gs.fit(X_train, y_train)

clf = gs.best_estimator_
clf.fit(X_train, y_train)
print('Test accuracy: %.3f' % clf.score(X_test, y_test))

```

As note in [2] a great thing to keep in mind is

> Please note that fitting a model with the best settings ( gs.best_estimator_ ) on the training set manually via clf.fit(X_train, y_train) after completing the grid search is not necessary. The GridSearchCV class has a refit parameter, which
will refit the gs.best_estimator_ to the whole training set automatically if we set refit=True (default).

## Nested Cross Validation

See [4]. Following [2], chapter 6, let's compare a decision tree and an svc.

```{python, eval= FALSE}

from sklearn.tree import DecisionTreeClassifier

gs_svc = GridSearchCV(estimator=pipe_svc,param_grid=param_grid,scoring='accuracy',cv=2)
scores_svc = cross_val_score(gs_svc, X_train, y_train,scoring='accuracy', cv=5)

gs_tree = GridSearchCV(estimator=DecisionTreeClassifier(random_state=0),
param_grid=[{'max_depth': [1, 2, 3,4, 5, 6,7, None]}],scoring='accuracy',cv=2)

scores_tree = cross_val_score(gs_tree, X_train, y_train,scoring='accuracy',cv=5)

```

\newpage

\begin{center}
\section{Performance Evaluation Metrics}
\end{center}


```{python, eval= FALSE}
from sklearn.metrics import confusion_matrix

y_pred = pipe_svc.predict(X_test)
confmat = confusion_matrix(y_true=y_test, y_pred=y_pred)

```

```{python, eval= FALSE}
from sklearn.metrics import precision_score
from sklearn.metrics import recall_score, f1_score
from sklearn.metrics import roc_curve, auc, roc_auc_score

precision_score(y_true=y_test, y_pred=y_pred)
f1_score(y_true=y_test, y_pred=y_pred)

```

## Making our own scorer

```{python, eval= FALSE}
pre_scorer = make_scorer(score_func=precision_score,pos_label=1, greater_is_better=True,average='micro')
```

## Resampling for Class Imbalances

```{python, eval= FALSE}
from sklearn.utils import resample

X_upsampled, y_upsampled = resample(X_imb[y_imb == 1],y_imb[y_imb == 1],replace=True,n_samples=X_imb[y_imb == 0].shape[0],random_state=123)

X_bal = np.vstack((X[y == 0], X_upsampled))
y_bal = np.hstack((y[y == 0], y_upsampled))
```


Alternatives:

- SMOTE;

- `imbalanced-learn` - see [5]


\newpage
\begin{center}
\section{Misc}
\end{center}


### Unzipping tarballs within Python and Working with the `with` expression

The `with` expressin is a really nice feature of python as it is the responsible for setting things up for task you want to do and then clean things down. A classic and very used example of it is to open a file and do some things. 

In order to open a file we must first to establish a connection with it. After this, we do what we gotta do and then must close the file. It looks like this

```{python, eval=FALSE}
file = open("file_name")

do something #eg file.read()

file.close()
```

Now using `with` it becomes 

```{python, eval=FALSE}
with open("file_name") as file:
  do something
```

The `with` expression sets the connection, waits for you to do your task and even close it for you. Another example is extracting tarballs, is illustrated as follows.

```{python, eval = FALSE}

import tarfile
with tarfile.open('file_name.tar.gz', 'r:gz') as tar:
  tar.extractall()

```

But how does it works? It's very well explained in [7] and I recommend it.

## Sourcing Python Files

\textbf{\underline{Source}:} StackOverflow  See [6]

```{python, eval = FALSE}
exec(open("NeuralNetMLP.py").read())
os.system("file.py") # need to import os
```

## Serializing Objects

The mainstream way is to use `pickle`. It can be done with `joblib`, but it is best suited for scheduling tasks, actually.

```{python, eval = FALSE}
import pickle

filename = 'object.sav'
pickle.dump(nn, open(filename, 'wb'))

loaded_object = pickle.load(open(filename, 'rb'))
loaded_object

import joblib

filename = 'object.sav'
joblib.dump(model, filename)
loaded_object = joblib.load(filename)
```

## Plotting with Seaborn in the Console

In order to display seaborn plots without some IDE that automatically handles plotting, you need to also call the matplotlib's pyplot submodule function `show`, as shown below.

```{python}
import seaborn as sb
import matplotlib.pyplot as plt

iris = sb.load_dataset('iris')

sb.relplot(data='iris',x='sepal_length',y='petal_length')
plt.show()
```

## Modules, projects, packages 

- **Module**: is just a file ending with a `.py` extension
- **Package**: usually is a directory containing at aleast a module *and* a file called __init__.py. 
    - When the __init__.py file is imported, the code within it will be run. So it is used to import other files (modules)
    - Specifying the __init__.py file: simply put you're importing modules, so the standard *import foo from asd as bla* works well. But you can also upgrade your init file to support more complex tasks, like unit testing. For example see the scikit-learn [linear model submodule](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/__init__.py).
- **Projects**: is a collection of packages

**Important**: If you're creating a python library, you **must** specify the __init__ files because they are the reference to what's being imported. Otherwise when importing the lib, python won't find the modules you want. See [this](https://stackoverflow.com/a/14132912)

## Python Programming nuggets

- [Assertions vs Exceptions](https://stefan.sofa-rockers.org/2018/04/16/assertions-and-exceptions/)
- [Solved!! UnboundLocalError: local variable referenced before assignment](https://careerkarma.com/blog/python-local-variable-referenced-before-assignment/)
- Python Packages [good overview](https://python-packaging-tutorial.readthedocs.io/en/latest/setup_py.html), a [video](https://www.youtube.com/watch?v=wCGsLqHOT2I), a [simplified version](https://oneraynyday.github.io/dev/2017/08/02/Python-File-Structure/) and more [nice explanations](https://docs.python-guide.org/writing/structure/)
- [Using wheel to create packages](https://hynek.me/articles/sharing-your-labor-of-love-pypi-quick-and-dirty/)
- [Automated Jyputers as altenative do automated Rmds](https://medium.com/capital-fund-management/automated-reports-with-jupyter-notebooks-using-jupytext-and-papermill-619e60c37330)
- [Kickstarting projects with cookiecutter](https://sourcery.ai/blog/python-best-practices/)
- [parsing dates with pandas](https://towardsdatascience.com/4-tricks-you-should-know-to-parse-date-columns-with-pandas-read-csv-27355bb2ad0e)
- [god bless `.format`!](https://pyformat.info/)
- [arrow function???? `def myfn() -> ??` not really, just documentation](https://medium.com/@thomas_k_r/whats-this-weird-arrow-notation-in-python-53d9e293113)

\section{References}

[1]. Pandas: https://pandas.pydata.org/pandas-docs/stable/getting_started/comparison/comparison_with_r.html#quick-reference

[2]. Raschka, S. and Mirjalili, V., 2019. Python Machine Learning. Birmingham: Packt Publishing, Limited.

[3]. Random search for hyper-parameter optimization. Bergstra J, Bengio Y. Journal of Machine Learning Research. pp. 281-305, 2012

[4]. Bias in Error Estimation When Using Cross-Validation for Model Selection, BMC Bioinformatics, S. Varma and R. Simon, 7(1): 91, 2006

[5]. https://github.com/scikit-learn-contrib/imbalanced-learn .

[6]. https://stackoverflow.com/a/6357529

[7]. https://effbot.org/zone/python-with-statement.htm