From ef3e47df923f2ba59d88b73759b5348c64759e82 Mon Sep 17 00:00:00 2001
From: dariusamiri <59167222+dariusamiri@users.noreply.github.com>
Date: Sat, 8 Jan 2022 00:18:34 +0330
Subject: [PATCH 01/16] Create index.md
---
notebooks/18_deep_neural_network/index.md | 201 ++++++++++++++++++++++
1 file changed, 201 insertions(+)
create mode 100644 notebooks/18_deep_neural_network/index.md
diff --git a/notebooks/18_deep_neural_network/index.md b/notebooks/18_deep_neural_network/index.md
new file mode 100644
index 00000000..864df4b7
--- /dev/null
+++ b/notebooks/18_deep_neural_network/index.md
@@ -0,0 +1,201 @@
+# Deep Neural Networks
+
+## Table of Content
+
+- [Introduction](#introduction)
+- [CNN Architecture](#CNN-Architecture)
+- [Conv Layer](#Conv-Layer)
+- [Pooling](#Pooling)
+- [Padding](#Padding)
+- [Stride](#Stride)
+- [Fully Connected Layer](#Fully-Connected-Layer)
+- [Activation Function](#Activation-Function)
+- [Loss Functions](#Loss-Functions)
+- [The Output Size of Conv Layer](#The-output-size-of-Conv-Layer)
+- [Conclusion](#Conclusion)
+- [References](#References)
+
+
+## Introduction
+Deep learning is a subfield of machine learning that deals with algorithms inspired by the structure and function of the brain. Deep learning is a subset of machine learning, which is a part of artificial intelligence (AI).
+
+
+CNN's are models to solve deep learning problems. Suppose that you have high-dimensional inputs such as images or videos. If we want to use MLPs, 2 (or more) dimensional inputs need to be converted to 1-dimensional vectors. This conversion increases the number of trainable parameters exponentially. Also, one important thing in these data is locality, it means that for example in an image, you can find features in near pixels (for examples corners and edges) but, far pixels can't give you efficient features. The solution for solving these problems is using CNNs.
+
+## CNN Architecture
+A CNN architecture is formed by a stack of distinct layers that transform the input volume into an output volume through a differentiable function. A few distinct types of layers are commonly used:
+
+* Fully Connected Layer
+* Convolutional layer
+* Pooling
+* Activation Function
+* Loss layer
+
+In the following, you can see images of two simple famous CNNs.
+
+
+
+
+
+
+
+
+
+
+## Fully Connected Layer
+
+Fully Connected layers in neural networks are those layers where all the inputs from one layer are connected to every activation unit of the next layer. In most popular CNN models, the last few layers are fully connected layers that compile the data extracted by previous layers to form the final output. It is the second most time-consuming layer second to the Convolution Layer.
+
+
+## Conv Layer
+This layer is the main difference between CNNs and MLPs. Convolution in the word refers to two operators between two functions. In mathematics convolution define as below:
+
+In this image, you can see an example for functions of convolution and two other similar operators.
+
+
+
+
+
+
+
+
+Here we’ll not talk about details, but convolutional layers are somehow enabling convolution operator on sub-matrices of the image. These layers have formed from some kernel with the same height, width, and depth. The number of these kernels is equal to the depth of the output. Also, the depth of each kernel must be equal to the depth of input. For example, if you have RGB data, your first convolutional layer kernels depth must be 3.
+In the context of a convolutional neural network, convolution is a linear operation that involves the multiplication of a set of weights with the input. A convolution layer has formed by 1 or more of these operations that each of them called a kernel. All kernels have the same height, width, and depth. To find the output of the layer, we put the first kernel on the top-right of the input and calculate the output of the kernel, and put it as the first cell of a matrix. After that, we move it to right and calculate again, and put the result in the second cell. When we receive to end of columns, we move the kernel down. we do this until we rich to the end of the image. We do this for all kernels and this is how we make the output of the convolutional layer.
+
+
+
+
+
+
+
+
+
+
+
+
+## Pooling
+
+Similar to the Convolutional Layer, the Pooling layer is responsible for reducing the spatial size of the Convolved Feature.
+While a lot of information is lost in the pooling layer, it also has a number of benefits to the Convolutional neural network. They help to reduce complexity, improve efficiency, and limit the risk of overfitting.
+This is to decrease the computational power required to process the data by reducing the number of parameters in the input.
+It sweeps a filter across the entire input but that does not have any weights. Instead, the kernel applies an aggregation function to the values within the receptive field, populating the output array.
+There are two types of Pooling:
+
+1. Max Pooling: it returns the maximum value from the portion of the image covered by the Kernel. and also performs as a Noise Suppressant. It discards the noisy activations altogether and also performs de-noising along with dimensionality reduction.
+2. Average Pooling: it returns the average of all the values from the portion of the image covered by the Kernel. and simply performs dimensionality reduction as a noise suppressing mechanism.
+Comparing these two, we can say that Max Pooling performs a lot better than Average Pooling.
+In the following, we will see a picture of how these two methods work.
+
+
+
+
+## Padding
+
+As you see, after applying convolutional layers, the size of the feature map is always smaller than the input, we have to do something to prevent our feature map from shrinking. This is where we use padding. Layers of zero-value pixels are added to surround the input with zeros so that our feature map will not shrink. By padding, we can control the shrinking of our inputs.
+Different padding mode is:
+* zeros(Default)
+* reflect
+* replicate or circular
+
+
+
+
+
+
+
+
+## Stride
+As we said before, when you're applying a kernel to the image, you have to move the kernel during the image. But sometimes you prefer to not move one pixel every time and move the kernel more than one pixel. This is stride. The stride specifies how many kernels have to move each time.
+
+
+
+
+
+
+
+## Activation Function
+
+An activation function is a function that is added to an artificial neural network in order to help the network learn complex patterns in the data. When compared with a neuron-based model that is in our brains, the activation function is at the end deciding what is to be fired to the next neuron. That is exactly what an activation function does in a neural network as well. It takes in the output signal from the previous cell and converts it into some form that can be taken as input to the next cell. The comparison can be summarized in the figure below.
+
+
+The most important feature in an activation function is its ability to add non-linearity into a neural network.
+Here we describe six famous activation functions:
+- **Relu(Rectified Linear Unit)** :
+The rectified linear activation function or ReLU for short is a piecewise linear function that will output the input directly if it is positive, otherwise, it will output zero. It has become the default activation function for many types of neural networks because a model that uses it is easier to train and often achieves better performance.
+- **Leaky ReLU**:
+It is an attempt to solve the dying ReLU problem. Usually, the value of the slope in the negative range is 0.01 or so.
+- **Sigmoid** :
+The main reason why we use the sigmoid function is that it exists between (0 to 1). Therefore, it is especially used for models where we have to predict the probability as an output. Since the probability of anything exists only between the range of 0 and 1, sigmoid is the right choice. The function is differentiable. That means, we can find the slope of the sigmoid curve at any two points. The function is monotonic but the function’s derivative is not.
+- **Tanh or hyperbolic tangent Activation Function** :
+The advantage is that the negative inputs will be mapped strongly negative and the zero inputs will be mapped near zero in the tanh graph. The function is differentiable. The function is monotonic while its derivative is not monotonic. The tanh function is mainly used in classification between two classes.
+- **Maxout** :
+The Maxout unit is an activation function that is itself trained by our model. A single Maxout unit can be interpreted as making a piecewise linear approximation to an arbitrary convex function. A Maxout unit takes the maximum value among the values from n linear functions. The number of linear functions is determined beforehand.
+- **ELU** :
+Exponential Linear Unit or its widely known name ELU is a function that tends to converge cost to zero faster and produce more accurate results. ELU is very similar to RELU except for negative inputs. They are both in identity function form for non-negative inputs. On the other hand, ELU becomes smooth slowly until its output is equal to -α whereas RELU sharply smoothes.
+
+
+
+## Loss Functions
+As part of the optimization algorithm, the error for the current state of the model must be estimated repeatedly. This requires the choice of an error function, conventionally called a loss function, that can be used to estimate the loss of the model so that the weights can be updated to reduce the loss on the next evaluation. These are two famous loss functions:
+
+* **Mean Squared Error Loss**:
+Mean squared error is calculated as the average of the squared differences between the predicted and actual values. The result is always positive regardless of the sign of the predicted and actual values and a perfect value is 0.0. The squaring means that larger mistakes result in more errors than smaller mistakes, meaning that the model is punished for making larger mistakes.
+* **Cross-Entropy**:
+Cross-entropy is a measure of the difference between two probability distributions for a given random variable or set of events. It is closely related to but is different from KL divergence that calculates the relative entropy between two probability distributions, whereas cross-entropy can be thought to calculate the total entropy between the distributions.
+
+## The output size of Conv Layer
+
+in_channels: Number of channels in the input image
+out_channels: Number of channels produced by the convolution
+kernel_size: Size of the convolving kernel
+stride: Stride of the convolution. Default: 1
+padding: Padding added to all four sides of the input. Default: 0
+dilation: Spacing between kernel elements. Default: 1
+
+
+$H_{out} = \lfloor \frac{H_{in} + 2×padding[0]−dilation[0]×(kernel_size[0]−1)−1 }{stride[0]} \rfloor + 1$
+
+$W_{out} = \lfloor \frac{W_{in} + 2×padding[1]−dilation[1]×(kernel_size[1]−1)−1 }{stride[1]} \rfloor + 1$
+
+
+## Conclusion
+Interest in deep learning is continuing to increase. Reasons for this interest include deep learning’s capacity to
+
+1. Improve accuracy of predictions, enabling improved data-driven decisions
+2. Learn from unstructured and unlabelled datasets, enable an analysis of unstructured data
+
+The number of times a phrase is searched on a search engine is a proxy for its popularity. You can see the frequency with which “Deep learning” was searched on Google below.
+
+
+
+What's Next For Deep Learning?
+No one knows!
+
+## References
+[Convolutional Neural Network](https://machinelearningmastery.com/convolutional-layers-for-deep-learning-neural-networks/)
+
+[Pooling link1](https://www.analyticsvidhya.com/blog/2021/05/convolutional-neural-networks-cnn/)
+
+[Pooling link2](https://www.ibm.com/cloud/learn/convolutional-neural-networks)
+
+[Pooling link3](https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53)
+
+[Loss Functions](https://machinelearningmastery.com/how-to-choose-loss-functions-when-training-deep-learning-neural-networks/)
+
+[The output size of Conv Layer](https://pytorch.org/docs/1.9.1/generated/torch.nn.Conv2d.html)
+
+
From e7189d75512c045d3914e77db375922def7987ad Mon Sep 17 00:00:00 2001
From: dariusamiri <59167222+dariusamiri@users.noreply.github.com>
Date: Sat, 8 Jan 2022 00:21:11 +0330
Subject: [PATCH 02/16] Create matadata.yml
---
notebooks/18_deep_neural_network/matadata.yml | 29 +++++++++++++++++++
1 file changed, 29 insertions(+)
create mode 100644 notebooks/18_deep_neural_network/matadata.yml
diff --git a/notebooks/18_deep_neural_network/matadata.yml b/notebooks/18_deep_neural_network/matadata.yml
new file mode 100644
index 00000000..9a8f2e76
--- /dev/null
+++ b/notebooks/18_deep_neural_network/matadata.yml
@@ -0,0 +1,29 @@
+title: # shown on browser tab
+
+header:
+ title: # title of your notebook
+ description: # short description of your notebook
+
+authors:
+ label:
+ position: top
+ content:
+ # list of notebook authors
+ - name: # name of author
+ role: Author
+ - name: # name of author
+ role: Author
+ - name: # name of author
+ role: Author # change this if you want
+ contact:
+ # list of contact information
+ - link: https://github.com/
+ icon: fab fa-github
+ # optionally add other contact information like
+ # - link: # contact link
+ # icon: # awsomefont tag for link (check: https://fontawesome.com/v5.15/icons)
+
+comments:
+ # enable comments for your post
+ label: false
+ kind: comments
From 733c82a85b9cae44701a0041b4ae3b94bc327cb7 Mon Sep 17 00:00:00 2001
From: dariusamiri <59167222+dariusamiri@users.noreply.github.com>
Date: Sat, 8 Jan 2022 00:25:41 +0330
Subject: [PATCH 03/16] Update index.yml
---
notebooks/index.yml | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/notebooks/index.yml b/notebooks/index.yml
index 1b2133c0..63753982 100644
--- a/notebooks/index.yml
+++ b/notebooks/index.yml
@@ -54,4 +54,6 @@ notebooks:
kind: S2021, LN, Notebook
#- notebook: notebooks/17_markov_decision_processes/
- notebook: notebooks/18_reinforcement_learning/
- kind: S2021, LN, Notebook
\ No newline at end of file
+ kind: S2021, LN, Notebook
+ - notebook: notebooks/18_deep_neural_network/
+ kind: S2021, LN, Notebook
From 629e82e60cde5c94254bf7f23e59a9d4eede59c6 Mon Sep 17 00:00:00 2001
From: dariusamiri <59167222+dariusamiri@users.noreply.github.com>
Date: Sat, 8 Jan 2022 00:33:11 +0330
Subject: [PATCH 04/16] Update matadata.yml
---
notebooks/18_deep_neural_network/matadata.yml | 20 +++++++++++--------
1 file changed, 12 insertions(+), 8 deletions(-)
diff --git a/notebooks/18_deep_neural_network/matadata.yml b/notebooks/18_deep_neural_network/matadata.yml
index 9a8f2e76..bcfa2ffe 100644
--- a/notebooks/18_deep_neural_network/matadata.yml
+++ b/notebooks/18_deep_neural_network/matadata.yml
@@ -1,26 +1,30 @@
-title: # shown on browser tab
+title: deep neural network # shown on browser tab
header:
- title: # title of your notebook
- description: # short description of your notebook
+ title: Deep Neural Network # title of your notebook
+ description: This note is about CNN's and their application # short description of your notebook
authors:
label:
position: top
content:
# list of notebook authors
- - name: # name of author
+ - name: Parham Chavoshian # name of author
role: Author
- - name: # name of author
+ - name: Amirhossein Javadi # name of author
role: Author
- - name: # name of author
+ - name: Dariush Amiri # name of author
role: Author # change this if you want
contact:
# list of contact information
- - link: https://github.com/
+ - link: https://github.com/parhampch
+ - link: https://github.com/Amirhosein-javadi
+ - link: https://github.com/dariusamiri
icon: fab fa-github
# optionally add other contact information like
- # - link: # contact link
+ - link: parhampch@gmail.com # contact link
+ - link: javadiamirhosein.2000@gmail.com # contact link
+ - link: dariush_amiri7@yahoo.com # contact link
# icon: # awsomefont tag for link (check: https://fontawesome.com/v5.15/icons)
comments:
From 0226cb5a1fa170a6ae7fdbe7e1fddad533a83ca1 Mon Sep 17 00:00:00 2001
From: dariusamiri <59167222+dariusamiri@users.noreply.github.com>
Date: Sat, 8 Jan 2022 00:59:58 +0330
Subject: [PATCH 05/16] Update index.yml
---
notebooks/index.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/notebooks/index.yml b/notebooks/index.yml
index 63753982..1e26cd1c 100644
--- a/notebooks/index.yml
+++ b/notebooks/index.yml
@@ -56,4 +56,4 @@ notebooks:
- notebook: notebooks/18_reinforcement_learning/
kind: S2021, LN, Notebook
- notebook: notebooks/18_deep_neural_network/
- kind: S2021, LN, Notebook
+ kind: F2021, LN, md
From 808d37633ea205ac40e5ce881598c37b9ca426b5 Mon Sep 17 00:00:00 2001
From: dariusamiri <59167222+dariusamiri@users.noreply.github.com>
Date: Sat, 8 Jan 2022 01:15:09 +0330
Subject: [PATCH 06/16] Update matadata.yml
---
notebooks/18_deep_neural_network/matadata.yml | 29 ++++++++++++-------
1 file changed, 18 insertions(+), 11 deletions(-)
diff --git a/notebooks/18_deep_neural_network/matadata.yml b/notebooks/18_deep_neural_network/matadata.yml
index bcfa2ffe..f72802da 100644
--- a/notebooks/18_deep_neural_network/matadata.yml
+++ b/notebooks/18_deep_neural_network/matadata.yml
@@ -7,24 +7,31 @@ header:
authors:
label:
position: top
+ kind: people
content:
# list of notebook authors
- name: Parham Chavoshian # name of author
- role: Author
- - name: Amirhossein Javadi # name of author
- role: Author
- - name: Dariush Amiri # name of author
- role: Author # change this if you want
- contact:
+ role: Author
+ contact:
# list of contact information
- - link: https://github.com/parhampch
- - link: https://github.com/Amirhosein-javadi
- - link: https://github.com/dariusamiri
- icon: fab fa-github
- # optionally add other contact information like
+ - link: https://github.com/parhampch
- link: parhampch@gmail.com # contact link
+ icon: fab fa-github
+ - name: Amirhossein Javadi # name of author
+ role: Author
+ contact:
+ # list of contact information
+ - link: https://github.com/Amirhosein-javadi
- link: javadiamirhosein.2000@gmail.com # contact link
+ icon: fab fa-github
+ - name: Dariush Amiri # name of author
+ role: Author # change this if you want
+ contact:
+ # list of contact information
+ - link: https://github.com/dariusamiri
- link: dariush_amiri7@yahoo.com # contact link
+ icon: fab fa-github
+ # optionally add other contact information like
# icon: # awsomefont tag for link (check: https://fontawesome.com/v5.15/icons)
comments:
From b5a3390c3e70907de0fa28f710f75266ec4fba1e Mon Sep 17 00:00:00 2001
From: dariusamiri <59167222+dariusamiri@users.noreply.github.com>
Date: Sat, 8 Jan 2022 01:21:47 +0330
Subject: [PATCH 07/16] Update matadata.yml
---
notebooks/18_deep_neural_network/matadata.yml | 36 +++++++++----------
1 file changed, 18 insertions(+), 18 deletions(-)
diff --git a/notebooks/18_deep_neural_network/matadata.yml b/notebooks/18_deep_neural_network/matadata.yml
index f72802da..0549e69f 100644
--- a/notebooks/18_deep_neural_network/matadata.yml
+++ b/notebooks/18_deep_neural_network/matadata.yml
@@ -10,27 +10,27 @@ authors:
kind: people
content:
# list of notebook authors
- - name: Parham Chavoshian # name of author
- role: Author
- contact:
+ - name: Parham Chavoshian # name of author
+ role: Author
+ contact:
# list of contact information
- - link: https://github.com/parhampch
- - link: parhampch@gmail.com # contact link
- icon: fab fa-github
- - name: Amirhossein Javadi # name of author
- role: Author
- contact:
+ - link: https://github.com/parhampch
+ - link: parhampch@gmail.com # contact link
+ icon: fab fa-github
+ - name: Amirhossein Javadi # name of author
+ role: Author
+ contact:
# list of contact information
- - link: https://github.com/Amirhosein-javadi
- - link: javadiamirhosein.2000@gmail.com # contact link
- icon: fab fa-github
- - name: Dariush Amiri # name of author
- role: Author # change this if you want
- contact:
+ - link: https://github.com/Amirhosein-javadi
+ - link: javadiamirhosein.2000@gmail.com # contact link
+ icon: fab fa-github
+ - name: Dariush Amiri # name of author
+ role: Author # change this if you want
+ contact:
# list of contact information
- - link: https://github.com/dariusamiri
- - link: dariush_amiri7@yahoo.com # contact link
- icon: fab fa-github
+ - link: https://github.com/dariusamiri
+ - link: dariush_amiri7@yahoo.com # contact link
+ icon: fab fa-github
# optionally add other contact information like
# icon: # awsomefont tag for link (check: https://fontawesome.com/v5.15/icons)
From 7a83bd8ad861cbbd3b77a5148e40515894410d8c Mon Sep 17 00:00:00 2001
From: dariusamiri <59167222+dariusamiri@users.noreply.github.com>
Date: Sat, 8 Jan 2022 01:33:32 +0330
Subject: [PATCH 08/16] Update matadata.yml
---
notebooks/18_deep_neural_network/matadata.yml | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/notebooks/18_deep_neural_network/matadata.yml b/notebooks/18_deep_neural_network/matadata.yml
index 0549e69f..73003174 100644
--- a/notebooks/18_deep_neural_network/matadata.yml
+++ b/notebooks/18_deep_neural_network/matadata.yml
@@ -15,21 +15,21 @@ authors:
contact:
# list of contact information
- link: https://github.com/parhampch
- - link: parhampch@gmail.com # contact link
+ # - link: parhampch@gmail.com # contact link
icon: fab fa-github
- name: Amirhossein Javadi # name of author
role: Author
contact:
# list of contact information
- link: https://github.com/Amirhosein-javadi
- - link: javadiamirhosein.2000@gmail.com # contact link
+ # - link: javadiamirhosein.2000@gmail.com # contact link
icon: fab fa-github
- name: Dariush Amiri # name of author
role: Author # change this if you want
contact:
# list of contact information
- link: https://github.com/dariusamiri
- - link: dariush_amiri7@yahoo.com # contact link
+ # - link: dariush_amiri7@yahoo.com # contact link
icon: fab fa-github
# optionally add other contact information like
# icon: # awsomefont tag for link (check: https://fontawesome.com/v5.15/icons)
From d19ef67068dbb8248b6aebd6d98539f08bad43e0 Mon Sep 17 00:00:00 2001
From: dariusamiri <59167222+dariusamiri@users.noreply.github.com>
Date: Sat, 8 Jan 2022 01:36:55 +0330
Subject: [PATCH 09/16] Update main.yml
update workflow main.yml 1:36 AM
---
.github/workflows/main.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 7e8ef31a..70726084 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -24,5 +24,5 @@ jobs:
github_token: ${{ secrets.GITHUB_TOKEN }}
enable_jekyll: true
allow_empty_commit: true
- publish_dir: .
+ publish_dir: './webified/'
exclude_assets: '.github'
From 0a8bd9f92ab00bc63bb0f0deeda5f11615507ea2 Mon Sep 17 00:00:00 2001
From: dariusamiri <59167222+dariusamiri@users.noreply.github.com>
Date: Sat, 8 Jan 2022 01:42:01 +0330
Subject: [PATCH 10/16] Update index.yml
---
notebooks/index.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/notebooks/index.yml b/notebooks/index.yml
index 1e26cd1c..a8b2641c 100644
--- a/notebooks/index.yml
+++ b/notebooks/index.yml
@@ -55,5 +55,5 @@ notebooks:
#- notebook: notebooks/17_markov_decision_processes/
- notebook: notebooks/18_reinforcement_learning/
kind: S2021, LN, Notebook
- - notebook: notebooks/18_deep_neural_network/
+ - md: notebooks/18_deep_neural_network/
kind: F2021, LN, md
From 01c3a2e4ca0ac658974451f8a5cd64048bf11505 Mon Sep 17 00:00:00 2001
From: dariusamiri <59167222+dariusamiri@users.noreply.github.com>
Date: Sat, 8 Jan 2022 02:03:32 +0330
Subject: [PATCH 11/16] Update matadata.yml
2:03
---
notebooks/18_deep_neural_network/matadata.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/notebooks/18_deep_neural_network/matadata.yml b/notebooks/18_deep_neural_network/matadata.yml
index 73003174..e62b5ba7 100644
--- a/notebooks/18_deep_neural_network/matadata.yml
+++ b/notebooks/18_deep_neural_network/matadata.yml
@@ -32,7 +32,7 @@ authors:
# - link: dariush_amiri7@yahoo.com # contact link
icon: fab fa-github
# optionally add other contact information like
- # icon: # awsomefont tag for link (check: https://fontawesome.com/v5.15/icons)
+ # icon: # awsomefont tag for link (check: https://fontawesome.com/v5.15/icons)
comments:
# enable comments for your post
From 6d9a73037127f14c7ef358afba3720778c456b82 Mon Sep 17 00:00:00 2001
From: Dariush <59167222+dariusamiri@users.noreply.github.com>
Date: Sun, 16 Jan 2022 00:43:29 +0330
Subject: [PATCH 12/16] Update index.md
---
notebooks/18_deep_neural_network/index.md | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/notebooks/18_deep_neural_network/index.md b/notebooks/18_deep_neural_network/index.md
index 864df4b7..cb797185 100644
--- a/notebooks/18_deep_neural_network/index.md
+++ b/notebooks/18_deep_neural_network/index.md
@@ -4,11 +4,11 @@
- [Introduction](#introduction)
- [CNN Architecture](#CNN-Architecture)
+- [Fully Connected Layer](#Fully-Connected-Layer)
- [Conv Layer](#Conv-Layer)
- [Pooling](#Pooling)
- [Padding](#Padding)
- [Stride](#Stride)
-- [Fully Connected Layer](#Fully-Connected-Layer)
- [Activation Function](#Activation-Function)
- [Loss Functions](#Loss-Functions)
- [The Output Size of Conv Layer](#The-output-size-of-Conv-Layer)
@@ -20,7 +20,7 @@
Deep learning is a subfield of machine learning that deals with algorithms inspired by the structure and function of the brain. Deep learning is a subset of machine learning, which is a part of artificial intelligence (AI).
-CNN's are models to solve deep learning problems. Suppose that you have high-dimensional inputs such as images or videos. If we want to use MLPs, 2 (or more) dimensional inputs need to be converted to 1-dimensional vectors. This conversion increases the number of trainable parameters exponentially. Also, one important thing in these data is locality, it means that for example in an image, you can find features in near pixels (for examples corners and edges) but, far pixels can't give you efficient features. The solution for solving these problems is using CNNs.
+CNNs are models to solve deep learning problems. Suppose that you have high-dimensional inputs such as images or videos. If we want to use MLPs, 2 (or more) dimensional inputs need to be converted to 1-dimensional vectors. This conversion increases the number of trainable parameters exponentially. Also, one important thing in these data is locality, it means that for example in an image, you can find features in near pixels (for example corners and edges) but, distant pixels can't give you efficient features. The solution for solving these problems is using CNNs.
## CNN Architecture
A CNN architecture is formed by a stack of distinct layers that transform the input volume into an output volume through a differentiable function. A few distinct types of layers are commonly used:
@@ -52,7 +52,7 @@ Fully Connected layers in neural networks are those layers where all the inputs
## Conv Layer
-This layer is the main difference between CNNs and MLPs. Convolution in the word refers to two operators between two functions. In mathematics convolution define as below:
+This layer is the main difference between CNNs and MLPs. Convolution in the word refers to two operators between two functions. In mathematics convolution is defined as below:
In this image, you can see an example for functions of convolution and two other similar operators.
From 2b6690339a1ec70f68abd44263235be44c576027 Mon Sep 17 00:00:00 2001
From: Dariush <59167222+dariusamiri@users.noreply.github.com>
Date: Wed, 26 Jan 2022 08:50:06 +0330
Subject: [PATCH 13/16] Update index.md
---
notebooks/18_deep_neural_network/index.md | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/notebooks/18_deep_neural_network/index.md b/notebooks/18_deep_neural_network/index.md
index cb797185..e90a80f8 100644
--- a/notebooks/18_deep_neural_network/index.md
+++ b/notebooks/18_deep_neural_network/index.md
@@ -26,7 +26,7 @@ CNNs are models to solve deep learning problems. Suppose that you have high-dime
A CNN architecture is formed by a stack of distinct layers that transform the input volume into an output volume through a differentiable function. A few distinct types of layers are commonly used:
* Fully Connected Layer
-* Convolutional layer
+* Convolutional Layer
* Pooling
* Activation Function
* Loss layer
@@ -66,7 +66,7 @@ In this image, you can see an example for functions of convolution and two other
Here we’ll not talk about details, but convolutional layers are somehow enabling convolution operator on sub-matrices of the image. These layers have formed from some kernel with the same height, width, and depth. The number of these kernels is equal to the depth of the output. Also, the depth of each kernel must be equal to the depth of input. For example, if you have RGB data, your first convolutional layer kernels depth must be 3.
-In the context of a convolutional neural network, convolution is a linear operation that involves the multiplication of a set of weights with the input. A convolution layer has formed by 1 or more of these operations that each of them called a kernel. All kernels have the same height, width, and depth. To find the output of the layer, we put the first kernel on the top-right of the input and calculate the output of the kernel, and put it as the first cell of a matrix. After that, we move it to right and calculate again, and put the result in the second cell. When we receive to end of columns, we move the kernel down. we do this until we rich to the end of the image. We do this for all kernels and this is how we make the output of the convolutional layer.
+In the context of a convolutional neural network, convolution is a linear operation that involves the multiplication of a set of weights with the input. A convolution layer has formed by 1 or more of these operations that each of them called a kernel. All kernels have the same height, width, and depth. To find the output of the layer put the first kernel on the top-right of the input, calculate the output of the kernel, and put it as the first cell of a matrix. After that, move it to right, calculate again, and put the result in the second cell. When reaching the last column, move the kernel down. Do this until reaching the end of the image. We do this for all kernels and this is how make the output of the convolutional layer.
Date: Wed, 26 Jan 2022 08:55:53 +0330
Subject: [PATCH 14/16] Update index.md
---
notebooks/18_deep_neural_network/index.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/notebooks/18_deep_neural_network/index.md b/notebooks/18_deep_neural_network/index.md
index e90a80f8..0cbeb339 100644
--- a/notebooks/18_deep_neural_network/index.md
+++ b/notebooks/18_deep_neural_network/index.md
@@ -117,7 +117,7 @@ Different padding modes are:
## Stride
-As we said before, when you're applying a kernel to the image, you have to move the kernel during the image. But sometimes you prefer to not move one pixel every time and move the kernel more than one pixel. This is stride. The stride specifies how many kernels have to move each time.
+As we said before, when you're applying a kernel to the image, you have to move the kernel along the image. But sometimes you prefer not to move one pixel every time and move the kernel more than one pixel. This is stride. The stride specifies how many kernels have to move each time.
Date: Sun, 30 Jan 2022 15:37:46 +0330
Subject: [PATCH 15/16] Update index.md
---
notebooks/18_deep_neural_network/index.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/notebooks/18_deep_neural_network/index.md b/notebooks/18_deep_neural_network/index.md
index 0cbeb339..1b783c41 100644
--- a/notebooks/18_deep_neural_network/index.md
+++ b/notebooks/18_deep_neural_network/index.md
@@ -20,7 +20,7 @@
Deep learning is a subfield of machine learning that deals with algorithms inspired by the structure and function of the brain. Deep learning is a subset of machine learning, which is a part of artificial intelligence (AI).
-CNNs are models to solve deep learning problems. Suppose that you have high-dimensional inputs such as images or videos. If we want to use MLPs, 2 (or more) dimensional inputs need to be converted to 1-dimensional vectors. This conversion increases the number of trainable parameters exponentially. Also, one important thing in these data is locality, it means that for example in an image, you can find features in near pixels (for example corners and edges) but, distant pixels can't give you efficient features. The solution for solving these problems is using CNNs.
+CNNs are models to solve deep learning problems. Suppose that you have high-dimensional inputs such as images or videos. If we want to use MLPs, 2 (or more) dimensional inputs need to be converted to 1-dimensional vectors. This conversion increases the number of trainable parameters exponentially. Also, one important thing in these data is locality, it means that for example in an image, you can find features in adjacent pixels (for example corners and edges) but, distant pixels can't give you efficient features. The solution for solving these problems is using CNNs.
## CNN Architecture
A CNN architecture is formed by a stack of distinct layers that transform the input volume into an output volume through a differentiable function. A few distinct types of layers are commonly used:
@@ -29,7 +29,7 @@ A CNN architecture is formed by a stack of distinct layers that transform the in
* Convolutional Layer
* Pooling
* Activation Function
-* Loss layer
+* Loss Layer
In the following, you can see images of two simple famous CNNs.
From 87cf40843c941bffc8e6f9e1448ac0212bacb86a Mon Sep 17 00:00:00 2001
From: Dariush <59167222+dariusamiri@users.noreply.github.com>
Date: Sat, 5 Feb 2022 09:33:08 +0330
Subject: [PATCH 16/16] Update index.md
---
notebooks/18_deep_neural_network/index.md | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/notebooks/18_deep_neural_network/index.md b/notebooks/18_deep_neural_network/index.md
index 1b783c41..824541fc 100644
--- a/notebooks/18_deep_neural_network/index.md
+++ b/notebooks/18_deep_neural_network/index.md
@@ -1,5 +1,11 @@
# Deep Neural Networks
+## The authors' name
+
+- Parham Chavoshian
+- Amirhossein Javadi
+- Dariush Amiri
+
## Table of Content
- [Introduction](#introduction)
@@ -91,8 +97,8 @@ This is to decrease the computational power required to process the data by redu
It sweeps a filter across the entire input but that does not have any weights. Instead, the kernel applies an aggregation function to the values within the receptive field, populating the output array.
There are two types of Pooling:
-1. Max Pooling: it returns the maximum value from the portion of the image covered by the Kernel. and also performs as a Noise Suppressant. It discards the noisy activations altogether and also performs de-noising along with dimensionality reduction.
-2. Average Pooling: it returns the average of all the values from the portion of the image covered by the Kernel, and simply performs dimensionality reduction as a noise suppressing mechanism.
+1. Max Pooling: It returns the maximum value from the portion of the image covered by the Kernel. And also performs as a Noise Suppressant. It discards the noisy activations altogether and also performs de-noising along with dimensionality reduction.
+2. Average Pooling: It returns the average of all the values from the portion of the image covered by the Kernel, and simply performs dimensionality reduction as a noise suppressing mechanism.
Comparing these two, we can say that Max Pooling performs a lot better than Average Pooling.
In the following, we will see a picture of how these two methods work.
+ 
+ 
+ 
+ 
+ 
+