18_deep_neural_network by dariusamiri · Pull Request #29 · sut-ai/notes

dariusamiri · 2022-01-07T21:12:57Z

update workflow main.yml 1:36 AM

2:03

nimajam41 · 2022-01-14T17:45:42Z

notebooks/18_deep_neural_network/index.md

@@ -0,0 +1,201 @@
+# Deep Neural Networks


Try to have a different format for title and write authors' names in your lecture note

nimajam41 · 2022-01-14T17:48:50Z

notebooks/18_deep_neural_network/index.md

+Deep learning is a subfield of machine learning that deals with algorithms inspired by the structure and function of the brain. Deep learning is a subset of machine learning, which is a part of artificial intelligence (AI).
+![](https://i.imgur.com/qhjJzDb.png)
+
+CNN's are models to solve deep learning problems. Suppose that you have high-dimensional inputs such as images or videos. If we want to use MLPs, 2 (or more) dimensional inputs need to be converted to 1-dimensional vectors. This conversion increases the number of trainable parameters exponentially. Also, one important thing in these data is locality, it means that for example in an image, you can find features in near pixels (for examples corners and edges) but, far pixels can't give you efficient features. The solution for solving these problems is using CNNs.


nimajam41 · 2022-01-14T17:50:29Z

notebooks/18_deep_neural_network/index.md

+Deep learning is a subfield of machine learning that deals with algorithms inspired by the structure and function of the brain. Deep learning is a subset of machine learning, which is a part of artificial intelligence (AI).
+![](https://i.imgur.com/qhjJzDb.png)
+
+CNN's are models to solve deep learning problems. Suppose that you have high-dimensional inputs such as images or videos. If we want to use MLPs, 2 (or more) dimensional inputs need to be converted to 1-dimensional vectors. This conversion increases the number of trainable parameters exponentially. Also, one important thing in these data is locality, it means that for example in an image, you can find features in near pixels (for examples corners and edges) but, far pixels can't give you efficient features. The solution for solving these problems is using CNNs.


find features in adjacent pixels

nimajam41 · 2022-01-14T17:51:09Z

notebooks/18_deep_neural_network/index.md

+Deep learning is a subfield of machine learning that deals with algorithms inspired by the structure and function of the brain. Deep learning is a subset of machine learning, which is a part of artificial intelligence (AI).
+![](https://i.imgur.com/qhjJzDb.png)
+
+CNN's are models to solve deep learning problems. Suppose that you have high-dimensional inputs such as images or videos. If we want to use MLPs, 2 (or more) dimensional inputs need to be converted to 1-dimensional vectors. This conversion increases the number of trainable parameters exponentially. Also, one important thing in these data is locality, it means that for example in an image, you can find features in near pixels (for examples corners and edges) but, far pixels can't give you efficient features. The solution for solving these problems is using CNNs.


for examples -> for example

nimajam41 · 2022-01-14T17:53:43Z

notebooks/18_deep_neural_network/index.md

+Deep learning is a subfield of machine learning that deals with algorithms inspired by the structure and function of the brain. Deep learning is a subset of machine learning, which is a part of artificial intelligence (AI).
+![](https://i.imgur.com/qhjJzDb.png)
+
+CNN's are models to solve deep learning problems. Suppose that you have high-dimensional inputs such as images or videos. If we want to use MLPs, 2 (or more) dimensional inputs need to be converted to 1-dimensional vectors. This conversion increases the number of trainable parameters exponentially. Also, one important thing in these data is locality, it means that for example in an image, you can find features in near pixels (for examples corners and edges) but, far pixels can't give you efficient features. The solution for solving these problems is using CNNs.


far pixels -> distant pixels

nimajam41 · 2022-01-14T17:59:15Z

notebooks/18_deep_neural_network/index.md

+A CNN architecture is formed by a stack of distinct layers that transform the input volume into an output volume through a differentiable function. A few distinct types of layers are commonly used:
+
+* Fully Connected Layer
+* Convolutional layer


layer -> Layer (to be synced with other bullets)

nimajam41 · 2022-01-14T18:01:04Z

notebooks/18_deep_neural_network/index.md

@@ -0,0 +1,201 @@
+# Deep Neural Networks
+
+## Table of Content


The order of contents doesn't match with the table. Revise them

nimajam41 · 2022-01-14T20:39:43Z

notebooks/18_deep_neural_network/index.md

+![](https://i.imgur.com/3nItEgk.png)
+
+## Conv Layer
+This layer is the main difference between CNNs and MLPs. Convolution in the word refers to two operators between two functions. In mathematics convolution define as below:  


define -> is defined

nimajam41 · 2022-01-14T21:54:13Z

notebooks/18_deep_neural_network/index.md

+
+
+Here we’ll not talk about details, but convolutional layers are somehow enabling convolution operator on sub-matrices of the image. These layers have formed from some kernel with the same height, width, and depth. The number of these kernels is equal to the depth of the output. Also, the depth of each kernel must be equal to the depth of input. For example, if you have RGB data, your first convolutional layer kernels depth must be 3.  
+In the context of a convolutional neural network, convolution is a linear operation that involves the multiplication of a set of weights with the input. A convolution layer has formed by 1 or more of these operations that each of them called a kernel. All kernels have the same height, width, and depth. To find the output of the layer, we put the first kernel on the top-right of the input and calculate the output of the kernel, and put it as the first cell of a matrix. After that, we move it to right and calculate again, and put the result in the second cell. When we receive to end of columns, we move the kernel down. we do this until we rich to the end of the image. We do this for all kernels and this is how we make the output of the convolutional layer.


When we reach the last column

nimajam41 · 2022-01-14T21:55:13Z

notebooks/18_deep_neural_network/index.md

+
+
+Here we’ll not talk about details, but convolutional layers are somehow enabling convolution operator on sub-matrices of the image. These layers have formed from some kernel with the same height, width, and depth. The number of these kernels is equal to the depth of the output. Also, the depth of each kernel must be equal to the depth of input. For example, if you have RGB data, your first convolutional layer kernels depth must be 3.  
+In the context of a convolutional neural network, convolution is a linear operation that involves the multiplication of a set of weights with the input. A convolution layer has formed by 1 or more of these operations that each of them called a kernel. All kernels have the same height, width, and depth. To find the output of the layer, we put the first kernel on the top-right of the input and calculate the output of the kernel, and put it as the first cell of a matrix. After that, we move it to right and calculate again, and put the result in the second cell. When we receive to end of columns, we move the kernel down. we do this until we rich to the end of the image. We do this for all kernels and this is how we make the output of the convolutional layer.


we -> We
rich -> reach

nimajam41 · 2022-01-14T21:55:54Z

notebooks/18_deep_neural_network/index.md

+
+
+Here we’ll not talk about details, but convolutional layers are somehow enabling convolution operator on sub-matrices of the image. These layers have formed from some kernel with the same height, width, and depth. The number of these kernels is equal to the depth of the output. Also, the depth of each kernel must be equal to the depth of input. For example, if you have RGB data, your first convolutional layer kernels depth must be 3.  
+In the context of a convolutional neural network, convolution is a linear operation that involves the multiplication of a set of weights with the input. A convolution layer has formed by 1 or more of these operations that each of them called a kernel. All kernels have the same height, width, and depth. To find the output of the layer, we put the first kernel on the top-right of the input and calculate the output of the kernel, and put it as the first cell of a matrix. After that, we move it to right and calculate again, and put the result in the second cell. When we receive to end of columns, we move the kernel down. we do this until we rich to the end of the image. We do this for all kernels and this is how we make the output of the convolutional layer.


Using so many "we"s in the context is a bad smell!

nimajam41 · 2022-01-14T21:57:04Z

notebooks/18_deep_neural_network/index.md

+
+## Pooling
+
+Similar to the Convolutional Layer, the Pooling layer is responsible for reducing the spatial size of the Convolved Feature.


convolved feature

nimajam41 · 2022-01-14T21:57:56Z

notebooks/18_deep_neural_network/index.md

+## Pooling
+
+Similar to the Convolutional Layer, the Pooling layer is responsible for reducing the spatial size of the Convolved Feature.
+While a lot of information is lost in the pooling layer, it also has a number of benefits to the Convolutional neural network. They help to reduce complexity, improve efficiency, and limit the risk of overfitting.


if you want to use capital case, use it for all words of a phrase: Convolutional Neural Network

nimajam41 · 2022-01-14T21:59:42Z

notebooks/18_deep_neural_network/index.md

+There are two types of Pooling:
+
+1. Max Pooling: it  returns the maximum value from the portion of the image covered by the Kernel. and also performs as a Noise Suppressant. It discards the noisy activations altogether and also performs de-noising along with dimensionality reduction.
+2. Average Pooling: it returns the average of all the values from the portion of the image covered by the Kernel. and simply performs dimensionality reduction as a noise suppressing mechanism. 


by the kernel, and simply

nimajam41 · 2022-01-14T22:00:38Z

notebooks/18_deep_neural_network/index.md

+
+## Padding
+
+As you see, after applying convolutional layers, the size of the feature map is always smaller than the input, we have to do something to prevent our feature map from shrinking. This is where we use padding. Layers of zero-value pixels are added to surround the input with zeros so that our feature map will not shrink. By padding, we can control the shrinking of our inputs.


smaller than the input. We have to ...

nimajam41 · 2022-01-14T22:01:28Z

notebooks/18_deep_neural_network/index.md

+## Padding
+
+As you see, after applying convolutional layers, the size of the feature map is always smaller than the input, we have to do something to prevent our feature map from shrinking. This is where we use padding. Layers of zero-value pixels are added to surround the input with zeros so that our feature map will not shrink. By padding, we can control the shrinking of our inputs.
+Different padding mode is:


Different padding modes are:

nimajam41 · 2022-01-14T22:02:07Z

notebooks/18_deep_neural_network/index.md

+As you see, after applying convolutional layers, the size of the feature map is always smaller than the input, we have to do something to prevent our feature map from shrinking. This is where we use padding. Layers of zero-value pixels are added to surround the input with zeros so that our feature map will not shrink. By padding, we can control the shrinking of our inputs.
+Different padding mode is:
+* zeros(Default)
+* reflect


Explain or hyperlink what this padding is

nimajam41 · 2022-01-14T22:02:13Z

notebooks/18_deep_neural_network/index.md

+Different padding mode is:
+* zeros(Default)
+* reflect
+* replicate or circular


Explain or hyperlink what this padding is

nimajam41 · 2022-01-14T23:03:26Z

notebooks/18_deep_neural_network/index.md

+
+
+## Stride
+As we said before, when you're applying a kernel to the image, you have to move the kernel during the image. But sometimes you prefer to not move one pixel every time and move the kernel more than one pixel. This is stride. The stride specifies how many kernels have to move each time.


during -> along

nimajam41 · 2022-01-14T23:04:47Z

notebooks/18_deep_neural_network/index.md

+
+
+## Stride
+As we said before, when you're applying a kernel to the image, you have to move the kernel during the image. But sometimes you prefer to not move one pixel every time and move the kernel more than one pixel. This is stride. The stride specifies how many kernels have to move each time.


to not move -> not to move

nimajam41 · 2022-01-14T23:12:04Z

notebooks/18_deep_neural_network/index.md

+## Table of Content
+
+- [Introduction](#introduction)
+- [CNN Architecture](#CNN-Architecture)


All mentioned layers and functions can be considered as subsections for this section

This seems unresolved yet.
I mean this structure:

CNN Architecture
- Fully Connected Layers
- Conv Layer
...

nimajam41

1- Review and revise your English mistakes
2- Edit Table of Contents

dariusamiri · 2022-01-15T16:14:10Z

سلام، خسته نباشید. هر دوی Fully Connected Layer

Convolutional layer

اصلاح شود؟ یا فقط آخری؟

…

On Jan 14, 2022, at 21:29, nimajam41 ***@***.***> wrote: @nimajam41 commented on this pull request. In notebooks/18_deep_neural_network/index.md <#29 (comment)>: > +- [The Output Size of Conv Layer](#The-output-size-of-Conv-Layer) +- [Conclusion](#Conclusion) +- [References](#References) + + +## Introduction +Deep learning is a subfield of machine learning that deals with algorithms inspired by the structure and function of the brain. Deep learning is a subset of machine learning, which is a part of artificial intelligence (AI). +![](https://i.imgur.com/qhjJzDb.png) + +CNN's are models to solve deep learning problems. Suppose that you have high-dimensional inputs such as images or videos. If we want to use MLPs, 2 (or more) dimensional inputs need to be converted to 1-dimensional vectors. This conversion increases the number of trainable parameters exponentially. Also, one important thing in these data is locality, it means that for example in an image, you can find features in near pixels (for examples corners and edges) but, far pixels can't give you efficient features. The solution for solving these problems is using CNNs. + +## CNN Architecture +A CNN architecture is formed by a stack of distinct layers that transform the input volume into an output volume through a differentiable function. A few distinct types of layers are commonly used: + +* Fully Connected Layer +* Convolutional layer layer -> Layer (to be synced with other bullets) — Reply to this email directly, view it on GitHub <#29 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AODND5RET3GXD4ZRWCM53ALUWBP75ANCNFSM5LPU6NEA>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.

dariusamiri · 2022-01-15T16:33:52Z

من دقیقا متوجه منظورتون از to be synced with other bullets نشدم. ممکن هست بیشتر توضیح بدین؟

…

On Jan 14, 2022, at 21:29, nimajam41 ***@***.***> wrote: @nimajam41 commented on this pull request. In notebooks/18_deep_neural_network/index.md <#29 (comment)>: > +- [The Output Size of Conv Layer](#The-output-size-of-Conv-Layer) +- [Conclusion](#Conclusion) +- [References](#References) + + +## Introduction +Deep learning is a subfield of machine learning that deals with algorithms inspired by the structure and function of the brain. Deep learning is a subset of machine learning, which is a part of artificial intelligence (AI). +![](https://i.imgur.com/qhjJzDb.png) + +CNN's are models to solve deep learning problems. Suppose that you have high-dimensional inputs such as images or videos. If we want to use MLPs, 2 (or more) dimensional inputs need to be converted to 1-dimensional vectors. This conversion increases the number of trainable parameters exponentially. Also, one important thing in these data is locality, it means that for example in an image, you can find features in near pixels (for examples corners and edges) but, far pixels can't give you efficient features. The solution for solving these problems is using CNNs. + +## CNN Architecture +A CNN architecture is formed by a stack of distinct layers that transform the input volume into an output volume through a differentiable function. A few distinct types of layers are commonly used: + +* Fully Connected Layer +* Convolutional layer layer -> Layer (to be synced with other bullets) — Reply to this email directly, view it on GitHub <#29 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AODND5RET3GXD4ZRWCM53ALUWBP75ANCNFSM5LPU6NEA>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.

nimajam41 · 2022-02-03T23:42:44Z

notebooks/18_deep_neural_network/index.md

+It sweeps a filter across the entire input but that does not have any weights. Instead, the kernel applies an aggregation function to the values within the receptive field, populating the output array.
+There are two types of Pooling:
+
+1. Max Pooling: it  returns the maximum value from the portion of the image covered by the Kernel. and also performs as a Noise Suppressant. It discards the noisy activations altogether and also performs de-noising along with dimensionality reduction.


Start sentences with capital letters

nimajam41

Thanks for your revision. Most of the issues have been resolved. Try to make the Table of Contents hierarchical, explain what different paddings are, and edit some wrong usages of grammar.

dariusamiri added 11 commits January 8, 2022 00:18

Create index.md

ef3e47d

Create matadata.yml

e7189d7

Update index.yml

733c82a

Update matadata.yml

629e82e

Update index.yml

0226cb5

Update matadata.yml

808d376

Update matadata.yml

b5a3390

Update matadata.yml

7a83bd8

Update main.yml

d19ef67

update workflow main.yml 1:36 AM

Update index.yml

0a8bd9f

Update matadata.yml

01c3a2e

2:03

sararajabzadeh requested a review from nimajam41 January 12, 2022 15:39

nimajam41 reviewed Jan 14, 2022

View reviewed changes

dariusamiri and others added 4 commits January 16, 2022 00:43

Update index.md

6d9a730

Update index.md

2b66903

Update index.md

30679da

Update index.md

862588a

nimajam41 reviewed Feb 3, 2022

View reviewed changes

Update index.md

87cf408



		Here we’ll not talk about details, but convolutional layers are somehow enabling convolution operator on sub-matrices of the image. These layers have formed from some kernel with the same height, width, and depth. The number of these kernels is equal to the depth of the output. Also, the depth of each kernel must be equal to the depth of input. For example, if you have RGB data, your first convolutional layer kernels depth must be 3.
		In the context of a convolutional neural network, convolution is a linear operation that involves the multiplication of a set of weights with the input. A convolution layer has formed by 1 or more of these operations that each of them called a kernel. All kernels have the same height, width, and depth. To find the output of the layer, we put the first kernel on the top-right of the input and calculate the output of the kernel, and put it as the first cell of a matrix. After that, we move it to right and calculate again, and put the result in the second cell. When we receive to end of columns, we move the kernel down. we do this until we rich to the end of the image. We do this for all kernels and this is how we make the output of the convolutional layer.


		## Pooling

		Similar to the Convolutional Layer, the Pooling layer is responsible for reducing the spatial size of the Convolved Feature.


		## Padding

		As you see, after applying convolutional layers, the size of the feature map is always smaller than the input, we have to do something to prevent our feature map from shrinking. This is where we use padding. Layers of zero-value pixels are added to surround the input with zeros so that our feature map will not shrink. By padding, we can control the shrinking of our inputs.



		## Stride
		As we said before, when you're applying a kernel to the image, you have to move the kernel during the image. But sometimes you prefer to not move one pixel every time and move the kernel more than one pixel. This is stride. The stride specifies how many kernels have to move each time.

Conversation

dariusamiri commented Jan 7, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nimajam41 left a comment

Choose a reason for hiding this comment

Uh oh!

dariusamiri commented Jan 15, 2022 via email

Uh oh!

dariusamiri commented Jan 15, 2022 via email

Uh oh!

nimajam41 Feb 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nimajam41 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nimajam41 Feb 3, 2022 •

edited

Loading