1
00:00:01,530 --> 00:00:20,239
[Music]

2
00:00:20,680 --> 00:00:29,840
now we will be listening from dr. V to

3
00:00:25,190 --> 00:00:33,050
ravichandran from MathWorks manglu he

4
00:00:29,840 --> 00:00:35,720
will be giving you a detailed exposure

5
00:00:33,050 --> 00:00:39,680
on neural network toolbox of MATLAB

6
00:00:35,720 --> 00:00:41,900
so dr. Bachand Rijn can you hear me yes

7
00:00:39,680 --> 00:00:44,450
can you hear me ma'am yes yes we can all

8
00:00:41,900 --> 00:00:48,260
very well hear you I think you can start

9
00:00:44,450 --> 00:00:50,000
now you can your introduction then about

10
00:00:48,260 --> 00:00:53,390
something about math works and MATLAB

11
00:00:50,000 --> 00:00:56,510
also and then warren oh thank you thank

12
00:00:53,390 --> 00:00:58,790
you sure good afternoon everyone thank

13
00:00:56,510 --> 00:01:03,140
you ma'am for the kind introduction so

14
00:00:58,790 --> 00:01:04,730
uh as mom mentioned I'm the Java

15
00:01:03,140 --> 00:01:07,130
chandran or you'll find me as Lakshmi

16
00:01:04,730 --> 00:01:11,450
Narayan Ramachandran on LinkedIn or on

17
00:01:07,130 --> 00:01:13,280
Google Scholar so I lead the education

18
00:01:11,450 --> 00:01:16,550
Technical Evangelist team at math works

19
00:01:13,280 --> 00:01:19,310
what do we do as a team we work with

20
00:01:16,550 --> 00:01:21,410
educational institutes to ensure that

21
00:01:19,310 --> 00:01:23,510
our students and faculty are able to use

22
00:01:21,410 --> 00:01:25,460
MATLAB and Simulink and a host of our

23
00:01:23,510 --> 00:01:27,890
other products effectively for the

24
00:01:25,460 --> 00:01:31,220
teaching research projects and many

25
00:01:27,890 --> 00:01:32,510
other things I work at math works India

26
00:01:31,220 --> 00:01:36,500
matchbooks is a company that develops

27
00:01:32,510 --> 00:01:39,950
MATLAB and Simulink and we've been in

28
00:01:36,500 --> 00:01:41,810
the India market for around or 10 years

29
00:01:39,950 --> 00:01:44,660
and mass works as a company has been

30
00:01:41,810 --> 00:01:47,930
around for 34 years and we are deeply

31
00:01:44,660 --> 00:01:50,750
rooted in education and you know a major

32
00:01:47,930 --> 00:01:52,730
part of our you know user base is based

33
00:01:50,750 --> 00:01:54,620
out of education and you're happy to

34
00:01:52,730 --> 00:01:58,340
take I have this opportunity present to

35
00:01:54,620 --> 00:02:01,490
true us by ni triple TR to talk to you

36
00:01:58,340 --> 00:02:03,740
about MATLAB the neural network toolbox

37
00:02:01,490 --> 00:02:07,520
workflow and tomorrow a little bit on

38
00:02:03,740 --> 00:02:09,649
optimization so um I was told that I

39
00:02:07,520 --> 00:02:12,650
have our time until afore PMR this

40
00:02:09,649 --> 00:02:14,330
afternoon so we'll be going over a

41
00:02:12,650 --> 00:02:16,450
little bit of the neural network toolbox

42
00:02:14,330 --> 00:02:19,790
workflow in the initial part of the

43
00:02:16,450 --> 00:02:21,920
session and this will be followed by

44
00:02:19,790 --> 00:02:24,500
what has been a natural extension to

45
00:02:21,920 --> 00:02:26,238
neural networks uh you know in the

46
00:02:24,500 --> 00:02:28,760
recent past especially it's become a big

47
00:02:26,239 --> 00:02:30,710
buzzword it's known as deep learning a

48
00:02:28,760 --> 00:02:33,260
lot of you know what machine

49
00:02:30,710 --> 00:02:34,550
is I'd like to show you what the

50
00:02:33,260 --> 00:02:36,320
difference between machine learning and

51
00:02:34,550 --> 00:02:38,750
deep learning is and then how you can

52
00:02:36,320 --> 00:02:40,280
use MATLAB to do that if time permits I

53
00:02:38,750 --> 00:02:41,360
will show you a few examples otherwise

54
00:02:40,280 --> 00:02:43,850
I'll show them tomorrow

55
00:02:41,360 --> 00:02:46,610
on the deep learning workflow and we can

56
00:02:43,850 --> 00:02:49,070
get started with the optimization so and

57
00:02:46,610 --> 00:02:51,740
I'd be happy to take questions in

58
00:02:49,070 --> 00:02:55,519
between or towards the end of the

59
00:02:51,740 --> 00:02:58,520
session and you know um of course I you

60
00:02:55,520 --> 00:03:00,650
have you been exposed to this webcast

61
00:02:58,520 --> 00:03:02,210
session for a lot longer than I have so

62
00:03:00,650 --> 00:03:04,670
there are a few etiquettes which I think

63
00:03:02,210 --> 00:03:08,720
uh man would have briefed you about so

64
00:03:04,670 --> 00:03:10,100
that sort of gives you know provides you

65
00:03:08,720 --> 00:03:13,010
with an introduction of me and what you

66
00:03:10,100 --> 00:03:13,340
can expect from the talk I just want a

67
00:03:13,010 --> 00:03:16,250
quick

68
00:03:13,340 --> 00:03:18,680
you know I I'll open my hangouts browser

69
00:03:16,250 --> 00:03:20,360
but maybe if you can unmute your lines

70
00:03:18,680 --> 00:03:22,850
one by one and you're like how many of

71
00:03:20,360 --> 00:03:26,690
you have worked on MATLAB or how many of

72
00:03:22,850 --> 00:03:28,910
you worked on Simulink maybe I can a

73
00:03:26,690 --> 00:03:38,210
couple of you tell me what you have done

74
00:03:28,910 --> 00:04:10,100
using MATLAB or Simulink anyone anyone

75
00:03:38,210 --> 00:04:11,990
from the CentOS sure if I'm not able to

76
00:04:10,100 --> 00:04:13,730
answer your question or if I don't touch

77
00:04:11,990 --> 00:04:16,370
upon those because today we'll be going

78
00:04:13,730 --> 00:04:17,810
over a basic workflow you can always get

79
00:04:16,370 --> 00:04:19,340
in touch with me offline and we'd be

80
00:04:17,810 --> 00:04:21,380
happy to talk to you more on that my

81
00:04:19,339 --> 00:04:24,619
email address is on the slide over here

82
00:04:21,380 --> 00:04:30,320
yeah okay anyone else saw that and

83
00:04:24,620 --> 00:04:31,760
descenders yeah okay I think many of

84
00:04:30,320 --> 00:04:35,390
them are working I have not explained

85
00:04:31,760 --> 00:04:38,919
them to a simile and MATLAB axis but I

86
00:04:35,390 --> 00:04:41,690
think I hope that they are they know

87
00:04:38,919 --> 00:04:43,289
yeah I will go over a brief I'll do a

88
00:04:41,690 --> 00:04:45,749
brief

89
00:04:43,289 --> 00:04:47,279
intro to MATLAB if needed do you think

90
00:04:45,749 --> 00:04:49,740
that would be helpful our participants

91
00:04:47,279 --> 00:04:56,219
just say yes or no a brief intro to

92
00:04:49,740 --> 00:04:57,360
MATLAB and all that okay sure I can go

93
00:04:56,219 --> 00:04:59,969
over that okay sure

94
00:04:57,360 --> 00:05:05,430
good okay that that helps okay sure so

95
00:04:59,969 --> 00:05:07,169
uh the talk today as I mentioned we

96
00:05:05,430 --> 00:05:09,360
wanted to introduce what is an

97
00:05:07,169 --> 00:05:10,948
artificial neural network I know mom

98
00:05:09,360 --> 00:05:12,930
might have spoken to you about what it

99
00:05:10,949 --> 00:05:16,050
is but a little bit of a background on

100
00:05:12,930 --> 00:05:24,809
what artificial neural networks what do

101
00:05:16,050 --> 00:05:28,349
we have as what is an artificial neural

102
00:05:24,809 --> 00:05:30,300
network and what is there so in this

103
00:05:28,349 --> 00:05:32,789
block diagram if you notice there is an

104
00:05:30,300 --> 00:05:34,830
input and there is an output yeah and

105
00:05:32,789 --> 00:05:37,770
the output is tested against a target

106
00:05:34,830 --> 00:05:40,139
output and then you you provide some

107
00:05:37,770 --> 00:05:42,149
form of a feedback here the feedback is

108
00:05:40,139 --> 00:05:44,339
provided to what is known as a neural

109
00:05:42,149 --> 00:05:46,589
network which includes connections

110
00:05:44,339 --> 00:05:48,990
between neurons and these connections

111
00:05:46,589 --> 00:05:51,389
are called weights what are neurons

112
00:05:48,990 --> 00:05:54,089
essentially it's one of the simplest

113
00:05:51,389 --> 00:05:56,789
elements of what is there as a cognitive

114
00:05:54,089 --> 00:05:59,879
system you can draw parallels to a

115
00:05:56,789 --> 00:06:01,860
control system which has feedback or you

116
00:05:59,879 --> 00:06:04,319
can also draw parallels to an adaptive

117
00:06:01,860 --> 00:06:05,939
or filter which also has feedback a lot

118
00:06:04,319 --> 00:06:08,309
of times what happens is we want to

119
00:06:05,939 --> 00:06:10,379
ensure that the output and the target

120
00:06:08,309 --> 00:06:12,389
are so close enough that by adjusting

121
00:06:10,379 --> 00:06:14,969
these weights the difference between

122
00:06:12,389 --> 00:06:17,009
them or the error there is minimized so

123
00:06:14,969 --> 00:06:19,619
that is essentially what an artificial

124
00:06:17,009 --> 00:06:22,680
neural network is so it's an input and

125
00:06:19,619 --> 00:06:24,659
output a neural network in here where we

126
00:06:22,680 --> 00:06:27,149
are just a weights to be able to ensure

127
00:06:24,659 --> 00:06:29,248
that the target is you know close enough

128
00:06:27,149 --> 00:06:32,369
to the output or output is close enough

129
00:06:29,249 --> 00:06:34,800
to the target yeah and the output is

130
00:06:32,369 --> 00:06:38,879
produced based odd on the input and the

131
00:06:34,800 --> 00:06:42,269
activation that is over there now what

132
00:06:38,879 --> 00:06:44,909
happens is in this block diagram I

133
00:06:42,269 --> 00:06:47,909
introduced neural networks as a block

134
00:06:44,909 --> 00:06:50,099
now what happens is the neuron model

135
00:06:47,909 --> 00:06:53,399
here is that you have multiple inputs to

136
00:06:50,099 --> 00:06:57,710
the neuron which take into account a lot

137
00:06:53,399 --> 00:06:59,690
of weights that is W 1 to W R ya and

138
00:06:57,710 --> 00:07:02,359
each input you know if you have our

139
00:06:59,690 --> 00:07:05,240
inputs each input is weighted with an

140
00:07:02,360 --> 00:07:08,150
appropriate weight parameter okay and

141
00:07:05,240 --> 00:07:10,370
the sum of the weighted inputs and it's

142
00:07:08,150 --> 00:07:12,380
not only the sum but you also have a

143
00:07:10,370 --> 00:07:15,530
bias that is fed into the transfer

144
00:07:12,380 --> 00:07:18,170
function okay which is known as the F so

145
00:07:15,530 --> 00:07:20,630
you have here which is the sum summation

146
00:07:18,170 --> 00:07:23,000
each of the inputs with the weights and

147
00:07:20,630 --> 00:07:24,469
there is some bias here which is fed

148
00:07:23,000 --> 00:07:27,080
into what is known as the transfer

149
00:07:24,470 --> 00:07:28,910
function a lot of you might be familiar

150
00:07:27,080 --> 00:07:30,890
with what is a transfer function from a

151
00:07:28,910 --> 00:07:33,740
signal processing perspective its output

152
00:07:30,890 --> 00:07:35,750
over input of course in this case it's a

153
00:07:33,740 --> 00:07:38,330
nonlinear transfer function for the most

154
00:07:35,750 --> 00:07:41,060
part which is like an log sigmoid or at

155
00:07:38,330 --> 00:07:44,539
an sigmoid or you may also end up having

156
00:07:41,060 --> 00:07:46,420
a pure linear transfer function the

157
00:07:44,540 --> 00:07:49,100
choice of this transfer function is

158
00:07:46,420 --> 00:07:51,920
dependent on the user based on his

159
00:07:49,100 --> 00:07:54,110
application and every neuron uses any

160
00:07:51,920 --> 00:07:56,600
differentiable transfer function to be

161
00:07:54,110 --> 00:07:58,880
able to generate their output yeah so

162
00:07:56,600 --> 00:08:01,070
what do we have here the weights the

163
00:07:58,880 --> 00:08:03,740
bias and then the transfer function so

164
00:08:01,070 --> 00:08:07,820
we have our in our elements in the input

165
00:08:03,740 --> 00:08:09,860
vector now what happens here is we have

166
00:08:07,820 --> 00:08:11,930
a single layer architecture which

167
00:08:09,860 --> 00:08:14,690
consists of the following parts that is

168
00:08:11,930 --> 00:08:17,360
the input layer the weight connectors

169
00:08:14,690 --> 00:08:19,969
which are there in a crisscross yeah and

170
00:08:17,360 --> 00:08:22,430
then you have the summation followed by

171
00:08:19,970 --> 00:08:24,260
the bias okay and then you have the

172
00:08:22,430 --> 00:08:27,050
activation layer which is essentially

173
00:08:24,260 --> 00:08:30,110
the transfer function and then finally

174
00:08:27,050 --> 00:08:32,390
the output okay so this essentially

175
00:08:30,110 --> 00:08:35,570
takes into account maybe what it might

176
00:08:32,390 --> 00:08:37,760
be for a log sig function and things

177
00:08:35,570 --> 00:08:40,000
like that so this is with respect to

178
00:08:37,760 --> 00:08:45,170
each of these inputs coming in

179
00:08:40,000 --> 00:08:48,260
separately yeah but MATLAB is a vector

180
00:08:45,170 --> 00:08:50,750
or you know a lot of the inputs can be

181
00:08:48,260 --> 00:08:52,520
vectorized you don't need to end up

182
00:08:50,750 --> 00:08:54,530
writing for loops for each of the

183
00:08:52,520 --> 00:08:56,569
vectors you don't know it's a MATLAB

184
00:08:54,530 --> 00:08:58,699
stands for matrix laboratory so a lot of

185
00:08:56,570 --> 00:09:01,580
you know that matrix multiplication is

186
00:08:58,700 --> 00:09:04,250
simple so what we can do is we can

187
00:09:01,580 --> 00:09:06,410
combine all the RS that is the input to

188
00:09:04,250 --> 00:09:09,050
one particular vector which is also

189
00:09:06,410 --> 00:09:11,600
their SP so you have R by one which is a

190
00:09:09,050 --> 00:09:14,750
P and then W is essentially the way

191
00:09:11,600 --> 00:09:16,850
to equate matrix which is say s by r and

192
00:09:14,750 --> 00:09:20,180
you have the bias here which is

193
00:09:16,850 --> 00:09:22,699
essentially b12b s which is also there

194
00:09:20,180 --> 00:09:24,979
as a vector which is of course a column

195
00:09:22,699 --> 00:09:27,170
vector this is also a column vector and

196
00:09:24,980 --> 00:09:29,750
then when you sum it you end up getting

197
00:09:27,170 --> 00:09:46,689
what the output is and that can be done

198
00:09:29,750 --> 00:09:46,690
a fast transfer function let me try yes

199
00:09:47,199 --> 00:09:58,459
so what we would sorry I think now you

200
00:09:55,160 --> 00:10:00,949
are here yeah so what we'd like to do

201
00:09:58,459 --> 00:10:03,229
ideally is to ensure that we set up the

202
00:10:00,949 --> 00:10:06,380
problem in such a way that it's all seen

203
00:10:03,230 --> 00:10:08,180
as a vector so you have R which is an

204
00:10:06,380 --> 00:10:10,459
input vector which is sorry P which is

205
00:10:08,180 --> 00:10:13,189
an input vector of size R the weights

206
00:10:10,459 --> 00:10:15,739
which are of size S by r the bias which

207
00:10:13,190 --> 00:10:17,930
are of size s by 1 which is a column

208
00:10:15,740 --> 00:10:20,720
vector and the output becomes s by 1 and

209
00:10:17,930 --> 00:10:22,459
which is password transfer function and

210
00:10:20,720 --> 00:10:25,850
you get this as well so this is

211
00:10:22,459 --> 00:10:29,599
essentially what the overall or we you

212
00:10:25,850 --> 00:10:32,000
will pass it in MATLAB to the you know

213
00:10:29,600 --> 00:10:33,649
in terms of vectors yeah because you

214
00:10:32,000 --> 00:10:37,810
don't want to take each of these inputs

215
00:10:33,649 --> 00:10:41,569
now this is a single layer architecture

216
00:10:37,810 --> 00:10:44,089
what happens within a few layers you

217
00:10:41,569 --> 00:10:46,819
know if you want to configure factors

218
00:10:44,089 --> 00:10:49,160
within a network you can say I have an

219
00:10:46,819 --> 00:10:51,380
input layer I have a hidden layer so you

220
00:10:49,160 --> 00:10:53,449
will essentially see a few things in

221
00:10:51,380 --> 00:10:54,649
terms of what the matrices are we'll

222
00:10:53,449 --> 00:10:56,810
come to that this is I

223
00:10:54,649 --> 00:10:58,130
w11 which is an identity matrix so

224
00:10:56,810 --> 00:11:00,529
you'll essentially have diagonal weights

225
00:10:58,130 --> 00:11:02,660
and then you have the bias or that is

226
00:11:00,529 --> 00:11:06,670
there here similarly what the output

227
00:11:02,660 --> 00:11:09,920
layer is so you have this as well and

228
00:11:06,670 --> 00:11:12,050
what this will give you what this input

229
00:11:09,920 --> 00:11:14,779
have what this network has is it has an

230
00:11:12,050 --> 00:11:17,508
input layer with two inputs it has a

231
00:11:14,779 --> 00:11:19,670
hidden layer with four neurons that is

232
00:11:17,509 --> 00:11:21,800
for rates and for biases which is why

233
00:11:19,670 --> 00:11:25,069
you see a four by two and a four by one

234
00:11:21,800 --> 00:11:27,680
and the activation here is a dancing

235
00:11:25,070 --> 00:11:31,370
Laer yeah and it's one line of command

236
00:11:27,680 --> 00:11:33,709
here which takes into account W the P

237
00:11:31,370 --> 00:11:35,600
that is the input and the B that's the

238
00:11:33,710 --> 00:11:38,150
bias and you just pass it through that

239
00:11:35,600 --> 00:11:40,010
that's my Eva and then the output layer

240
00:11:38,150 --> 00:11:42,620
has three neurons which means it has

241
00:11:40,010 --> 00:11:44,900
three weights and then has three biases

242
00:11:42,620 --> 00:11:47,000
and I get the output as a three vector

243
00:11:44,900 --> 00:11:50,300
and the activation is a pure linear

244
00:11:47,000 --> 00:11:53,060
layer yeah so this is how you can

245
00:11:50,300 --> 00:11:55,520
configure what the input is going to be

246
00:11:53,060 --> 00:11:57,829
in this case one vector what the hidden

247
00:11:55,520 --> 00:11:59,329
layer is which has four neurons and what

248
00:11:57,830 --> 00:12:01,130
the output layer is which has three

249
00:11:59,330 --> 00:12:03,080
neurons this is how you end up

250
00:12:01,130 --> 00:12:06,560
conferring configuring the two layers

251
00:12:03,080 --> 00:12:11,210
that is e1 and e2 in the form of two

252
00:12:06,560 --> 00:12:14,000
particular lines of code yeah okay now I

253
00:12:11,210 --> 00:12:15,950
want to draw your attention to what is

254
00:12:14,000 --> 00:12:19,540
there in the documentation in the form

255
00:12:15,950 --> 00:12:23,230
of other network architectures yeah so

256
00:12:19,540 --> 00:12:25,699
we have a very very powerful

257
00:12:23,230 --> 00:12:28,760
documentation very well documented set

258
00:12:25,700 --> 00:12:31,310
of features within MATLAB and I'd like

259
00:12:28,760 --> 00:12:34,400
to show you what we have here ok of

260
00:12:31,310 --> 00:12:36,380
course I am in India so you can access

261
00:12:34,400 --> 00:12:39,140
it from the web or you can also access

262
00:12:36,380 --> 00:12:41,270
it from our you know from within MATLAB

263
00:12:39,140 --> 00:12:43,939
so in this case I'm just defining the

264
00:12:41,270 --> 00:12:46,220
shallow neural network architectures so

265
00:12:43,940 --> 00:12:48,020
it actually shows what is a historical

266
00:12:46,220 --> 00:12:50,300
and alternative so you have the adaptive

267
00:12:48,020 --> 00:12:52,730
neural network you have the radial basis

268
00:12:50,300 --> 00:12:55,699
neural network perceptron if you want

269
00:12:52,730 --> 00:12:58,100
linear hopfield probabilistic any of

270
00:12:55,700 --> 00:13:01,130
those let's go into the radial basis

271
00:12:58,100 --> 00:13:02,660
neural network it shows you the same

272
00:13:01,130 --> 00:13:04,820
thing that I had over here what the

273
00:13:02,660 --> 00:13:06,680
neuron model is it says what the

274
00:13:04,820 --> 00:13:09,530
transfer function for the radial basis

275
00:13:06,680 --> 00:13:11,900
function is how does the plot occur the

276
00:13:09,530 --> 00:13:16,850
rad mass transfer function I can zoom in

277
00:13:11,900 --> 00:13:18,530
a little more and then you know it says

278
00:13:16,850 --> 00:13:20,930
what is a neural network architecture

279
00:13:18,530 --> 00:13:23,390
you know it says I have the input layer

280
00:13:20,930 --> 00:13:26,239
which is of course the same the radial

281
00:13:23,390 --> 00:13:29,060
basis here is of one particular unlinked

282
00:13:26,240 --> 00:13:31,340
and the only change I have here is I use

283
00:13:29,060 --> 00:13:33,739
the function called rad bass and the

284
00:13:31,340 --> 00:13:35,120
linear layer which is essentially the

285
00:13:33,740 --> 00:13:38,570
second layer here which is the output

286
00:13:35,120 --> 00:13:39,140
layer over here and it gives me enough

287
00:13:38,570 --> 00:13:42,470
information

288
00:13:39,140 --> 00:13:44,300
on how do I set it up so the output of

289
00:13:42,470 --> 00:13:47,000
the first layer can be obtained by the

290
00:13:44,300 --> 00:13:50,300
following code what do I have here so

291
00:13:47,000 --> 00:13:52,850
essentially I'm gonna say rad bass yeah

292
00:13:50,300 --> 00:13:55,670
and I'm essentially gonna say take the

293
00:13:52,850 --> 00:13:58,250
take the distance and I'll be sending

294
00:13:55,670 --> 00:14:00,260
you this information but it's just as

295
00:13:58,250 --> 00:14:04,399
one particular line of code which takes

296
00:14:00,260 --> 00:14:06,560
into account one the bias then to the

297
00:14:04,399 --> 00:14:08,630
input and what the value here is and

298
00:14:06,560 --> 00:14:10,849
then overall take the net earth or

299
00:14:08,630 --> 00:14:14,959
uniform the network and then use the

300
00:14:10,850 --> 00:14:17,089
radial basis function yeah so in some

301
00:14:14,959 --> 00:14:19,430
way you don't need to end up writing

302
00:14:17,089 --> 00:14:21,920
these types of lines of code it's much

303
00:14:19,430 --> 00:14:23,810
easier in some time you will see how

304
00:14:21,920 --> 00:14:26,990
much easier it is to use the Brittain

305
00:14:23,810 --> 00:14:29,329
functions or apps that we have but what

306
00:14:26,990 --> 00:14:33,680
I wanted to show you is if you want to

307
00:14:29,329 --> 00:14:36,109
understand now each of these is done you

308
00:14:33,680 --> 00:14:41,000
only need to end up writing net equals

309
00:14:36,110 --> 00:14:43,220
new radial basis over here B and I have

310
00:14:41,000 --> 00:14:45,860
the target vectors that RTP is the input

311
00:14:43,220 --> 00:14:48,800
vector T is the target vector and spread

312
00:14:45,860 --> 00:14:52,100
is the spread constant which is for the

313
00:14:48,800 --> 00:14:54,890
radial basis layer yeah and it creates

314
00:14:52,100 --> 00:14:57,170
many neurons and you know it ends up

315
00:14:54,890 --> 00:15:00,019
showing a lot of things that are seen

316
00:14:57,170 --> 00:15:02,660
here as the output but what I would like

317
00:15:00,019 --> 00:15:04,579
to just emphasize here is it's one or

318
00:15:02,660 --> 00:15:06,529
two lines of code that you need to use

319
00:15:04,579 --> 00:15:09,430
you don't need to end up writing a lot

320
00:15:06,529 --> 00:15:11,839
of code and this is why we end up

321
00:15:09,430 --> 00:15:14,630
positioning what is known as a neural

322
00:15:11,839 --> 00:15:15,620
network toolbox for neural networks of

323
00:15:14,630 --> 00:15:18,589
course you can write your own

324
00:15:15,620 --> 00:15:21,079
functionalities but once you or the

325
00:15:18,589 --> 00:15:23,029
students have understood what um you

326
00:15:21,079 --> 00:15:25,250
know the concept behind neural networks

327
00:15:23,029 --> 00:15:26,959
is it's easier for you to use the

328
00:15:25,250 --> 00:15:28,790
built-in functions of course when you

329
00:15:26,959 --> 00:15:30,500
teach them initially when you teach

330
00:15:28,790 --> 00:15:32,300
students or when you're trying out do

331
00:15:30,500 --> 00:15:34,220
not use the functions because I believe

332
00:15:32,300 --> 00:15:36,199
that if someone just uses a

333
00:15:34,220 --> 00:15:37,670
functionality and does not understand

334
00:15:36,199 --> 00:15:39,500
the background behind the functionality

335
00:15:37,670 --> 00:15:41,149
there is no point and most of us are

336
00:15:39,500 --> 00:15:43,070
educators here who would like to ensure

337
00:15:41,149 --> 00:15:47,690
that the student understands ik concept

338
00:15:43,070 --> 00:15:49,520
yeah so that is another thing so so

339
00:15:47,690 --> 00:15:51,920
we'll be looking at the radial basis or

340
00:15:49,520 --> 00:15:52,750
neural networks over here so the radial

341
00:15:51,920 --> 00:15:55,360
basis function

342
00:15:52,750 --> 00:15:56,860
essentially what we do is it has a

343
00:15:55,360 --> 00:15:59,340
radial basis new on the transfer

344
00:15:56,860 --> 00:16:01,630
function here is rad bass of n which is

345
00:15:59,340 --> 00:16:04,840
represented this way which is the radial

346
00:16:01,630 --> 00:16:08,439
basis function it has a maximum one when

347
00:16:04,840 --> 00:16:11,800
the input is zero and when the distance

348
00:16:08,440 --> 00:16:14,020
between W and P reduces the output

349
00:16:11,800 --> 00:16:16,449
increases okay and there's more

350
00:16:14,020 --> 00:16:18,670
information on what the radial basis

351
00:16:16,450 --> 00:16:20,050
neuron does as a detector and what the

352
00:16:18,670 --> 00:16:22,300
bias does with respect to the

353
00:16:20,050 --> 00:16:24,219
sensitivity and all that and I'll be

354
00:16:22,300 --> 00:16:27,189
passing on these slides to you as all

355
00:16:24,220 --> 00:16:28,900
you know after the session so this is

356
00:16:27,190 --> 00:16:31,840
something that's there in the theory of

357
00:16:28,900 --> 00:16:35,800
the radial basis noodle network yeah now

358
00:16:31,840 --> 00:16:39,220
what I want to show you is a couple of

359
00:16:35,800 --> 00:16:41,829
examples as mama requested on how you

360
00:16:39,220 --> 00:16:44,470
can use the neural network toolbox the

361
00:16:41,830 --> 00:16:46,030
built in apps and all of that stuff but

362
00:16:44,470 --> 00:16:48,310
before that I want to go over the

363
00:16:46,030 --> 00:16:50,290
workflow so I'll come back to this in

364
00:16:48,310 --> 00:16:52,750
just a little bit this example for a

365
00:16:50,290 --> 00:16:55,020
time series or or or time delay neural

366
00:16:52,750 --> 00:16:57,610
network but I wanna point out the

367
00:16:55,020 --> 00:16:59,770
workflow for using neural networks can

368
00:16:57,610 --> 00:17:02,350
someone tell me what would be the first

369
00:16:59,770 --> 00:17:11,589
step in using a neural network a first

370
00:17:02,350 --> 00:17:13,420
step in any mathematics problem awesome

371
00:17:11,589 --> 00:17:15,220
and I'm so happy that you said because

372
00:17:13,420 --> 00:17:18,339
I'm so happy that you didn't tell it was

373
00:17:15,220 --> 00:17:20,170
the algorithm it's typically what we

374
00:17:18,339 --> 00:17:22,869
need to collect us data that is the most

375
00:17:20,170 --> 00:17:25,000
important thing the next step is to

376
00:17:22,869 --> 00:17:26,800
create the network yeah and we will take

377
00:17:25,000 --> 00:17:28,960
into account a lot of criteria when we

378
00:17:26,800 --> 00:17:30,460
need to create the network next what we

379
00:17:28,960 --> 00:17:32,800
do is we need to configure the network

380
00:17:30,460 --> 00:17:34,270
you know we'll need to see which is the

381
00:17:32,800 --> 00:17:35,680
algorithm that would go and behind and

382
00:17:34,270 --> 00:17:37,150
creating the network you would say how

383
00:17:35,680 --> 00:17:38,080
many layers you need when configuring

384
00:17:37,150 --> 00:17:39,790
you might need to say what does a

385
00:17:38,080 --> 00:17:41,230
minimization think that you need to do

386
00:17:39,790 --> 00:17:43,899
or what algorithm you might need to use

387
00:17:41,230 --> 00:17:45,580
and then you initialize the weights and

388
00:17:43,900 --> 00:17:47,860
the biases because it's not going to

389
00:17:45,580 --> 00:17:50,620
work right away in the first instance

390
00:17:47,860 --> 00:17:52,270
yeah and then you end up training the

391
00:17:50,620 --> 00:18:04,689
network can someone tell me what this

392
00:17:52,270 --> 00:18:06,720
training process means why would you do

393
00:18:04,690 --> 00:18:06,720
that

394
00:18:17,370 --> 00:18:20,469
[Music]

395
00:18:26,650 --> 00:18:32,059
any other answer over all why would you

396
00:18:30,140 --> 00:18:39,640
need to perform any form of training on

397
00:18:32,059 --> 00:18:41,870
a data set so then it should behave yes

398
00:18:39,640 --> 00:18:44,299
or the algorithm behaves the way you

399
00:18:41,870 --> 00:18:45,949
want it to you give it sample data and

400
00:18:44,299 --> 00:18:47,540
you say that this is me this is a ground

401
00:18:45,950 --> 00:18:49,460
truth ensure that you adjust the weights

402
00:18:47,540 --> 00:18:52,070
in such a way that if you end up finding

403
00:18:49,460 --> 00:18:54,169
similar data ensure that you classify

404
00:18:52,070 --> 00:18:55,850
properly that's in the classification so

405
00:18:54,169 --> 00:18:57,620
not you write the same thing over here

406
00:18:55,850 --> 00:19:00,139
as well we want to ensure that the

407
00:18:57,620 --> 00:19:02,780
network is trained because we want the

408
00:19:00,140 --> 00:19:05,570
target and the output to match as much

409
00:19:02,780 --> 00:19:07,668
as possible so that will be the training

410
00:19:05,570 --> 00:19:09,260
process following that you will validate

411
00:19:07,669 --> 00:19:10,520
the next book what what do you mean by

412
00:19:09,260 --> 00:19:12,530
validating the network you would have

413
00:19:10,520 --> 00:19:14,660
set the weights you might be giving new

414
00:19:12,530 --> 00:19:16,129
data yeah the old data was used to train

415
00:19:14,660 --> 00:19:18,200
but you might be giving new data to

416
00:19:16,130 --> 00:19:20,660
validate the network and then see what

417
00:19:18,200 --> 00:19:23,000
the error coefficient maybe is and then

418
00:19:20,660 --> 00:19:25,160
finally you use the network is this

419
00:19:23,000 --> 00:19:27,640
clear to everyone of you like any

420
00:19:25,160 --> 00:19:31,220
questions on this front this is the

421
00:19:27,640 --> 00:19:32,840
classic statistics workflow you know it

422
00:19:31,220 --> 00:19:34,700
need not be just neural networks it can

423
00:19:32,840 --> 00:19:37,250
be for any classification algorithm

424
00:19:34,700 --> 00:19:39,260
except that in that case it may not be a

425
00:19:37,250 --> 00:19:41,270
network it will be the choice of an

426
00:19:39,260 --> 00:19:43,160
algorithm choice of the parameters and

427
00:19:41,270 --> 00:19:45,320
then training the classifier and

428
00:19:43,160 --> 00:19:50,870
validating it so any questions at this

429
00:19:45,320 --> 00:19:53,540
point of time on this workflow ok I will

430
00:19:50,870 --> 00:19:56,059
assume that you know or the answer is no

431
00:19:53,540 --> 00:19:58,460
I have a few screenshots on what will be

432
00:19:56,059 --> 00:20:00,860
the process but I will also be walking

433
00:19:58,460 --> 00:20:02,660
you through each of them our neural

434
00:20:00,860 --> 00:20:02,990
network our classifiers that are over

435
00:20:02,660 --> 00:20:05,570
there

436
00:20:02,990 --> 00:20:08,540
so before that I want to say how do you

437
00:20:05,570 --> 00:20:10,760
collect data so the key thing which is

438
00:20:08,540 --> 00:20:13,010
very nice about MATLAB is that you are

439
00:20:10,760 --> 00:20:15,740
able to access data into MATLAB in many

440
00:20:13,010 --> 00:20:17,629
ways it need not be just a mat file it

441
00:20:15,740 --> 00:20:19,720
can be from an excel sheet it could be

442
00:20:17,630 --> 00:20:22,340
text it could be binary it could be

443
00:20:19,720 --> 00:20:23,990
audio/video files images a lot of times

444
00:20:22,340 --> 00:20:25,519
now there are images that need to be

445
00:20:23,990 --> 00:20:28,580
classified or images that need to be

446
00:20:25,519 --> 00:20:31,429
trained so deep learning is very very uh

447
00:20:28,580 --> 00:20:32,990
you know um useful in computer vision

448
00:20:31,429 --> 00:20:33,860
problems we'll take a look at that in

449
00:20:32,990 --> 00:20:35,570
just a little bit

450
00:20:33,860 --> 00:20:37,639
you also can read from scientific

451
00:20:35,570 --> 00:20:39,980
formats like extended markup language

452
00:20:37,639 --> 00:20:42,860
you can read data from multiple

453
00:20:39,980 --> 00:20:44,750
databases or from other languages or if

454
00:20:42,860 --> 00:20:47,510
you have measurement hardware like say

455
00:20:44,750 --> 00:20:51,049
um you know if you need to acquire data

456
00:20:47,510 --> 00:20:53,210
serially or you know you store it and

457
00:20:51,049 --> 00:20:54,590
afterwards you take it in you can do it

458
00:20:53,210 --> 00:20:56,480
using what is known as the data

459
00:20:54,590 --> 00:21:00,860
acquisition toolbox or the instrument

460
00:20:56,480 --> 00:21:02,840
control toolbox yeah now so MATLAB as

461
00:21:00,860 --> 00:21:04,850
such as I mentioned there is something

462
00:21:02,840 --> 00:21:06,740
known as apps which are over there I'll

463
00:21:04,850 --> 00:21:08,809
just show you a bunch of screenshots and

464
00:21:06,740 --> 00:21:11,360
then we will walk through the whole

465
00:21:08,809 --> 00:21:13,158
process right away so you need to you

466
00:21:11,360 --> 00:21:15,529
you have an app which I'll also show you

467
00:21:13,159 --> 00:21:18,380
in MATLAB you have an app which does the

468
00:21:15,529 --> 00:21:21,019
neural network start you can choose what

469
00:21:18,380 --> 00:21:23,059
sort of a thing that what what sort of

470
00:21:21,019 --> 00:21:26,960
an application you need to perform in

471
00:21:23,059 --> 00:21:28,549
with respect to neural networks next you

472
00:21:26,960 --> 00:21:29,750
end up getting say if you're looking at

473
00:21:28,549 --> 00:21:31,549
the fitting tool you may get a

474
00:21:29,750 --> 00:21:33,529
screenshot like this and it says you

475
00:21:31,549 --> 00:21:35,360
know what are what is a what would why

476
00:21:33,529 --> 00:21:37,700
you need this and what is the neural

477
00:21:35,360 --> 00:21:40,760
network wouldn't look like and then you

478
00:21:37,700 --> 00:21:43,580
can select data and I'll show you in in

479
00:21:40,760 --> 00:21:46,370
in the next few minutes on how you can

480
00:21:43,580 --> 00:21:49,100
load data I might use a example data set

481
00:21:46,370 --> 00:21:51,110
and after which I can select you know I

482
00:21:49,100 --> 00:21:53,059
can partition the data for training and

483
00:21:51,110 --> 00:21:55,100
validation if you realize we mentioned

484
00:21:53,059 --> 00:21:57,470
that we might need to train a little bit

485
00:21:55,100 --> 00:21:59,240
and then validate it and then test it so

486
00:21:57,470 --> 00:22:01,220
we split it up in such a way that a lot

487
00:21:59,240 --> 00:22:02,779
of data is used for training and some is

488
00:22:01,220 --> 00:22:06,350
used for validation and some is used for

489
00:22:02,779 --> 00:22:08,120
testing yeah and then you can define the

490
00:22:06,350 --> 00:22:09,980
basic network architecture we'll come to

491
00:22:08,120 --> 00:22:12,500
that and then choose a training

492
00:22:09,980 --> 00:22:13,880
algorithm I'll come to that also in just

493
00:22:12,500 --> 00:22:16,490
a little bit you can you have help

494
00:22:13,880 --> 00:22:19,429
documentation in every component at this

495
00:22:16,490 --> 00:22:21,409
point of time and there are multiple

496
00:22:19,429 --> 00:22:23,659
training algorithms that are over there

497
00:22:21,409 --> 00:22:26,929
starting from a basic gradient descent

498
00:22:23,659 --> 00:22:29,149
to say or the Levenberg Marquardt model

499
00:22:26,929 --> 00:22:31,129
or you can define your own training

500
00:22:29,149 --> 00:22:33,969
algorithms as well so you have support

501
00:22:31,129 --> 00:22:37,149
from existing algorithms and also

502
00:22:33,970 --> 00:22:40,299
are you know from but your own user to

503
00:22:37,149 --> 00:22:42,610
find our training algorithms the overall

504
00:22:40,299 --> 00:22:46,029
learning workflow which I want to point

505
00:22:42,610 --> 00:22:48,908
out is you iterate till you find the

506
00:22:46,029 --> 00:22:52,059
best model that is you have your data

507
00:22:48,909 --> 00:22:53,860
you pre-process the data in whatever way

508
00:22:52,059 --> 00:22:55,210
you want you may be able to perform

509
00:22:53,860 --> 00:22:56,949
supervised learning class of

510
00:22:55,210 --> 00:23:01,179
classification regression and get the

511
00:22:56,950 --> 00:23:04,000
model this is an iterative process okay

512
00:23:01,179 --> 00:23:05,529
in the prediction what you end up doing

513
00:23:04,000 --> 00:23:08,379
is you have the new data

514
00:23:05,529 --> 00:23:10,750
you take that pre-process thing and you

515
00:23:08,379 --> 00:23:13,149
take that particular model and are able

516
00:23:10,750 --> 00:23:16,659
to predict it so the training step is

517
00:23:13,149 --> 00:23:18,580
very very important in this case okay

518
00:23:16,659 --> 00:23:20,379
and of course you have the option of

519
00:23:18,580 --> 00:23:22,990
evaluating the network we will come to

520
00:23:20,379 --> 00:23:25,240
that in some time and then finally be

521
00:23:22,990 --> 00:23:29,889
able to generate MATLAB code or Simulink

522
00:23:25,240 --> 00:23:32,379
model automatically ok so I want to go

523
00:23:29,889 --> 00:23:35,110
back to the same I want to go back to

524
00:23:32,379 --> 00:23:37,899
that set of things where we said what

525
00:23:35,110 --> 00:23:40,479
each of the things is so we will collect

526
00:23:37,899 --> 00:23:42,039
the data create the network configure

527
00:23:40,480 --> 00:23:44,440
the network initialize the weights and

528
00:23:42,039 --> 00:23:47,649
biases strain it validate it and use the

529
00:23:44,440 --> 00:23:49,840
network yeah let's switch to MATLAB for

530
00:23:47,649 --> 00:23:52,928
the next ten to fifteen minutes I'm

531
00:23:49,840 --> 00:23:55,899
going to be using MATLAB oh you know

532
00:23:52,929 --> 00:23:57,700
just so that you're able to see what is

533
00:23:55,899 --> 00:24:00,008
being done I'm not a very big fan of

534
00:23:57,700 --> 00:24:02,710
only using PowerPoint slides so I prefer

535
00:24:00,009 --> 00:24:04,919
using up moving here okay

536
00:24:02,710 --> 00:24:07,870
how many of you are familiar with this

537
00:24:04,919 --> 00:24:10,419
setup of MATLAB can I have a you know

538
00:24:07,870 --> 00:24:12,158
show of her or rather can people tell me

539
00:24:10,419 --> 00:24:19,629
what's a version of MATLAB people are

540
00:24:12,159 --> 00:24:23,230
using 17b here you see in our Institute

541
00:24:19,629 --> 00:24:31,889
also we are using 14 okay and any other

542
00:24:23,230 --> 00:24:34,929
answer any other centers okay sure so um

543
00:24:31,889 --> 00:24:37,918
every year there are two releases in

544
00:24:34,929 --> 00:24:40,179
MATLAB so one of the releases is called

545
00:24:37,919 --> 00:24:43,629
2017 and the other release is called

546
00:24:40,179 --> 00:24:45,460
2017 this year in in another two weeks

547
00:24:43,629 --> 00:24:46,779
there'll be a release called or in a

548
00:24:45,460 --> 00:24:48,260
week there'll be a release called our

549
00:24:46,779 --> 00:24:49,610
2018 a

550
00:24:48,260 --> 00:24:53,810
and in September you will have a release

551
00:24:49,610 --> 00:24:56,060
called our 2018 B yeah and there are

552
00:24:53,810 --> 00:24:59,120
updates to the functionalities and

553
00:24:56,060 --> 00:25:03,379
features especially the neural network

554
00:24:59,120 --> 00:25:06,050
toolbox had a huge you know change in 17

555
00:25:03,380 --> 00:25:07,730
a and 17 B because neural network again

556
00:25:06,050 --> 00:25:09,379
got a lot of attention because of deep

557
00:25:07,730 --> 00:25:12,080
learning because neural networks was

558
00:25:09,380 --> 00:25:14,180
almost branded as a dead area like in

559
00:25:12,080 --> 00:25:16,100
you know the early 2000s but in the last

560
00:25:14,180 --> 00:25:19,640
four to five years there's been a lot of

561
00:25:16,100 --> 00:25:21,199
you know um interest back in neural

562
00:25:19,640 --> 00:25:22,550
networks will see why that is that's

563
00:25:21,200 --> 00:25:23,900
because of deep learning and we'll see

564
00:25:22,550 --> 00:25:26,180
what the reason for that is when I

565
00:25:23,900 --> 00:25:29,480
switch to the deep learning deep

566
00:25:26,180 --> 00:25:33,320
learning topic but what but the reason I

567
00:25:29,480 --> 00:25:35,120
asked you is in in 2012 B that was

568
00:25:33,320 --> 00:25:38,149
around five years back we introduced

569
00:25:35,120 --> 00:25:42,139
this tabs here at one of the tabs here

570
00:25:38,150 --> 00:25:45,290
is called as the apps okay and if you're

571
00:25:42,140 --> 00:25:48,080
not able to see it let me try to change

572
00:25:45,290 --> 00:25:50,149
my display setting to ensure that people

573
00:25:48,080 --> 00:25:52,610
are you you are able to see

574
00:25:50,150 --> 00:25:54,950
I'd like to ensure that all of you are

575
00:25:52,610 --> 00:25:59,270
able to see it

576
00:25:54,950 --> 00:26:02,420
is this better yes okay so you see

577
00:25:59,270 --> 00:26:04,550
something known as apps and in this apps

578
00:26:02,420 --> 00:26:06,650
if you notice I have marked a few

579
00:26:04,550 --> 00:26:09,379
favorites here which has curve fitting

580
00:26:06,650 --> 00:26:12,200
distribution fitter optimization I

581
00:26:09,380 --> 00:26:13,910
created my own app which is called our

582
00:26:12,200 --> 00:26:15,230
you know NEC app I'll show you what that

583
00:26:13,910 --> 00:26:17,330
is I can show you how to create an app

584
00:26:15,230 --> 00:26:19,670
later in the match starts an

585
00:26:17,330 --> 00:26:22,159
optimization you have like multiple apps

586
00:26:19,670 --> 00:26:24,650
which is neural network clustering or

587
00:26:22,160 --> 00:26:26,750
neural networks fitting neural net for

588
00:26:24,650 --> 00:26:30,290
pattern recognition or your neural

589
00:26:26,750 --> 00:26:32,390
network times CDs yeah but let's not do

590
00:26:30,290 --> 00:26:36,290
that and let's start with something else

591
00:26:32,390 --> 00:26:40,100
that I have the way I do is I just type

592
00:26:36,290 --> 00:26:44,440
neural network start yeah and then start

593
00:26:40,100 --> 00:26:44,439
can anyone tell me what I might get here

594
00:26:49,240 --> 00:26:53,990
has anyone used this you have used it

595
00:26:52,450 --> 00:26:57,740
okay sure

596
00:26:53,990 --> 00:27:00,020
now in this in this you can see what the

597
00:26:57,740 --> 00:27:01,850
you know more information is you can see

598
00:27:00,020 --> 00:27:03,440
what the examples are or what the data

599
00:27:01,850 --> 00:27:05,510
let's go and take a look at what these

600
00:27:03,440 --> 00:27:08,630
okay this seems to not work with in

601
00:27:05,510 --> 00:27:10,340
terms of clicking it opened so if you

602
00:27:08,630 --> 00:27:11,929
notice it it showed you know there's a

603
00:27:10,340 --> 00:27:13,970
lot of datasets that's already there

604
00:27:11,929 --> 00:27:16,190
there's some body fat our dataset or

605
00:27:13,970 --> 00:27:18,890
chemical data set let's look at what

606
00:27:16,190 --> 00:27:21,490
this body fat data set is is you know

607
00:27:18,890 --> 00:27:24,320
it's body fat percentage data set so

608
00:27:21,490 --> 00:27:26,900
essentially it loads are you know around

609
00:27:24,320 --> 00:27:30,678
arm it loads to variables which is

610
00:27:26,900 --> 00:27:32,950
essentially for 252 people 13 parameters

611
00:27:30,679 --> 00:27:36,049
that is their age weight height and

612
00:27:32,950 --> 00:27:40,340
multiple things about their body yeah

613
00:27:36,049 --> 00:27:41,480
and then essentially I have the target

614
00:27:40,340 --> 00:27:44,149
which is essentially going to be

615
00:27:41,480 --> 00:27:46,610
estimated from the input what are the

616
00:27:44,150 --> 00:27:48,169
body fat target is it has here I have

617
00:27:46,610 --> 00:27:49,490
all these circumference a height weight

618
00:27:48,169 --> 00:27:52,610
and everything and I need to get what

619
00:27:49,490 --> 00:27:54,530
what the body fat target is so I can I

620
00:27:52,610 --> 00:27:56,389
can get what data sets of ants or what

621
00:27:54,530 --> 00:28:00,408
the examples are and things like that

622
00:27:56,390 --> 00:28:02,570
okay now shall we take maybe an example

623
00:28:00,409 --> 00:28:04,400
of the fitting up initially so you can

624
00:28:02,570 --> 00:28:07,428
type an ft tool if you want or I can

625
00:28:04,400 --> 00:28:12,500
just click on that okay who can tell me

626
00:28:07,429 --> 00:28:14,650
okay let me try to sorry I don't mean to

627
00:28:12,500 --> 00:28:14,650
do that

628
00:28:18,190 --> 00:28:28,010
okay okay unfortunately I'm not able to

629
00:28:25,789 --> 00:28:32,450
see the few of the buttons that I need

630
00:28:28,010 --> 00:28:35,270
to see i'm unfortunately gonna have to

631
00:28:32,450 --> 00:28:37,700
reset my display thing to a little

632
00:28:35,270 --> 00:28:40,100
lesser because it doesn't seem to show

633
00:28:37,700 --> 00:28:46,549
but if an if in case you're not able to

634
00:28:40,100 --> 00:28:49,399
see please do let me know okay you able

635
00:28:46,549 --> 00:28:56,570
to see it still yeah okay so if you

636
00:28:49,400 --> 00:28:58,820
notice here in zoom in sometimes so so

637
00:28:56,570 --> 00:29:00,168
if you notice over here it says of

638
00:28:58,820 --> 00:29:02,450
course welcome to the neural network

639
00:29:00,169 --> 00:29:04,100
thing i it will say examples so this is

640
00:29:02,450 --> 00:29:07,909
a neural network so all I need to say is

641
00:29:04,100 --> 00:29:09,918
next yeah and you can get the data from

642
00:29:07,909 --> 00:29:12,440
workspace of course I don't have any

643
00:29:09,919 --> 00:29:13,880
data which is in the workspace okay and

644
00:29:12,440 --> 00:29:15,440
what the target is so I'll say load

645
00:29:13,880 --> 00:29:17,060
example data set

646
00:29:15,440 --> 00:29:19,880
we said we'll look at the body fat

647
00:29:17,060 --> 00:29:23,360
example so we are essentially getting in

648
00:29:19,880 --> 00:29:25,520
a 313 by 252 matrix which essentially

649
00:29:23,360 --> 00:29:27,979
takes into account all of these for 252

650
00:29:25,520 --> 00:29:30,230
people and what the targets which is to

651
00:29:27,980 --> 00:29:32,090
be estimated from the input so I'll

652
00:29:30,230 --> 00:29:34,670
import it okay

653
00:29:32,090 --> 00:29:36,590
and if you notice in terms of the input

654
00:29:34,670 --> 00:29:38,630
it's essentially body fat input data

655
00:29:36,590 --> 00:29:41,419
would check and in terms of output I'd

656
00:29:38,630 --> 00:29:44,090
say body fat target I can either say the

657
00:29:41,420 --> 00:29:49,040
samples are rows or columns so let's

658
00:29:44,090 --> 00:29:53,629
take a look so what is it here in this

659
00:29:49,040 --> 00:29:56,450
case yeah so essentially we will end up

660
00:29:53,630 --> 00:29:58,220
saying that it's it's there as you know

661
00:29:56,450 --> 00:30:00,950
because it's a particular row we'll say

662
00:29:58,220 --> 00:30:03,770
that each of the column has has the data

663
00:30:00,950 --> 00:30:07,310
for one particular person yeah so it

664
00:30:03,770 --> 00:30:08,990
says 252 samples of 13 element and 252

665
00:30:07,310 --> 00:30:10,820
samples of one element we essentially

666
00:30:08,990 --> 00:30:13,040
wanted for one element if we say the

667
00:30:10,820 --> 00:30:15,590
other way round it will say 13 samples

668
00:30:13,040 --> 00:30:18,740
of 252 elements which is wrong so we

669
00:30:15,590 --> 00:30:21,320
wanted the matrix columns okay next I

670
00:30:18,740 --> 00:30:23,960
will go to next okay

671
00:30:21,320 --> 00:30:26,899
now can someone tell me what is

672
00:30:23,960 --> 00:30:29,990
happening in this setup here so I have

673
00:30:26,900 --> 00:30:31,880
the thing where I need to choose some

674
00:30:29,990 --> 00:30:33,560
for training choose some for validation

675
00:30:31,880 --> 00:30:35,510
choose some for testing can someone tell

676
00:30:33,560 --> 00:30:36,950
me what's going to happen there may I

677
00:30:35,510 --> 00:30:40,340
have a question here what yours

678
00:30:36,950 --> 00:30:46,030
invalidation interesting why don't you

679
00:30:40,340 --> 00:30:49,220
tell me what's the validation thing

680
00:30:46,030 --> 00:30:53,930
where we do not know what the answer is

681
00:30:49,220 --> 00:30:55,820
okay given are you able to read this

682
00:30:53,930 --> 00:30:57,590
statement here what is there below

683
00:30:55,820 --> 00:30:59,090
validation they have no effect on

684
00:30:57,590 --> 00:31:00,860
training and so provide an independent

685
00:30:59,090 --> 00:31:02,600
measure of network performance during

686
00:31:00,860 --> 00:31:06,469
and after training not as testing can

687
00:31:02,600 --> 00:31:08,300
you read what is a validation to measure

688
00:31:06,470 --> 00:31:10,100
Network generalization and to wall to

689
00:31:08,300 --> 00:31:13,909
training when generalization stops in

690
00:31:10,100 --> 00:31:16,070
going okay so when so even though the

691
00:31:13,910 --> 00:31:20,030
training is done the validation will

692
00:31:16,070 --> 00:31:22,820
ensure that if there is Network

693
00:31:20,030 --> 00:31:25,520
generalization if it's not improving it

694
00:31:22,820 --> 00:31:27,110
will stop it over there okay so

695
00:31:25,520 --> 00:31:29,220
validation is to ensure that you're

696
00:31:27,110 --> 00:31:31,350
trained training process because

697
00:31:29,220 --> 00:31:33,480
is gonna take a very long time correct

698
00:31:31,350 --> 00:31:36,480
and we want to ensure that the training

699
00:31:33,480 --> 00:31:38,580
is in a mode where the input oh sorry

700
00:31:36,480 --> 00:31:41,130
the output and the target are matching

701
00:31:38,580 --> 00:31:42,750
if they don't match we will not be going

702
00:31:41,130 --> 00:31:45,150
so the validation is essentially where

703
00:31:42,750 --> 00:31:47,400
we test other output against the ground

704
00:31:45,150 --> 00:31:51,110
root to say that hey yes it's working

705
00:31:47,400 --> 00:31:54,090
fine go ahead and train more data okay

706
00:31:51,110 --> 00:31:57,629
now I can go and change it to say

707
00:31:54,090 --> 00:32:00,209
validation as 5% and I'll say I need 25%

708
00:31:57,630 --> 00:32:02,669
of testing the default is essentially 70

709
00:32:00,210 --> 00:32:04,590
15 15 let's go with this but we'll

710
00:32:02,669 --> 00:32:06,179
change it in some time okay

711
00:32:04,590 --> 00:32:07,918
we'll maybe reduce the training thing

712
00:32:06,179 --> 00:32:11,960
and we'll we'll see what the effect is

713
00:32:07,919 --> 00:32:16,020
so any questions at this point of time

714
00:32:11,960 --> 00:32:19,440
okay next I'll go to next and I'll see

715
00:32:16,020 --> 00:32:21,418
oh you know define the fitting Network I

716
00:32:19,440 --> 00:32:26,610
will say what is a number of hidden

717
00:32:21,419 --> 00:32:30,360
neurons okay so if you if you recall in

718
00:32:26,610 --> 00:32:32,549
my arm sorry not here in my thing I

719
00:32:30,360 --> 00:32:36,449
should be able to say how many neurons

720
00:32:32,549 --> 00:32:38,730
do I have or in this case I should be

721
00:32:36,450 --> 00:32:40,320
able to say in this hidden layer what it

722
00:32:38,730 --> 00:32:42,780
is going to be what that n is going to

723
00:32:40,320 --> 00:32:44,100
be are you okay with ten or do you want

724
00:32:42,780 --> 00:32:45,928
to change the number or we'll come back

725
00:32:44,100 --> 00:32:47,908
and take a look at what be what it might

726
00:32:45,929 --> 00:32:51,780
of how it might affect we'll do that

727
00:32:47,909 --> 00:32:54,240
afterwards let it be 10 at this time now

728
00:32:51,780 --> 00:32:56,250
what I do is it it also says

729
00:32:54,240 --> 00:32:57,750
recommendation return to this panel and

730
00:32:56,250 --> 00:32:59,549
change the number of neurons if the

731
00:32:57,750 --> 00:33:02,429
network does not perform well after

732
00:32:59,549 --> 00:33:05,908
training so it's obvious now I go next

733
00:33:02,429 --> 00:33:07,890
and I can choose a training algorithm

734
00:33:05,909 --> 00:33:10,350
there are three of them that I can

735
00:33:07,890 --> 00:33:13,200
choose for this particular example but

736
00:33:10,350 --> 00:33:16,969
as I mentioned there are many many of

737
00:33:13,200 --> 00:33:20,400
those that can be used which is shown in

738
00:33:16,970 --> 00:33:23,100
this particular slide so if you notice I

739
00:33:20,400 --> 00:33:27,000
can choose many many of those algorithms

740
00:33:23,100 --> 00:33:29,699
in this training component okay now let

741
00:33:27,000 --> 00:33:33,390
me choose a default one and what I will

742
00:33:29,700 --> 00:33:35,909
do is I will say train okay so this

743
00:33:33,390 --> 00:33:38,250
Levenberg market algorithm says it

744
00:33:35,909 --> 00:33:40,559
requires more memory but less time and

745
00:33:38,250 --> 00:33:42,810
the training automatically stop stops

746
00:33:40,559 --> 00:33:45,690
when the generalization stops improving

747
00:33:42,810 --> 00:33:47,280
okay or if I want to say Bayesian

748
00:33:45,690 --> 00:33:48,780
regularization it will say it requires

749
00:33:47,280 --> 00:33:51,030
more time but can result in good

750
00:33:48,780 --> 00:33:53,100
generalization for small or noisy data

751
00:33:51,030 --> 00:33:56,790
sets it's essentially shows what it is

752
00:33:53,100 --> 00:33:58,679
and it also says you can train multiple

753
00:33:56,790 --> 00:34:00,960
times but then it will end up generating

754
00:33:58,680 --> 00:34:04,320
different results due to different

755
00:34:00,960 --> 00:34:07,970
initial conditions and sampling okay now

756
00:34:04,320 --> 00:34:11,549
let's train it and what I got

757
00:34:07,970 --> 00:34:13,949
immediately was a network that was

758
00:34:11,550 --> 00:34:18,150
trained like this how many inputs do I

759
00:34:13,949 --> 00:34:21,989
have here 13 and what is the number of

760
00:34:18,150 --> 00:34:24,120
hidden layers then and the output is one

761
00:34:21,989 --> 00:34:26,908
how many iterations do you see over here

762
00:34:24,120 --> 00:34:29,668
it says around six sixteen iterations

763
00:34:26,909 --> 00:34:31,950
okay and I'm able to see how many

764
00:34:29,668 --> 00:34:34,049
validations in it perform six of them

765
00:34:31,949 --> 00:34:35,819
and all six of them are correct so

766
00:34:34,050 --> 00:34:39,090
they're the generalization actually

767
00:34:35,820 --> 00:34:41,730
reduced yeah so I can go here and check

768
00:34:39,090 --> 00:34:45,090
what the mean square error is or the

769
00:34:41,730 --> 00:34:47,610
r-value is it's it's in some format it's

770
00:34:45,090 --> 00:34:50,669
say point eight two or point eight eight

771
00:34:47,610 --> 00:34:52,950
for the training 0.86 for the validation

772
00:34:50,668 --> 00:34:55,529
and point eight two for the or testing

773
00:34:52,949 --> 00:34:57,899
and here the a mean square error is also

774
00:34:55,530 --> 00:35:01,110
there in some format now let's say let's

775
00:34:57,900 --> 00:35:02,760
plot the fit okay this has more than one

776
00:35:01,110 --> 00:35:07,650
element so it doesn't okay plot the

777
00:35:02,760 --> 00:35:14,310
error histogram okay so can you tell me

778
00:35:07,650 --> 00:35:15,930
how should the error histogram look what

779
00:35:14,310 --> 00:35:18,180
is a what is error essentially should it

780
00:35:15,930 --> 00:35:21,029
be a target minus output is that correct

781
00:35:18,180 --> 00:35:23,640
and that error should be close to zero

782
00:35:21,030 --> 00:35:26,130
through more the number of values closer

783
00:35:23,640 --> 00:35:28,230
to zero this yellow thing is zero more

784
00:35:26,130 --> 00:35:30,240
the number of values closer to zero it

785
00:35:28,230 --> 00:35:33,210
means it's a better fit this should

786
00:35:30,240 --> 00:35:35,520
ideally be tapered down over here okay

787
00:35:33,210 --> 00:35:37,860
so of course there are values here but

788
00:35:35,520 --> 00:35:40,110
there are things with more errors yeah

789
00:35:37,860 --> 00:35:42,090
and if you notice the Train it was a

790
00:35:40,110 --> 00:35:44,190
training thing but with respect to the

791
00:35:42,090 --> 00:35:46,650
test of course we had a few values here

792
00:35:44,190 --> 00:35:49,080
but there are errors that are occurring

793
00:35:46,650 --> 00:35:53,520
with a little bit of higher magnitude of

794
00:35:49,080 --> 00:35:55,290
error as well okay now you can also plot

795
00:35:53,520 --> 00:35:56,700
the regression value which will

796
00:35:55,290 --> 00:35:59,310
essentially end up showing

797
00:35:56,700 --> 00:36:03,240
you what it is you know with respect to

798
00:35:59,310 --> 00:36:05,490
the tar this is the fit this is the data

799
00:36:03,240 --> 00:36:07,140
so if it lies on the other side it's

800
00:36:05,490 --> 00:36:08,490
correct if it lies on this side it's not

801
00:36:07,140 --> 00:36:10,230
correct so you have those sort of

802
00:36:08,490 --> 00:36:12,479
parameters that we need to take a look

803
00:36:10,230 --> 00:36:14,579
at so you have all of these plots let's

804
00:36:12,480 --> 00:36:18,300
go back and change a few things what

805
00:36:14,579 --> 00:36:20,160
would you like to change here number of

806
00:36:18,300 --> 00:36:24,720
neurons so shall we change it to maybe

807
00:36:20,160 --> 00:36:27,540
16 or or maybe let's change it to 32

808
00:36:24,720 --> 00:36:32,279
we'll see how that is and then let's go

809
00:36:27,540 --> 00:36:35,490
to next and let's train okay now how

810
00:36:32,280 --> 00:36:37,079
many iterations did it take right now 12

811
00:36:35,490 --> 00:36:41,220
as opposed to how much did it take

812
00:36:37,079 --> 00:36:43,140
previously being 16 and is the error

813
00:36:41,220 --> 00:36:45,240
here

814
00:36:43,140 --> 00:36:47,848
you know the R value closer to 1 is

815
00:36:45,240 --> 00:36:52,500
better yeah and let's plot the error

816
00:36:47,849 --> 00:36:54,750
histogram is it much better yep closer

817
00:36:52,500 --> 00:36:56,609
to 0 do you see more values yeah and

818
00:36:54,750 --> 00:36:58,619
there are not too many with too many

819
00:36:56,609 --> 00:37:01,319
errors so this should essentially be

820
00:36:58,619 --> 00:37:03,810
where we need to have a lot of values

821
00:37:01,319 --> 00:37:05,940
yeah now let's maybe take a different

822
00:37:03,810 --> 00:37:09,599
regular maybe Bayesian regularization

823
00:37:05,940 --> 00:37:13,109
algorithm and let's retrain and we did

824
00:37:09,599 --> 00:37:15,359
say that it takes a lot of memory and

825
00:37:13,109 --> 00:37:17,940
hence it also is time but they may be

826
00:37:15,359 --> 00:37:20,220
more accuracy so the Bayesian thing

827
00:37:17,940 --> 00:37:22,170
essentially says requires more time but

828
00:37:20,220 --> 00:37:24,689
can result in good generalization for

829
00:37:22,170 --> 00:37:26,550
difficult small or noisy data sets and

830
00:37:24,690 --> 00:37:30,599
the training stops according to the

831
00:37:26,550 --> 00:37:33,240
adaptive weight minimization so it still

832
00:37:30,599 --> 00:37:36,900
is doing a lot of iterations it still is

833
00:37:33,240 --> 00:37:38,640
trying to see what it is what the

834
00:37:36,900 --> 00:37:41,250
parameter is changing and if you notice

835
00:37:38,640 --> 00:37:44,359
the MU value is changing between point

836
00:37:41,250 --> 00:37:47,430
zero zero five to one so it does take

837
00:37:44,359 --> 00:37:49,500
considerable amount of time yeah but it

838
00:37:47,430 --> 00:37:53,190
can result in a good generalization so

839
00:37:49,500 --> 00:37:55,950
you are also able to track how much time

840
00:37:53,190 --> 00:37:57,780
it is taking say if in case you want to

841
00:37:55,950 --> 00:38:01,828
report your findings in a research paper

842
00:37:57,780 --> 00:38:05,609
you can say on MATLAB our 2017 P on an a

843
00:38:01,829 --> 00:38:07,380
GB or a hard drive or a GB ROM this is

844
00:38:05,609 --> 00:38:10,610
how much it takes so how many iterations

845
00:38:07,380 --> 00:38:13,890
did it take here

846
00:38:10,610 --> 00:38:15,930
741 but let's take a look at what the

847
00:38:13,890 --> 00:38:18,779
value what the error value our value

848
00:38:15,930 --> 00:38:20,850
here is 0.89 and we do have the training

849
00:38:18,780 --> 00:38:24,930
MSDS let's take a look at the error

850
00:38:20,850 --> 00:38:27,990
histogram yeah so you do see that there

851
00:38:24,930 --> 00:38:31,080
is um there is more error that is spread

852
00:38:27,990 --> 00:38:35,580
out but it is still contained within

853
00:38:31,080 --> 00:38:38,220
plus 10 and minus 10 yeah so it's it

854
00:38:35,580 --> 00:38:40,620
depends on what your parameters are and

855
00:38:38,220 --> 00:38:43,470
how many do you how many or new neurons

856
00:38:40,620 --> 00:38:45,540
do you have and things like that so it's

857
00:38:43,470 --> 00:38:48,839
up to the data that you have which you

858
00:38:45,540 --> 00:38:51,270
will need to select how much you want to

859
00:38:48,840 --> 00:38:52,770
be doing training on how much do you

860
00:38:51,270 --> 00:38:55,320
want to be doing validation and how much

861
00:38:52,770 --> 00:38:57,390
you want to be doing testing on now I

862
00:38:55,320 --> 00:39:01,170
want to show the effect of maybe

863
00:38:57,390 --> 00:39:03,509
reducing the training data set and let's

864
00:39:01,170 --> 00:39:05,100
increase the validation and testing some

865
00:39:03,510 --> 00:39:09,270
how much how many samples am I using

866
00:39:05,100 --> 00:39:12,810
here for training 45% as opposed to 70%

867
00:39:09,270 --> 00:39:15,390
how much am I using it for validation 25

868
00:39:12,810 --> 00:39:17,009
and around a 30 let's go and check what

869
00:39:15,390 --> 00:39:19,830
happens here and it's going to be around

870
00:39:17,010 --> 00:39:22,590
10 hidden neurons and then I'm gonna use

871
00:39:19,830 --> 00:39:25,529
the same Levenberg market algorithm now

872
00:39:22,590 --> 00:39:27,660
let's check what happens did I have too

873
00:39:25,530 --> 00:39:31,170
many iterations for the thing for the

874
00:39:27,660 --> 00:39:33,540
training no it was 13 why because my

875
00:39:31,170 --> 00:39:35,460
training data is lesser is that right

876
00:39:33,540 --> 00:39:38,400
yeah and now let's see the error

877
00:39:35,460 --> 00:39:40,230
histogram is it more significant is the

878
00:39:38,400 --> 00:39:43,380
error more significant compared to what

879
00:39:40,230 --> 00:39:46,440
it was earlier yeah because the training

880
00:39:43,380 --> 00:39:49,230
data is lesser so unless your training

881
00:39:46,440 --> 00:39:52,140
data ends up having the best possible or

882
00:39:49,230 --> 00:39:54,960
all cases it may not end up giving you

883
00:39:52,140 --> 00:39:58,650
the best performance okay now let's go

884
00:39:54,960 --> 00:40:01,500
back and change it to a point where will

885
00:39:58,650 --> 00:40:04,080
reduce the validation so it's a 10% this

886
00:40:01,500 --> 00:40:07,080
is now increased to 60 percent let's go

887
00:40:04,080 --> 00:40:09,870
and check what happens again yeah I'll

888
00:40:07,080 --> 00:40:16,830
train it and when I plot the errors

889
00:40:09,870 --> 00:40:18,870
histogram it's definitely include with

890
00:40:16,830 --> 00:40:21,090
the minimum error you know lying between

891
00:40:18,870 --> 00:40:23,098
minus 9 to 11 in the previous case it

892
00:40:21,090 --> 00:40:25,440
went on to 40 and a few others

893
00:40:23,099 --> 00:40:27,390
even though it looks spread out we are

894
00:40:25,440 --> 00:40:31,769
doing it in terms of bins if you

895
00:40:27,390 --> 00:40:35,549
realized yeah so this is essentially how

896
00:40:31,769 --> 00:40:39,419
you can perform to overall training of a

897
00:40:35,549 --> 00:40:42,029
network and then also end up validating

898
00:40:39,420 --> 00:40:44,459
it and then finally are trying to see

899
00:40:42,029 --> 00:40:45,959
how the classification works now when

900
00:40:44,459 --> 00:40:49,109
you go next over here

901
00:40:45,959 --> 00:40:51,029
you can evaluate the network okay you

902
00:40:49,109 --> 00:40:53,609
can perform additional tests if you want

903
00:40:51,029 --> 00:40:56,039
on data or you can adjust the network

904
00:40:53,609 --> 00:40:58,019
size or you can import a lot larger data

905
00:40:56,039 --> 00:41:01,670
set or you can say train again and few

906
00:40:58,019 --> 00:41:04,680
other things this is optional after that

907
00:41:01,670 --> 00:41:06,539
you can get an application deploy

908
00:41:04,680 --> 00:41:08,538
solution what is the first thing here

909
00:41:06,539 --> 00:41:11,969
what does it say here

910
00:41:08,539 --> 00:41:15,569
MATLAB function so it will essentially

911
00:41:11,969 --> 00:41:17,969
end up creating what is a built-in

912
00:41:15,569 --> 00:41:20,849
MATLAB function so it essentially says

913
00:41:17,969 --> 00:41:25,170
it's my neural network function where X

914
00:41:20,849 --> 00:41:28,109
is my input yeah and I'm not giving any

915
00:41:25,170 --> 00:41:29,400
outputs and what are the outputs or any

916
00:41:28,109 --> 00:41:32,969
other additional input what are my

917
00:41:29,400 --> 00:41:36,329
outputs I end up having you know my Y

918
00:41:32,969 --> 00:41:39,630
which is the essentially what the output

919
00:41:36,329 --> 00:41:42,719
value over TS time steps is I ended up

920
00:41:39,630 --> 00:41:44,999
getting what my gains and the offsets

921
00:41:42,719 --> 00:41:47,160
are that is a bias and the offsets what

922
00:41:44,999 --> 00:41:50,218
is the number of layers what is a bias

923
00:41:47,160 --> 00:41:52,799
in each of the layers if you notice that

924
00:41:50,219 --> 00:41:56,390
is layer one and layer two and then what

925
00:41:52,799 --> 00:42:00,029
are the I and L values and then what my

926
00:41:56,390 --> 00:42:01,890
output layer is and how do I simulate it

927
00:42:00,029 --> 00:42:04,589
the all of this function was

928
00:42:01,890 --> 00:42:06,779
automatically or general all of this

929
00:42:04,589 --> 00:42:10,319
function was all automatically generated

930
00:42:06,779 --> 00:42:12,209
is that helpful when you understand the

931
00:42:10,319 --> 00:42:14,339
neural network work so you know MATLAB

932
00:42:12,209 --> 00:42:15,808
all you need to do is ensure that you're

933
00:42:14,339 --> 00:42:17,099
able to train the parameters

934
00:42:15,809 --> 00:42:19,769
appropriately you know train the

935
00:42:17,099 --> 00:42:22,380
parameters well so it says it uses a

936
00:42:19,769 --> 00:42:23,910
sigmoid transfer function so there

937
00:42:22,380 --> 00:42:25,890
multiple things over here so these are

938
00:42:23,910 --> 00:42:28,890
the module functions which are there so

939
00:42:25,890 --> 00:42:31,709
this code is automatically generated by

940
00:42:28,890 --> 00:42:36,810
the neural network toolbox now the

941
00:42:31,709 --> 00:42:41,190
second thing is if you want to deploy

942
00:42:36,810 --> 00:42:43,920
you know it into see an executable yeah

943
00:42:41,190 --> 00:42:46,200
if you want to or generate a C code yeah

944
00:42:43,920 --> 00:42:47,670
you can just get a matte matrix only

945
00:42:46,200 --> 00:42:49,980
function because the previous one

946
00:42:47,670 --> 00:42:51,600
actually had a cell array so here what

947
00:42:49,980 --> 00:42:53,670
you can do is you can get an executable

948
00:42:51,600 --> 00:42:56,520
using MATLAB compiler yeah it will give

949
00:42:53,670 --> 00:42:59,190
you an exe but if you want a C code you

950
00:42:56,520 --> 00:43:01,050
can just get a matrix only function the

951
00:42:59,190 --> 00:43:04,020
next thing I wanted to show you was what

952
00:43:01,050 --> 00:43:06,810
does this Simulink equivalent model of

953
00:43:04,020 --> 00:43:08,340
this look like so for the first time it

954
00:43:06,810 --> 00:43:10,110
takes a little while for simile to open

955
00:43:08,340 --> 00:43:12,810
up because it's generating the model and

956
00:43:10,110 --> 00:43:14,760
Simulink is a pretty heavy platform with

957
00:43:12,810 --> 00:43:16,140
a lot of toolboxes and you know the

958
00:43:14,760 --> 00:43:23,520
blocks are corresponding to each of

959
00:43:16,140 --> 00:43:27,509
those toolboxes and what do I see here I

960
00:43:23,520 --> 00:43:30,960
have an input X I'm an output Y and at

961
00:43:27,510 --> 00:43:33,690
the function fitting I get I can double

962
00:43:30,960 --> 00:43:36,150
click on it and say it's a mass it's a

963
00:43:33,690 --> 00:43:42,450
function fitting but I can go in I can

964
00:43:36,150 --> 00:43:45,390
say right click it does my computer's a

965
00:43:42,450 --> 00:43:49,529
little slow apologize can go and say

966
00:43:45,390 --> 00:43:53,730
mask and I can say look under mask it

967
00:43:49,530 --> 00:43:55,860
has two layers ok and one of them is the

968
00:43:53,730 --> 00:43:57,540
input layer which is going in here which

969
00:43:55,860 --> 00:44:00,060
is the first layer and the second layer

970
00:43:57,540 --> 00:44:02,160
which is the which is so we have a two

971
00:44:00,060 --> 00:44:04,230
layer Network and the output layer let's

972
00:44:02,160 --> 00:44:06,870
take a look at what the layer 1 looks

973
00:44:04,230 --> 00:44:10,800
like we'll do mask and look under mask

974
00:44:06,870 --> 00:44:13,710
again so I have the D list I have my

975
00:44:10,800 --> 00:44:17,010
initial weights and the bias and I have

976
00:44:13,710 --> 00:44:19,530
the transfer function let me go look at

977
00:44:17,010 --> 00:44:22,170
what this is a tan cig function let me

978
00:44:19,530 --> 00:44:24,450
go under mask and say look under mask it

979
00:44:22,170 --> 00:44:26,700
essentially shows me what the overall

980
00:44:24,450 --> 00:44:29,189
thing is you know it says minus 2 e

981
00:44:26,700 --> 00:44:31,740
power u plus 1 and then reciprocal of

982
00:44:29,190 --> 00:44:33,600
that plus 2 and all that yeah it's a

983
00:44:31,740 --> 00:44:36,870
pretty straightforward thing let's look

984
00:44:33,600 --> 00:44:40,020
at what's under the weights it we go to

985
00:44:36,870 --> 00:44:43,770
masks and we say look under masks if you

986
00:44:40,020 --> 00:44:46,860
notice it essentially has a weights

987
00:44:43,770 --> 00:44:49,750
function or away it's block for each of

988
00:44:46,860 --> 00:44:51,880
them I can look under each of them

989
00:44:49,750 --> 00:44:54,520
and actually I cannot because it's

990
00:44:51,880 --> 00:44:56,710
something that will show me what the

991
00:44:54,520 --> 00:44:59,290
constant value is so this essentially

992
00:44:56,710 --> 00:45:02,050
takes into account what the weight here

993
00:44:59,290 --> 00:45:03,610
is for each of them you know like the

994
00:45:02,050 --> 00:45:05,980
ten particular weights that are there

995
00:45:03,610 --> 00:45:07,720
and for each of the layers so this is

996
00:45:05,980 --> 00:45:10,210
for one of the layers one of the weeds

997
00:45:07,720 --> 00:45:11,830
okay and the overall function fitting

998
00:45:10,210 --> 00:45:13,870
here the layer two here will end up

999
00:45:11,830 --> 00:45:16,390
having ten more of those so let's take a

1000
00:45:13,870 --> 00:45:19,000
look that bad and I look at the mask I

1001
00:45:16,390 --> 00:45:22,089
have the same and here it's a second

1002
00:45:19,000 --> 00:45:23,740
weight and I look under mask I have a

1003
00:45:22,090 --> 00:45:27,670
similar thing where this a set of

1004
00:45:23,740 --> 00:45:30,370
weights the input here and then what the

1005
00:45:27,670 --> 00:45:33,190
overall output is going to be this

1006
00:45:30,370 --> 00:45:35,670
particularly I like Simulink in this

1007
00:45:33,190 --> 00:45:38,740
case because it gives a very visual

1008
00:45:35,670 --> 00:45:42,130
representation one because a lot of

1009
00:45:38,740 --> 00:45:44,560
times if you notice when I started out

1010
00:45:42,130 --> 00:45:46,390
my presentation itself I started out

1011
00:45:44,560 --> 00:45:48,549
with this sort of a block diagram if you

1012
00:45:46,390 --> 00:45:50,950
realize or you know with this type of a

1013
00:45:48,550 --> 00:45:53,410
block diagram and what you get and

1014
00:45:50,950 --> 00:45:55,960
Simulink is pretty close to what you

1015
00:45:53,410 --> 00:45:58,390
have as a neural network in literature

1016
00:45:55,960 --> 00:46:01,660
so it's very helpful to be able to see

1017
00:45:58,390 --> 00:46:03,339
what pictorially it represents students

1018
00:46:01,660 --> 00:46:05,440
are able to learn better you are able to

1019
00:46:03,340 --> 00:46:08,230
explain it better to in your research

1020
00:46:05,440 --> 00:46:10,690
publications and more importantly you

1021
00:46:08,230 --> 00:46:12,850
can have streaming data coming in you

1022
00:46:10,690 --> 00:46:15,550
not using your data acquisition devices

1023
00:46:12,850 --> 00:46:17,770
and you can set the time to say infinity

1024
00:46:15,550 --> 00:46:21,430
and as the data comes in you'd be able

1025
00:46:17,770 --> 00:46:26,890
to you know provide the output and it

1026
00:46:21,430 --> 00:46:29,379
can function in real time and you can

1027
00:46:26,890 --> 00:46:31,270
also just get a basic graphical diagram

1028
00:46:29,380 --> 00:46:32,890
of the neural network which is a general

1029
00:46:31,270 --> 00:46:34,750
thing you know if you want to just put

1030
00:46:32,890 --> 00:46:36,460
it in your publications you can just end

1031
00:46:34,750 --> 00:46:37,990
up are saying this is what I have my

1032
00:46:36,460 --> 00:46:40,180
function look like okay the next part

1033
00:46:37,990 --> 00:46:43,720
but this is what my overall neural

1034
00:46:40,180 --> 00:46:48,460
network looks like yeah I'm sorry how do

1035
00:46:43,720 --> 00:46:53,919
this display solution I just clicked on

1036
00:46:48,460 --> 00:46:56,860
neural network diagram okay so all of

1037
00:46:53,920 --> 00:46:59,380
this is contained within that new neural

1038
00:46:56,860 --> 00:47:01,720
network n n start okay I'll go over it

1039
00:46:59,380 --> 00:47:03,309
again for another example but at this

1040
00:47:01,720 --> 00:47:08,569
time is the workflow key

1041
00:47:03,309 --> 00:47:10,400
yeah okay now I'm gonna go next and you

1042
00:47:08,569 --> 00:47:12,859
know it C is do you want to save a

1043
00:47:10,400 --> 00:47:14,660
result you know save it to the workspace

1044
00:47:12,859 --> 00:47:16,279
and all that stuff you can do any of

1045
00:47:14,660 --> 00:47:17,629
those at this point of time I'm just

1046
00:47:16,279 --> 00:47:21,980
gonna close it I'm not gonna bother

1047
00:47:17,630 --> 00:47:26,329
seaming it okay now what I would like

1048
00:47:21,980 --> 00:47:28,789
you to do or explore is next or what we

1049
00:47:26,329 --> 00:47:31,430
would like to do is next maybe look at

1050
00:47:28,789 --> 00:47:35,269
what it is for a time series dynamic

1051
00:47:31,430 --> 00:47:37,999
time series yeah so what is a time

1052
00:47:35,269 --> 00:47:40,038
series here can you can you tell me what

1053
00:47:37,999 --> 00:47:43,939
can you tell me where time series

1054
00:47:40,039 --> 00:47:57,170
analysis is important any answers I can

1055
00:47:43,940 --> 00:47:59,210
get to answers anyone what's it why is a

1056
00:47:57,170 --> 00:48:00,819
time series analysis is important

1057
00:47:59,210 --> 00:48:07,160
because time series is a pretty

1058
00:48:00,819 --> 00:48:09,230
important term in these days so any any

1059
00:48:07,160 --> 00:48:13,670
response why would you need to perform

1060
00:48:09,230 --> 00:48:14,660
time series analysis some sort of a

1061
00:48:13,670 --> 00:48:24,890
prediction can you give me an

1062
00:48:14,660 --> 00:48:30,920
application stock exchange great so that

1063
00:48:24,890 --> 00:48:34,129
is one yeah definitely so the thing is

1064
00:48:30,920 --> 00:48:36,950
say for instance if I can if I want to

1065
00:48:34,130 --> 00:48:40,160
see you know over the year I've seen

1066
00:48:36,950 --> 00:48:42,169
what or data I have you know maybe I'll

1067
00:48:40,160 --> 00:48:45,499
say one of the airline's wants to take

1068
00:48:42,170 --> 00:48:48,170
into account what the delay was with

1069
00:48:45,499 --> 00:48:50,180
respect to a particular aircraft you

1070
00:48:48,170 --> 00:48:53,989
know over the over the over the years

1071
00:48:50,180 --> 00:48:55,609
and say in November Delhi has fog or

1072
00:48:53,989 --> 00:48:58,190
Delhi has a you know

1073
00:48:55,609 --> 00:49:00,170
fog in January also that contributes to

1074
00:48:58,190 --> 00:49:01,880
a delay in Bangalore because they're

1075
00:49:00,170 --> 00:49:03,950
coming incoming aircraft is from Delhi

1076
00:49:01,880 --> 00:49:05,720
can be routed elsewhere so they're able

1077
00:49:03,950 --> 00:49:09,230
to predict it by ensuring that they

1078
00:49:05,720 --> 00:49:11,058
learn the model initially yeah so as it

1079
00:49:09,230 --> 00:49:13,549
is mentioned your prediction is a kind

1080
00:49:11,059 --> 00:49:15,700
of dynamic filtering in which past

1081
00:49:13,549 --> 00:49:18,220
values are used to pray

1082
00:49:15,700 --> 00:49:21,040
the future values okay and dynamic

1083
00:49:18,220 --> 00:49:23,189
neural networks include either a tabbed

1084
00:49:21,040 --> 00:49:25,119
delay line for nonlinear filtering

1085
00:49:23,190 --> 00:49:28,810
applications so you can provide a

1086
00:49:25,119 --> 00:49:31,690
feedback or it could just be a thing

1087
00:49:28,810 --> 00:49:33,940
based on what the past values are okay

1088
00:49:31,690 --> 00:49:36,520
so let's take a nonlinear input-output

1089
00:49:33,940 --> 00:49:38,470
you do have a nonlinear orto regressive

1090
00:49:36,520 --> 00:49:40,660
with some extra external input or

1091
00:49:38,470 --> 00:49:42,910
non-linear autoregressive with no

1092
00:49:40,660 --> 00:49:45,549
external input let's just take and walk

1093
00:49:42,910 --> 00:49:48,460
on nonlinear input-output these

1094
00:49:45,550 --> 00:49:51,310
predictive models are used in say you

1095
00:49:48,460 --> 00:49:53,260
know this predictive maintenance of you

1096
00:49:51,310 --> 00:49:56,020
know systems like say a jet propulsion

1097
00:49:53,260 --> 00:49:59,260
engine may not know you may not know

1098
00:49:56,020 --> 00:50:01,210
when it is going to go bad so if you're

1099
00:49:59,260 --> 00:50:02,710
able to collect the data and say at this

1100
00:50:01,210 --> 00:50:05,040
time we should perform a maintenance

1101
00:50:02,710 --> 00:50:07,119
check they can definitely be a lot more

1102
00:50:05,040 --> 00:50:09,279
savings involved rather than going and

1103
00:50:07,119 --> 00:50:11,380
checking every time yeah so in a lot of

1104
00:50:09,280 --> 00:50:13,780
chemical processes robotics or aerospace

1105
00:50:11,380 --> 00:50:17,380
systems manufacturing systems it is used

1106
00:50:13,780 --> 00:50:20,440
now non-linear input-output time series

1107
00:50:17,380 --> 00:50:23,290
analysis okay I will load an example

1108
00:50:20,440 --> 00:50:25,119
dataset again so what I'm gonna do is

1109
00:50:23,290 --> 00:50:30,220
I'm gonna take into account

1110
00:50:25,119 --> 00:50:32,109
maybe pollution mortality okay so what I

1111
00:50:30,220 --> 00:50:34,299
have here is I have the pollution input

1112
00:50:32,109 --> 00:50:38,859
so which essentially says what is the

1113
00:50:34,300 --> 00:50:45,040
temperature relative humidity a few of

1114
00:50:38,859 --> 00:50:48,220
the harmful harmful gases ozone and the

1115
00:50:45,040 --> 00:50:49,779
particulate matter okay and then I have

1116
00:50:48,220 --> 00:50:51,189
the pollution target which will

1117
00:50:49,780 --> 00:50:54,069
essentially say what is the total

1118
00:50:51,190 --> 00:50:55,150
mortality respiratory mortality that is

1119
00:50:54,069 --> 00:50:56,619
how many people may end up getting

1120
00:50:55,150 --> 00:50:58,089
respiratory disorders how many people

1121
00:50:56,619 --> 00:51:00,490
end up getting cardiovascular disorders

1122
00:50:58,089 --> 00:51:05,410
is this a good data set is that okay

1123
00:51:00,490 --> 00:51:07,209
with you guys okay and I have 508 cell

1124
00:51:05,410 --> 00:51:09,700
arrays with eight by one vectors for

1125
00:51:07,210 --> 00:51:12,730
each of them so my data has 500 data

1126
00:51:09,700 --> 00:51:15,250
points or 500 time steps so I'll import

1127
00:51:12,730 --> 00:51:18,790
it so I have the pollution input and I

1128
00:51:15,250 --> 00:51:21,190
have the pollution target okay now as I

1129
00:51:18,790 --> 00:51:22,869
said it's a 508 hour time steps of eight

1130
00:51:21,190 --> 00:51:25,690
elements it's gathered over five hundred

1131
00:51:22,869 --> 00:51:27,940
and eight time steps now let's go next

1132
00:51:25,690 --> 00:51:29,420
okay I use the same thing I'll use

1133
00:51:27,940 --> 00:51:32,780
around what do you call

1134
00:51:29,420 --> 00:51:34,760
you and you in interestingly in this

1135
00:51:32,780 --> 00:51:37,250
case what has happened is you end up

1136
00:51:34,760 --> 00:51:39,440
having what is five hundred and eight

1137
00:51:37,250 --> 00:51:41,030
times three it's gonna be fifteen twenty

1138
00:51:39,440 --> 00:51:43,250
four yeah you need to end up getting

1139
00:51:41,030 --> 00:51:46,250
fifteen twenty four tiger time steps so

1140
00:51:43,250 --> 00:51:48,290
essentially end up taking around seventy

1141
00:51:46,250 --> 00:51:50,839
person for training and remaining for

1142
00:51:48,290 --> 00:51:54,680
validation and testing as always let's

1143
00:51:50,839 --> 00:51:57,259
go next and here what you can also in

1144
00:51:54,680 --> 00:51:59,419
terms of a time delay why do you need

1145
00:51:57,260 --> 00:52:02,030
this time delay is you need to ensure

1146
00:51:59,420 --> 00:52:03,650
that how many time steps can the network

1147
00:52:02,030 --> 00:52:06,559
respond to how much do you want to go

1148
00:52:03,650 --> 00:52:08,599
back in time yeah and the number of

1149
00:52:06,559 --> 00:52:10,910
hidden neurons is ten we will change

1150
00:52:08,599 --> 00:52:13,040
this parameter you know how is the

1151
00:52:10,910 --> 00:52:14,868
output how many times is the output a

1152
00:52:13,040 --> 00:52:16,490
function of the input and things like

1153
00:52:14,869 --> 00:52:19,640
that you'd be able to say that that's a

1154
00:52:16,490 --> 00:52:21,680
delay if you want it by off the only to

1155
00:52:19,640 --> 00:52:23,509
be t minus 1 and t minus 2 this will be

1156
00:52:21,680 --> 00:52:26,299
2 if you want it to take and talk on

1157
00:52:23,510 --> 00:52:28,339
many more like it's almost like an F IR

1158
00:52:26,299 --> 00:52:31,400
filter or an IR filter with how many

1159
00:52:28,339 --> 00:52:34,759
previous samples do you want ok any

1160
00:52:31,400 --> 00:52:36,950
questions so far so P increase the delay

1161
00:52:34,760 --> 00:52:38,809
is what will happen the increase at

1162
00:52:36,950 --> 00:52:40,549
belay you will end up having more more

1163
00:52:38,809 --> 00:52:43,670
delays and you will end up getting more

1164
00:52:40,549 --> 00:52:45,980
data to make a decision yeah because if

1165
00:52:43,670 --> 00:52:51,349
you notice your your wife will be a

1166
00:52:45,980 --> 00:52:53,030
function of more data points now let me

1167
00:52:51,349 --> 00:52:58,490
change it back to two at this point of

1168
00:52:53,030 --> 00:53:01,849
time and then say I'll say next and I'll

1169
00:52:58,490 --> 00:53:05,270
use the same model and let's perform the

1170
00:53:01,849 --> 00:53:07,160
training it again took around 16

1171
00:53:05,270 --> 00:53:10,220
iterations there was some gradient there

1172
00:53:07,160 --> 00:53:14,149
was some validation check yeah and let's

1173
00:53:10,220 --> 00:53:16,069
plot the error histogram and of course

1174
00:53:14,150 --> 00:53:18,470
you know you see that there is a lot of

1175
00:53:16,069 --> 00:53:20,089
concentration around zero which means

1176
00:53:18,470 --> 00:53:23,000
the data is correct and we are getting

1177
00:53:20,089 --> 00:53:25,578
lesser error yeah and yellow bar there

1178
00:53:23,000 --> 00:53:30,140
indicates a zero error let's plot the

1179
00:53:25,579 --> 00:53:33,140
response yeah so if you notice here it's

1180
00:53:30,140 --> 00:53:38,180
a time series thing so my error is a

1181
00:53:33,140 --> 00:53:41,270
yellow one okay and I have the training

1182
00:53:38,180 --> 00:53:43,430
target represented in blue and the

1183
00:53:41,270 --> 00:53:46,880
training output represented in a blue

1184
00:53:43,430 --> 00:53:48,740
plus the validation and this one so the

1185
00:53:46,880 --> 00:53:52,280
overall thing here the black one is a

1186
00:53:48,740 --> 00:54:01,970
response and when you look closely let

1187
00:53:52,280 --> 00:54:05,990
me try to zoom in okay so if you notice

1188
00:54:01,970 --> 00:54:09,259
here I do have the response which is in

1189
00:54:05,990 --> 00:54:11,750
this way and it essentially connects

1190
00:54:09,260 --> 00:54:13,670
what my training output is and in a few

1191
00:54:11,750 --> 00:54:15,500
cases you see that the error is lesser

1192
00:54:13,670 --> 00:54:18,380
in a few cases you see that the error is

1193
00:54:15,500 --> 00:54:20,329
more and that's because we need to be

1194
00:54:18,380 --> 00:54:22,280
checked what the validation target or

1195
00:54:20,329 --> 00:54:24,680
the test target were so it was mixed up

1196
00:54:22,280 --> 00:54:26,900
in in a lot of these cases but overall

1197
00:54:24,680 --> 00:54:29,660
if you notice the target minus output

1198
00:54:26,900 --> 00:54:32,390
most of the error is less than twenty

1199
00:54:29,660 --> 00:54:35,480
here and which means the classification

1200
00:54:32,390 --> 00:54:38,839
is pretty decent now we can also say

1201
00:54:35,480 --> 00:54:41,270
plot the error autocorrelation so closer

1202
00:54:38,839 --> 00:54:44,089
the error autocorrelation is to this it

1203
00:54:41,270 --> 00:54:45,890
means that it's a better a of course

1204
00:54:44,089 --> 00:54:47,480
this has to be the maximum value in this

1205
00:54:45,890 --> 00:54:50,930
case if you notice a confidence limit of

1206
00:54:47,480 --> 00:54:54,290
it's around say 8% it's around 8% in

1207
00:54:50,930 --> 00:54:56,569
this case or around 8 and this is with

1208
00:54:54,290 --> 00:54:59,779
respect to lag depending on how much

1209
00:54:56,569 --> 00:55:03,079
delay you end up have it okay now let's

1210
00:54:59,780 --> 00:55:05,750
go back and change the number of delays

1211
00:55:03,079 --> 00:55:08,569
so let's change it to maybe four okay

1212
00:55:05,750 --> 00:55:12,560
and let's say next and I'll use the same

1213
00:55:08,569 --> 00:55:14,390
thing and if you notice over here around

1214
00:55:12,560 --> 00:55:16,549
eleven iterations were sufficient there

1215
00:55:14,390 --> 00:55:18,828
were six validation checks let's plot

1216
00:55:16,550 --> 00:55:21,619
the error histogram and if you notice

1217
00:55:18,829 --> 00:55:24,619
here it's a lot better now you know I

1218
00:55:21,619 --> 00:55:26,780
have around 500 data points in the you

1219
00:55:24,619 --> 00:55:30,079
know close to zero bin and around 400

1220
00:55:26,780 --> 00:55:31,819
data points you know with flow again you

1221
00:55:30,079 --> 00:55:35,020
know less than -3 or less than minus 4

1222
00:55:31,819 --> 00:55:38,119
yeah and here it's almost zero here and

1223
00:55:35,020 --> 00:55:40,520
let's say the error autocorrelation it's

1224
00:55:38,119 --> 00:55:42,440
a lot better you know as the value

1225
00:55:40,520 --> 00:55:47,450
changes it's a lot better and I don't

1226
00:55:42,440 --> 00:55:49,790
have too much of an okay so same thing I

1227
00:55:47,450 --> 00:55:51,919
can go next I can adjust the network

1228
00:55:49,790 --> 00:55:54,109
size I can import a larger data set if

1229
00:55:51,920 --> 00:55:56,089
m1 I don't want to do any of those I can

1230
00:55:54,109 --> 00:55:56,558
say I want a similar diagram or I want

1231
00:55:56,089 --> 00:55:58,390
an you

1232
00:55:56,559 --> 00:56:00,160
diagram it essentially should have a

1233
00:55:58,390 --> 00:56:02,199
delay element the weight and the bias is

1234
00:56:00,160 --> 00:56:04,449
the hidden and this is our output layer

1235
00:56:02,199 --> 00:56:10,779
okay or I can generate a MATLAB function

1236
00:56:04,449 --> 00:56:13,539
if I want okay so this is one of the

1237
00:56:10,779 --> 00:56:21,390
things that we wanted to show you man

1238
00:56:13,539 --> 00:56:24,900
wanted you know please yes

1239
00:56:21,390 --> 00:56:27,189
you just told about the time series yeah

1240
00:56:24,900 --> 00:56:28,959
functioning how can we make the

1241
00:56:27,189 --> 00:56:33,269
prediction through the tool box could

1242
00:56:28,959 --> 00:56:36,848
you just explain that prediction

1243
00:56:33,269 --> 00:56:38,618
prediction in the time series app how

1244
00:56:36,849 --> 00:56:40,930
can we make that actually the time

1245
00:56:38,619 --> 00:56:42,819
series toolbox has made it its own

1246
00:56:40,930 --> 00:56:46,089
output on the basis of the input and the

1247
00:56:42,819 --> 00:56:49,449
target yes if I want to get this load

1248
00:56:46,089 --> 00:56:52,479
the data then how will I mean for that

1249
00:56:49,449 --> 00:56:55,719
okay let me take a look at that I'll

1250
00:56:52,479 --> 00:56:57,879
take the same example okay and I'll go

1251
00:56:55,719 --> 00:57:02,769
next I'll do the same thing and I will

1252
00:56:57,880 --> 00:57:06,279
end up doing that I'll train and I'll go

1253
00:57:02,769 --> 00:57:08,169
next now you can say you know you want

1254
00:57:06,279 --> 00:57:10,269
to perform additional tests or you want

1255
00:57:08,170 --> 00:57:12,789
to take a look at additional data you

1256
00:57:10,269 --> 00:57:14,948
can lower that in here as an input and

1257
00:57:12,789 --> 00:57:17,109
then say you want the output to be

1258
00:57:14,949 --> 00:57:18,670
stored here and you can either perform

1259
00:57:17,109 --> 00:57:20,078
the testing of the network or you can

1260
00:57:18,670 --> 00:57:23,859
perform the classification right here

1261
00:57:20,079 --> 00:57:25,989
itself okay or the other thing is if you

1262
00:57:23,859 --> 00:57:28,269
want to predict the better way would be

1263
00:57:25,989 --> 00:57:30,789
to create a MATLAB function this way and

1264
00:57:28,269 --> 00:57:32,169
give the input here for the prediction

1265
00:57:30,789 --> 00:57:41,609
and you'll end up getting the output

1266
00:57:32,170 --> 00:57:46,569
here doesn't make sense so essentially

1267
00:57:41,609 --> 00:57:49,089
use the end tool to generate your

1268
00:57:46,569 --> 00:57:51,069
network and then either take a semi link

1269
00:57:49,089 --> 00:57:53,549
model or a MATLAB model and provide the

1270
00:57:51,069 --> 00:57:56,549
input here and get the predicted value

1271
00:57:53,549 --> 00:57:56,549
okay

1272
00:57:57,119 --> 00:58:01,689
we're in the Similan diagram so I can go

1273
00:57:59,920 --> 00:58:03,699
to similan here and it will end up

1274
00:58:01,689 --> 00:58:07,379
creating so here instead of the constant

1275
00:58:03,699 --> 00:58:07,380
you can end up giving your input

1276
00:58:09,390 --> 00:58:15,600
yeah so definitely you can go and say

1277
00:58:12,120 --> 00:58:17,370
edit match in this case it was different

1278
00:58:15,600 --> 00:58:18,779
inputs and all that over here but you

1279
00:58:17,370 --> 00:58:20,759
can end up giving your input over here

1280
00:58:18,780 --> 00:58:23,310
okay all better

1281
00:58:20,760 --> 00:58:26,600
in in do it using the MATLAB function

1282
00:58:23,310 --> 00:58:30,779
where you can give your input as X okay

1283
00:58:26,600 --> 00:58:33,630
okay but what if like I have got 120

1284
00:58:30,780 --> 00:58:36,660
points and I want to get the 121st point

1285
00:58:33,630 --> 00:58:39,510
then how will I get on to it I have the

1286
00:58:36,660 --> 00:58:42,990
other values I want to get further for

1287
00:58:39,510 --> 00:58:45,240
example for the next five days on the

1288
00:58:42,990 --> 00:58:49,640
basis of the past hundred days so how

1289
00:58:45,240 --> 00:58:49,640
will I be able to enter that five days

1290
00:58:51,260 --> 00:58:59,640
able to predict the future data yeah

1291
00:58:55,760 --> 00:59:09,570
sure so in that case what you can end up

1292
00:58:59,640 --> 00:59:12,810
doing is you end up pain you do have the

1293
00:59:09,570 --> 00:59:15,270
X put the luck put the last she was 0 at

1294
00:59:12,810 --> 00:59:17,100
this point of time okay and then see

1295
00:59:15,270 --> 00:59:20,550
what you get it should work that way

1296
00:59:17,100 --> 00:59:22,860
otherwise you can also use the fitting

1297
00:59:20,550 --> 00:59:25,080
app where you might have you might have

1298
00:59:22,860 --> 00:59:28,440
generated a neural network for this and

1299
00:59:25,080 --> 00:59:33,480
then you can put that in as an input and

1300
00:59:28,440 --> 00:59:37,130
or take the prediction I will send you

1301
00:59:33,480 --> 00:59:37,130
the workflow on this when I send you the

1302
00:59:39,050 --> 00:59:43,140
similarly you have the same

1303
00:59:40,950 --> 00:59:45,569
functionality for the pattern

1304
00:59:43,140 --> 00:59:48,629
recognition app and for a clustering app

1305
00:59:45,570 --> 00:59:51,930
so I'm happy to either go over this or

1306
00:59:48,630 --> 00:59:53,730
I'm happy to move on to deep learning

1307
00:59:51,930 --> 00:59:55,680
for computer vision problems and take a

1308
00:59:53,730 --> 01:00:07,230
couple of applications there whatever

1309
00:59:55,680 --> 01:00:09,899
works for you it's a similar wrap map so

1310
01:00:07,230 --> 01:00:13,440
and we can send if we can send them how

1311
01:00:09,900 --> 01:00:14,730
to get how to you know maybe a couple of

1312
01:00:13,440 --> 01:00:20,310
examples which are there we can send

1313
01:00:14,730 --> 01:00:22,320
them that example looks okay now also

1314
01:00:20,310 --> 01:00:23,130
mom will we be able to get the email

1315
01:00:22,320 --> 01:00:25,560
addresses of the

1316
01:00:23,130 --> 01:00:26,940
depends to be able to send the thing

1317
01:00:25,560 --> 01:00:29,400
I'll talk to you about that but we would

1318
01:00:26,940 --> 01:00:32,400
like to send them I have the email any

1319
01:00:29,400 --> 01:00:35,580
of all the coordinators of the centers

1320
01:00:32,400 --> 01:00:38,370
who are so I can give that to you

1321
01:00:35,580 --> 01:00:40,200
awesome them and then they will send it

1322
01:00:38,370 --> 01:00:44,700
they will send it on their own

1323
01:00:40,200 --> 01:00:46,140
sure definitely thank you ma'am no the

1324
01:00:44,700 --> 01:00:49,589
thing is the next thing I would like to

1325
01:00:46,140 --> 01:00:51,810
talk about is deep learning for computer

1326
01:00:49,590 --> 01:00:54,510
vision applications using MATLAB for the

1327
01:00:51,810 --> 01:00:56,670
next 40 minutes I'll go over basics of

1328
01:00:54,510 --> 01:01:00,390
deep learning and why did it gain

1329
01:00:56,670 --> 01:01:02,040
importance and maybe two examples and I

1330
01:01:00,390 --> 01:01:04,589
will go over the third example tomorrow

1331
01:01:02,040 --> 01:01:10,590
when we are talking about the next part

1332
01:01:04,590 --> 01:01:11,760
okay so agenda for this talk is going to

1333
01:01:10,590 --> 01:01:13,350
be what is deep learning why deep

1334
01:01:11,760 --> 01:01:15,360
learning and we'll go over a few things

1335
01:01:13,350 --> 01:01:17,580
based on how much short time we have

1336
01:01:15,360 --> 01:01:19,410
okay can someone tell me what deep

1337
01:01:17,580 --> 01:01:38,819
learning is someone from the audience

1338
01:01:19,410 --> 01:01:40,470
someone from one of the centers good

1339
01:01:38,820 --> 01:01:42,420
so essentially learning is about

1340
01:01:40,470 --> 01:01:46,109
automation and you know we want to

1341
01:01:42,420 --> 01:01:48,600
ensure that we don't instruct the

1342
01:01:46,110 --> 01:01:50,370
computers on what to do but instead they

1343
01:01:48,600 --> 01:01:52,700
should learn on on their own that is

1344
01:01:50,370 --> 01:01:55,529
enabling them to get some artificial

1345
01:01:52,700 --> 01:01:58,109
intelligence so deep learning is based

1346
01:01:55,530 --> 01:01:59,520
on the neural network architecture that

1347
01:01:58,110 --> 01:02:02,340
is a computer system architecture of

1348
01:01:59,520 --> 01:02:05,280
human brain and nervous system the term

1349
01:02:02,340 --> 01:02:08,490
deep refers to the number of layers that

1350
01:02:05,280 --> 01:02:10,500
is neurons and rules in the network more

1351
01:02:08,490 --> 01:02:12,060
the number of layers deeper the network

1352
01:02:10,500 --> 01:02:14,220
at this point of time if you notice

1353
01:02:12,060 --> 01:02:16,080
there was one hidden layer and one

1354
01:02:14,220 --> 01:02:19,529
output layer but if there are more

1355
01:02:16,080 --> 01:02:22,710
layers its the deeper the network okay

1356
01:02:19,530 --> 01:02:25,950
and deep learning is a type of learning

1357
01:02:22,710 --> 01:02:28,340
in which a model or a computer or any of

1358
01:02:25,950 --> 01:02:30,960
those networks learns to perform

1359
01:02:28,340 --> 01:02:34,260
classification tasks directly from

1360
01:02:30,960 --> 01:02:35,570
images text or sound yeah so a lot of

1361
01:02:34,260 --> 01:02:38,930
you might have noticed

1362
01:02:35,570 --> 01:02:41,660
yeah when you're able to put your photos

1363
01:02:38,930 --> 01:02:45,169
on Google photos or Facebook or I photos

1364
01:02:41,660 --> 01:02:47,359
it can say it will detect the face not

1365
01:02:45,170 --> 01:02:50,360
only that but will it will say is this

1366
01:02:47,360 --> 01:02:52,400
this person is it this person yeah how

1367
01:02:50,360 --> 01:02:55,250
did it do that it's it has learned and

1368
01:02:52,400 --> 01:02:57,530
now it is trying to validate yeah on

1369
01:02:55,250 --> 01:02:59,360
what it is and it will say yes in a lot

1370
01:02:57,530 --> 01:03:00,830
of cases it does it automatically and

1371
01:02:59,360 --> 01:03:03,410
then you know you can give it feedback

1372
01:03:00,830 --> 01:03:05,960
that it is correct or not what are a few

1373
01:03:03,410 --> 01:03:07,640
of the applications so there's going to

1374
01:03:05,960 --> 01:03:10,250
be a video playing here if it doesn't

1375
01:03:07,640 --> 01:03:13,339
help it if it doesn't play really well

1376
01:03:10,250 --> 01:03:15,440
do not bother I can I will be showing it

1377
01:03:13,340 --> 01:03:19,580
to you or I can send a link to you so on

1378
01:03:15,440 --> 01:03:23,300
the left hand side I have a green part

1379
01:03:19,580 --> 01:03:25,430
and I also have a blue component and the

1380
01:03:23,300 --> 01:03:27,530
green component there keeps changing and

1381
01:03:25,430 --> 01:03:30,620
you know it's sort of labeling something

1382
01:03:27,530 --> 01:03:32,810
as blue component and then it's showing

1383
01:03:30,620 --> 01:03:35,509
some it's trying to see something else

1384
01:03:32,810 --> 01:03:37,340
in terms of what is happening or in the

1385
01:03:35,510 --> 01:03:40,750
picture can someone tell me what's

1386
01:03:37,340 --> 01:03:44,690
happening here did the video play fine

1387
01:03:40,750 --> 01:03:46,670
no yeah okay so essentially what's

1388
01:03:44,690 --> 01:03:49,730
happened is you try to detect the blue

1389
01:03:46,670 --> 01:03:52,670
objects as objects that are moving okay

1390
01:03:49,730 --> 01:03:54,620
green as the navigatable path

1391
01:03:52,670 --> 01:03:57,020
it's an autonomous driving system so

1392
01:03:54,620 --> 01:03:59,390
it's trying to perform segmentation but

1393
01:03:57,020 --> 01:04:01,250
it is using a bit of semantics or a

1394
01:03:59,390 --> 01:04:03,650
little bit of intelligence in trying to

1395
01:04:01,250 --> 01:04:07,130
make a decision because it knows that

1396
01:04:03,650 --> 01:04:09,800
the blue vehicle is moving yeah or it

1397
01:04:07,130 --> 01:04:13,220
will move sometime it also notices that

1398
01:04:09,800 --> 01:04:15,830
the road is which it has you know for

1399
01:04:13,220 --> 01:04:18,500
movement is quite big and in a few cases

1400
01:04:15,830 --> 01:04:21,020
it's also misclassifying it I'd like to

1401
01:04:18,500 --> 01:04:23,750
do this second example where it's trying

1402
01:04:21,020 --> 01:04:25,850
to detect you know a vehicle with a

1403
01:04:23,750 --> 01:04:28,280
confidence of one saying it is a vehicle

1404
01:04:25,850 --> 01:04:30,170
in a few cases here if you notice it

1405
01:04:28,280 --> 01:04:31,970
detected something else is a vehicle but

1406
01:04:30,170 --> 01:04:34,010
it's not but in most cases it's able to

1407
01:04:31,970 --> 01:04:35,720
detect the vehicle when it is trying to

1408
01:04:34,010 --> 01:04:37,190
change lanes such that it does not

1409
01:04:35,720 --> 01:04:40,669
collide okay

1410
01:04:37,190 --> 01:04:43,070
so how did this happen this is

1411
01:04:40,670 --> 01:04:45,920
essentially happening where the

1412
01:04:43,070 --> 01:04:46,940
application or the network learns on its

1413
01:04:45,920 --> 01:04:50,119
own

1414
01:04:46,940 --> 01:04:52,670
learn to classify that this is a vehicle

1415
01:04:50,119 --> 01:04:59,270
this is not a vehicle this is not a

1416
01:04:52,670 --> 01:05:00,859
navigatable path okay now in a lot of

1417
01:04:59,270 --> 01:05:02,839
applications say in terms of

1418
01:05:00,859 --> 01:05:04,490
classification of a highway scene or you

1419
01:05:02,839 --> 01:05:06,410
know in a few cases you might have a

1420
01:05:04,490 --> 01:05:08,538
video which has some fog or rain you

1421
01:05:06,410 --> 01:05:10,009
might need to remove it or you know a

1422
01:05:08,539 --> 01:05:12,109
lot of times you have the human

1423
01:05:10,010 --> 01:05:14,599
awareness on for robots where we need to

1424
01:05:12,109 --> 01:05:16,279
ensure that the robots essentially are

1425
01:05:14,599 --> 01:05:18,049
aware where the humans are and don't

1426
01:05:16,280 --> 01:05:21,079
collide with each other so you need to

1427
01:05:18,049 --> 01:05:24,160
say what bear the humans are and things

1428
01:05:21,079 --> 01:05:26,630
like that okay so I want to talk about

1429
01:05:24,160 --> 01:05:28,819
traditional machine learning and then

1430
01:05:26,630 --> 01:05:33,010
what is deep learning can someone tell

1431
01:05:28,819 --> 01:05:33,009
me what do you mean by machine learning

1432
01:05:34,510 --> 01:05:44,180
anyone to train the machine okay you

1433
01:05:42,619 --> 01:05:53,000
perform it specific task

1434
01:05:44,180 --> 01:05:55,549
okay anything else okay to make the

1435
01:05:53,000 --> 01:05:57,859
decisions okay let me ask you this I'm

1436
01:05:55,549 --> 01:06:01,250
assuming a lot of you either as kids or

1437
01:05:57,859 --> 01:06:03,470
or now who may have kids do end up our

1438
01:06:01,250 --> 01:06:06,349
teaching your kid different shapes and

1439
01:06:03,470 --> 01:06:09,140
all that yeah so let's say we wanted kid

1440
01:06:06,349 --> 01:06:12,109
to be able to identify triangle square

1441
01:06:09,140 --> 01:06:15,020
or you know a square circle and all that

1442
01:06:12,109 --> 01:06:17,480
stuff yeah so for him to identify him or

1443
01:06:15,020 --> 01:06:20,390
her to identify the shape what would you

1444
01:06:17,480 --> 01:06:23,420
first or tell them would you ask them to

1445
01:06:20,390 --> 01:06:25,490
look at the number of sides yeah and if

1446
01:06:23,420 --> 01:06:27,079
it has three sides it's a triangle if it

1447
01:06:25,490 --> 01:06:28,879
has four sides is a rectangle or a

1448
01:06:27,079 --> 01:06:31,490
square or a rhombus or a parallelogram

1449
01:06:28,880 --> 01:06:33,710
so a circle means infinite number of

1450
01:06:31,490 --> 01:06:35,930
sides or it's round Pentagon but now

1451
01:06:33,710 --> 01:06:37,940
when it is a square when it is four

1452
01:06:35,930 --> 01:06:40,490
sides how does it know to classify a

1453
01:06:37,940 --> 01:06:42,440
square or rectangle you will say take an

1454
01:06:40,490 --> 01:06:45,229
additional feature which is length of

1455
01:06:42,440 --> 01:06:47,270
the sides or the angle yeah so see if

1456
01:06:45,230 --> 01:06:50,750
it's a rhombus and things like that so

1457
01:06:47,270 --> 01:06:53,960
those are known as features and these

1458
01:06:50,750 --> 01:06:57,950
are handcrafted we specify we tell the

1459
01:06:53,960 --> 01:07:00,150
kid to add you know that look at the

1460
01:06:57,950 --> 01:07:02,129
number of sides then look at the

1461
01:07:00,150 --> 01:07:04,200
of the side look at the angle between

1462
01:07:02,130 --> 01:07:07,200
the sides we tell that the feature

1463
01:07:04,200 --> 01:07:10,169
extraction mechanism is something that's

1464
01:07:07,200 --> 01:07:12,180
that that the kid does the way we tell

1465
01:07:10,170 --> 01:07:14,819
it to yeah these are handcrafted

1466
01:07:12,180 --> 01:07:16,589
features after that it performs that la

1467
01:07:14,819 --> 01:07:18,869
ossification where there is machine

1468
01:07:16,589 --> 01:07:21,029
learning okay so that is traditional

1469
01:07:18,869 --> 01:07:24,119
machine learning where the features are

1470
01:07:21,029 --> 01:07:26,190
handcrafted and it sort we tell the

1471
01:07:24,119 --> 01:07:30,000
person on what the features that need to

1472
01:07:26,190 --> 01:07:32,490
be extracted are in deep learning it

1473
01:07:30,000 --> 01:07:34,859
performs end to end learning by learning

1474
01:07:32,490 --> 01:07:37,890
the features representation and the

1475
01:07:34,859 --> 01:07:39,779
tasks directly there is no handcrafted

1476
01:07:37,890 --> 01:07:42,720
Ness over there it will essentially

1477
01:07:39,779 --> 01:07:45,480
dwell into what are them what are

1478
01:07:42,720 --> 01:07:47,399
different layers which you know are

1479
01:07:45,480 --> 01:07:49,500
taken different features or which

1480
01:07:47,400 --> 01:07:52,380
represent different features and learns

1481
01:07:49,500 --> 01:07:55,890
automatically okay and then it is able

1482
01:07:52,380 --> 01:07:58,380
to identify what it is okay so we do not

1483
01:07:55,890 --> 01:08:01,950
give it what features it should be

1484
01:07:58,380 --> 01:08:04,410
looking at doesn't make sense yeah so

1485
01:08:01,950 --> 01:08:06,359
essentially the kid if it's not told

1486
01:08:04,410 --> 01:08:07,859
what the shapes are and all that stuff

1487
01:08:06,359 --> 01:08:10,348
and it looks at it on its own and

1488
01:08:07,859 --> 01:08:12,119
assumes that hey it has four sides now I

1489
01:08:10,349 --> 01:08:13,910
know this so if it's able to learn on

1490
01:08:12,119 --> 01:08:16,739
its own it's one that's deep learning

1491
01:08:13,910 --> 01:08:19,979
everyone feeling with this workflow or

1492
01:08:16,738 --> 01:08:22,108
this understanding okay great so the

1493
01:08:19,979 --> 01:08:23,729
end-to-end learning and the feature and

1494
01:08:22,109 --> 01:08:26,100
learning and classification are all

1495
01:08:23,729 --> 01:08:28,468
performed within the network itself we

1496
01:08:26,100 --> 01:08:31,980
do not help it with the feature

1497
01:08:28,469 --> 01:08:33,600
extraction now why deep learning why did

1498
01:08:31,979 --> 01:08:37,349
it gained so much prominence and the

1499
01:08:33,600 --> 01:08:39,239
recent past so in the beginning of this

1500
01:08:37,350 --> 01:08:42,900
decade what happened was machine

1501
01:08:39,238 --> 01:08:46,080
learning had almost an error of say 30

1502
01:08:42,899 --> 01:08:48,540
percent or 25 percent yeah but then in

1503
01:08:46,080 --> 01:08:50,729
the last few years deep learning

1504
01:08:48,540 --> 01:08:52,859
algorithms which got or reintroduced

1505
01:08:50,729 --> 01:08:54,389
thanks to neural networks or new neural

1506
01:08:52,859 --> 01:08:56,460
networks which got reintroduce thanks to

1507
01:08:54,390 --> 01:08:58,770
deep learning has surpassed the human

1508
01:08:56,460 --> 01:09:00,140
accuracy human accuracies around 7.5

1509
01:08:58,770 --> 01:09:03,210
percent in terms of classification

1510
01:09:00,140 --> 01:09:04,589
they've surpassed the human accuracy but

1511
01:09:03,210 --> 01:09:07,589
how did that happen

1512
01:09:04,589 --> 01:09:10,408
yeah and this is essentially four there

1513
01:09:07,589 --> 01:09:12,899
were a few imagine there's a database of

1514
01:09:10,408 --> 01:09:13,679
images called imagenet and we were

1515
01:09:12,899 --> 01:09:16,618
looking at what the

1516
01:09:13,679 --> 01:09:19,649
top five errors over there are so what

1517
01:09:16,618 --> 01:09:22,288
are the deep learning enablers so a lot

1518
01:09:19,649 --> 01:09:24,238
of computers have come with GPU nvidia

1519
01:09:22,288 --> 01:09:26,880
graphic processing units or different

1520
01:09:24,238 --> 01:09:29,818
graphic processing units so essentially

1521
01:09:26,880 --> 01:09:32,069
there's been like almost 60 times faster

1522
01:09:29,819 --> 01:09:35,639
training in almost less than three years

1523
01:09:32,069 --> 01:09:38,249
okay and you have labeled public

1524
01:09:35,639 --> 01:09:40,439
datasets say like you you all know off

1525
01:09:38,249 --> 01:09:42,688
capture that is like say if what you

1526
01:09:40,439 --> 01:09:44,579
need to enter to be able to validate a

1527
01:09:42,689 --> 01:09:47,429
transfer you know to be able to validate

1528
01:09:44,578 --> 01:09:49,259
a transaction the capture is a labeled

1529
01:09:47,429 --> 01:09:50,819
public data set because you are entering

1530
01:09:49,259 --> 01:09:52,859
and it will try to see what the value

1531
01:09:50,819 --> 01:09:56,130
over there is or what the text over

1532
01:09:52,859 --> 01:09:58,260
there is and then there are different

1533
01:09:56,130 --> 01:10:01,530
types of world-class models which

1534
01:09:58,260 --> 01:10:04,320
already have pre trained say Google net

1535
01:10:01,530 --> 01:10:08,459
or Alex net already has a set of images

1536
01:10:04,320 --> 01:10:10,049
that are pre-trained okay and you have

1537
01:10:08,459 --> 01:10:12,539
around say 20,000 images which are

1538
01:10:10,050 --> 01:10:14,459
already labeled oh you know around 200

1539
01:10:12,539 --> 01:10:15,989
categories and things like that or or

1540
01:10:14,459 --> 01:10:19,349
there are models which are there and

1541
01:10:15,989 --> 01:10:23,129
these have served as a starting point

1542
01:10:19,349 --> 01:10:25,199
for deep learning okay and typically

1543
01:10:23,130 --> 01:10:26,670
what we like to see is where does deep

1544
01:10:25,199 --> 01:10:28,018
learning fit into the engineering

1545
01:10:26,670 --> 01:10:30,090
curriculum because it's an advanced

1546
01:10:28,019 --> 01:10:31,800
concept so perhaps you know you look at

1547
01:10:30,090 --> 01:10:33,539
undergraduate courses in image

1548
01:10:31,800 --> 01:10:36,420
processing or computer vision or on

1549
01:10:33,539 --> 01:10:38,039
neural networks or you know a course on

1550
01:10:36,420 --> 01:10:39,689
adaptive signal processing or methods

1551
01:10:38,039 --> 01:10:41,909
for autonomous design please do reach

1552
01:10:39,689 --> 01:10:43,409
out to me if you want to work on any of

1553
01:10:41,909 --> 01:10:46,549
these we are more than happy to support

1554
01:10:43,409 --> 01:10:51,438
you on that now what I want to do is

1555
01:10:46,550 --> 01:10:56,189
just just show you five lines of code

1556
01:10:51,439 --> 01:11:02,269
okay for deep learning okay so I'm gonna

1557
01:10:56,189 --> 01:11:02,269
go back to my MATLAB and I'm gonna open

1558
01:11:06,409 --> 01:11:15,780
okay now what I'd like to show you is

1559
01:11:12,260 --> 01:11:17,760
essentially five lines so I'm gonna sort

1560
01:11:15,780 --> 01:11:19,768
of have this on the side here and

1561
01:11:17,760 --> 01:11:22,590
perform the same actions just so that

1562
01:11:19,769 --> 01:11:26,499
you are able to see what I'm doing okay

1563
01:11:22,590 --> 01:11:30,519
so let me

1564
01:11:26,499 --> 01:11:36,659
um minimize this let me also minimize

1565
01:11:30,519 --> 01:11:39,729
this and let me minimize this one okay

1566
01:11:36,659 --> 01:11:43,179
okay so the first step I'm going to be

1567
01:11:39,729 --> 01:11:45,239
doing is net equals Alex net okay

1568
01:11:43,179 --> 01:11:48,639
so what I'm doing is I have a

1569
01:11:45,239 --> 01:11:51,820
pre-trained model okay and MATLAB is

1570
01:11:48,639 --> 01:11:53,559
able to access it for me or internally

1571
01:11:51,820 --> 01:11:56,139
from an add-on Explorer I'll show you

1572
01:11:53,559 --> 01:11:58,960
how to do that I can say net equals Alex

1573
01:11:56,139 --> 01:12:00,489
net and when I run it it essentially of

1574
01:11:58,960 --> 01:12:02,979
course it says it's busy because it's

1575
01:12:00,489 --> 01:12:05,978
trying to load it what do I have your

1576
01:12:02,979 --> 01:12:09,219
net equals its it has layers it's a

1577
01:12:05,979 --> 01:12:11,320
twenty five by one neural network with a

1578
01:12:09,219 --> 01:12:14,170
convolution neural network we'll come to

1579
01:12:11,320 --> 01:12:18,759
that okay next what I'm going to do is

1580
01:12:14,170 --> 01:12:22,929
I'm going to read as an image which is

1581
01:12:18,760 --> 01:12:25,119
already there in matlab which is peppers

1582
01:12:22,929 --> 01:12:28,960
and I will store it and what is known as

1583
01:12:25,119 --> 01:12:36,280
an image one or something okay so when I

1584
01:12:28,960 --> 01:12:39,010
say I am sure of image one essentially

1585
01:12:36,280 --> 01:12:40,659
is capsicum okay it is capsicum which is

1586
01:12:39,010 --> 01:12:44,050
known as peppers in other parts of the

1587
01:12:40,659 --> 01:12:46,839
world okay now all I need to do is I

1588
01:12:44,050 --> 01:12:49,989
need to resize the image such that it

1589
01:12:46,840 --> 01:12:56,590
fits into the Alex Nets our framework so

1590
01:12:49,989 --> 01:13:02,558
I'll say image 1 re resized equals I am

1591
01:12:56,590 --> 01:13:09,039
resize of I image 1 comma I just put in

1592
01:13:02,559 --> 01:13:12,400
the okay and then all I need to do is

1593
01:13:09,039 --> 01:13:17,440
one line of code to say what it is so

1594
01:13:12,400 --> 01:13:19,808
label equals pacify equal network that I

1595
01:13:17,440 --> 01:13:24,369
have defined an Alex net and pick the

1596
01:13:19,809 --> 01:13:26,679
image that is resized okay and it

1597
01:13:24,369 --> 01:13:30,489
performs a classification and what did I

1598
01:13:26,679 --> 01:13:32,650
get the classification as well okay

1599
01:13:30,489 --> 01:13:35,678
which is essentially what it is and then

1600
01:13:32,650 --> 01:13:37,808
I can this is essentially what it is and

1601
01:13:35,679 --> 01:13:39,130
it gives me the right in at the right

1602
01:13:37,809 --> 01:13:40,990
example

1603
01:13:39,130 --> 01:13:43,150
or the right answer for this particular

1604
01:13:40,990 --> 01:13:45,070
example yeah so how many lines of code

1605
01:13:43,150 --> 01:13:47,610
did I write can someone summarize what I

1606
01:13:45,070 --> 01:13:51,120
have done in these five lines of code

1607
01:13:47,610 --> 01:13:51,120
what is the first step

1608
01:13:52,260 --> 01:13:58,110
new neural network choose the pre train

1609
01:13:55,330 --> 01:14:00,580
model and then what am I doing here

1610
01:13:58,110 --> 01:14:02,769
reading the image and then displaying it

1611
01:14:00,580 --> 01:14:05,470
and then resizing it the only line of

1612
01:14:02,770 --> 01:14:06,600
code is what I am writing is classify is

1613
01:14:05,470 --> 01:14:13,960
that correct

1614
01:14:06,600 --> 01:14:15,730
okay so essentially what I used here was

1615
01:14:13,960 --> 01:14:18,340
what is known as Alex net you have

1616
01:14:15,730 --> 01:14:22,959
multiple other networks and the way I

1617
01:14:18,340 --> 01:14:24,610
can get that in MATLAB is go in to the

1618
01:14:22,960 --> 01:14:29,260
home and I have something known as

1619
01:14:24,610 --> 01:14:31,240
add-ons and I can say get add-ons while

1620
01:14:29,260 --> 01:14:33,010
that opens up I will tell you so these

1621
01:14:31,240 --> 01:14:34,540
add-ons are something but both math

1622
01:14:33,010 --> 01:14:36,910
works develops and third-party

1623
01:14:34,540 --> 01:14:39,310
developers and user contributed people

1624
01:14:36,910 --> 01:14:42,010
develop like it's like file exchange but

1625
01:14:39,310 --> 01:14:45,880
it is within MATLAB and I can go and say

1626
01:14:42,010 --> 01:14:48,220
Alex net okay and if you notice over

1627
01:14:45,880 --> 01:14:51,160
here it says neural networks toolbox

1628
01:14:48,220 --> 01:14:51,850
model for Alex net yeah and it's a math

1629
01:14:51,160 --> 01:14:53,920
ox feature

1630
01:14:51,850 --> 01:14:56,650
it contains pre-trained thing and in my

1631
01:14:53,920 --> 01:15:02,020
case it is installed I can also end up

1632
01:14:56,650 --> 01:15:03,429
saying Google net if I want and google

1633
01:15:02,020 --> 01:15:05,410
net of course there is something that I

1634
01:15:03,430 --> 01:15:06,730
have here neural networks toolbox for

1635
01:15:05,410 --> 01:15:10,480
the Google net I don't have it installed

1636
01:15:06,730 --> 01:15:12,969
all I need to do is go here and then say

1637
01:15:10,480 --> 01:15:15,730
install and it will end up installing it

1638
01:15:12,970 --> 01:15:24,550
and it says it's functional for our 2017

1639
01:15:15,730 --> 01:15:26,559
B and beyond okay now so you're also

1640
01:15:24,550 --> 01:15:29,710
able to import models from other

1641
01:15:26,560 --> 01:15:31,510
frameworks so if you notice was it just

1642
01:15:29,710 --> 01:15:33,970
one line of code that I mentioned and

1643
01:15:31,510 --> 01:15:37,240
that'll be the same thing even for say

1644
01:15:33,970 --> 01:15:39,070
Google net or many other things and if I

1645
01:15:37,240 --> 01:15:41,380
want to import the models I can just say

1646
01:15:39,070 --> 01:15:43,269
import Kiera's layer or input the tensor

1647
01:15:41,380 --> 01:15:45,610
flow Karros model layer Kara's Network

1648
01:15:43,270 --> 01:15:46,600
any of those and you're able to download

1649
01:15:45,610 --> 01:15:50,679
from within

1650
01:15:46,600 --> 01:15:52,970
MATLAB yeah deep learning as I mentioned

1651
01:15:50,680 --> 01:15:55,310
uses a neural network architecture

1652
01:15:52,970 --> 01:15:57,620
there's an input layer hidden layers an

1653
01:15:55,310 --> 01:16:00,020
output layer in the previous talk in the

1654
01:15:57,620 --> 01:16:02,480
previous saw or topic when we talked

1655
01:16:00,020 --> 01:16:05,120
about the hidden layers was one in this

1656
01:16:02,480 --> 01:16:08,299
case it's n number of hidden layers okay

1657
01:16:05,120 --> 01:16:10,550
so I'll go over what this convolutional

1658
01:16:08,300 --> 01:16:12,230
neural network does is but the problem

1659
01:16:10,550 --> 01:16:14,810
the only the problem statement

1660
01:16:12,230 --> 01:16:16,790
essentially is the input and we need to

1661
01:16:14,810 --> 01:16:18,470
get the output we have the feature

1662
01:16:16,790 --> 01:16:20,630
learning and then we have the

1663
01:16:18,470 --> 01:16:22,190
classification that is being done who

1664
01:16:20,630 --> 01:16:24,410
learns a feature the network's learns a

1665
01:16:22,190 --> 01:16:26,030
feature who performs a classification

1666
01:16:24,410 --> 01:16:32,930
that's also done and it's completely

1667
01:16:26,030 --> 01:16:36,259
automated ok so any questions so far so

1668
01:16:32,930 --> 01:16:39,320
it is like that we should have a network

1669
01:16:36,260 --> 01:16:42,620
full of pre-trained you make this then

1670
01:16:39,320 --> 01:16:45,259
we'll be able to do yes or you can

1671
01:16:42,620 --> 01:16:47,870
create your own pre-trained images also

1672
01:16:45,260 --> 01:16:50,510
we will be talking about that do or you

1673
01:16:47,870 --> 01:16:52,370
can you know like use the pre trained

1674
01:16:50,510 --> 01:16:55,130
thing but then add your own layers as

1675
01:16:52,370 --> 01:16:58,550
well they will come to that as well so

1676
01:16:55,130 --> 01:17:01,790
now what I want to show you is a you

1677
01:16:58,550 --> 01:17:03,860
know a new feature in MATLAB which is

1678
01:17:01,790 --> 01:17:08,019
known as the live editor or the live

1679
01:17:03,860 --> 01:17:13,670
script so what I'd like you to see is

1680
01:17:08,020 --> 01:17:16,370
this so if you notice here I have the

1681
01:17:13,670 --> 01:17:19,160
same thing if you notice I have the same

1682
01:17:16,370 --> 01:17:21,260
text that I had before but here I have

1683
01:17:19,160 --> 01:17:23,059
what is known as the code and then I

1684
01:17:21,260 --> 01:17:24,880
have the classification and I of course

1685
01:17:23,060 --> 01:17:28,010
it says there's some error here or and

1686
01:17:24,880 --> 01:17:30,860
we'll we'll see why that is but is this

1687
01:17:28,010 --> 01:17:33,650
seen in a nice way in terms of like

1688
01:17:30,860 --> 01:17:36,469
almost like a textbook yeah where you

1689
01:17:33,650 --> 01:17:38,809
are able to show a little bit of text

1690
01:17:36,470 --> 01:17:40,700
you're able to put in MATLAB code you

1691
01:17:38,810 --> 01:17:42,430
will also be able to see the output I'll

1692
01:17:40,700 --> 01:17:46,340
show you another example of that

1693
01:17:42,430 --> 01:17:50,540
nice it's called as a live script okay

1694
01:17:46,340 --> 01:17:54,350
so let me run this for you so I will run

1695
01:17:50,540 --> 01:17:56,870
all I'm hoping it runs without or

1696
01:17:54,350 --> 01:17:59,260
showing any errors it's performing the

1697
01:17:56,870 --> 01:18:02,000
running here it's doing the network

1698
01:17:59,260 --> 01:18:03,830
loading and then it's performing the

1699
01:18:02,000 --> 01:18:06,140
classification and it's already

1700
01:18:03,830 --> 01:18:07,010
classified it so if you notice all of

1701
01:18:06,140 --> 01:18:09,710
the

1702
01:18:07,010 --> 01:18:12,650
puts so let me say I remove this yeah

1703
01:18:09,710 --> 01:18:15,050
let me run it all of my output plot

1704
01:18:12,650 --> 01:18:17,750
everything comes in the same window and

1705
01:18:15,050 --> 01:18:20,150
it's almost like a very nice notebook

1706
01:18:17,750 --> 01:18:22,820
yeah it's like one notebook where I have

1707
01:18:20,150 --> 01:18:24,739
the network properties I say what the

1708
01:18:22,820 --> 01:18:26,989
image thing is and then what the image

1709
01:18:24,739 --> 01:18:29,239
looks like and then I resize it and then

1710
01:18:26,989 --> 01:18:33,080
tell me what the variable name is and

1711
01:18:29,239 --> 01:18:36,169
then I can also export it to a PDF or a

1712
01:18:33,080 --> 01:18:38,989
HTML or a latex file so you save a lot

1713
01:18:36,170 --> 01:18:41,180
of time in rewriting or copy pasting

1714
01:18:38,989 --> 01:18:43,160
into save word or later or something

1715
01:18:41,180 --> 01:18:45,980
like that because it's already in the

1716
01:18:43,160 --> 01:18:48,170
form of a nice rich text editor yeah and

1717
01:18:45,980 --> 01:18:52,280
this is something that was introduced in

1718
01:18:48,170 --> 01:18:56,630
our 2016 it okay this is extension is

1719
01:18:52,280 --> 01:18:59,349
dot ml x okay any questions on this or

1720
01:18:56,630 --> 01:19:01,730
what are the live script go to that legs

1721
01:18:59,350 --> 01:19:05,290
how can you go to that live script let

1722
01:19:01,730 --> 01:19:08,780
me just show you an example so you can

1723
01:19:05,290 --> 01:19:11,030
you can hear in 17 B it says new script

1724
01:19:08,780 --> 01:19:14,989
or new live script you can click on that

1725
01:19:11,030 --> 01:19:17,509
and then I will I will have I will say

1726
01:19:14,989 --> 01:19:21,160
it's a text I want to put a heading I

1727
01:19:17,510 --> 01:19:24,260
will say and I triple is PR training

1728
01:19:21,160 --> 01:19:30,080
okay and then I'll put some I'll put

1729
01:19:24,260 --> 01:19:35,300
some heading same example one and then

1730
01:19:30,080 --> 01:19:38,360
I'll say plot sine wave then I will say

1731
01:19:35,300 --> 01:19:41,840
it is not text but now it will go into

1732
01:19:38,360 --> 01:19:51,429
code I will click on code okay and I

1733
01:19:41,840 --> 01:19:59,090
will say T equals minus PI pi

1734
01:19:51,430 --> 01:20:01,880
and I'll say plot sign off okay and now

1735
01:19:59,090 --> 01:20:08,660
when I run this particular section

1736
01:20:01,880 --> 01:20:11,330
I got yes is that nice yeah and what I

1737
01:20:08,660 --> 01:20:17,660
can do is I can go here and say X label

1738
01:20:11,330 --> 01:20:19,489
equals I can put it in here and as soon

1739
01:20:17,660 --> 01:20:20,909
as I run that section it will end up

1740
01:20:19,489 --> 01:20:23,519
getting updated right away

1741
01:20:20,909 --> 01:20:25,949
okay and I can also say perform

1742
01:20:23,520 --> 01:20:27,929
annotations or move it around or zoom it

1743
01:20:25,949 --> 01:20:30,540
and it will get reflected here as well

1744
01:20:27,929 --> 01:20:32,730
okay next what I can do is I will say I

1745
01:20:30,540 --> 01:20:36,600
want a text and I want it to be an

1746
01:20:32,730 --> 01:20:40,409
example so I'll say example - okay and

1747
01:20:36,600 --> 01:20:46,530
I'll say I want to perform take an AA

1748
01:20:40,409 --> 01:20:50,099
metric samples I can say that and then

1749
01:20:46,530 --> 01:20:56,190
I'll go to code I'll say Sims of X and

1750
01:20:50,100 --> 01:21:04,710
I'll say y equals sign of maybe let's

1751
01:20:56,190 --> 01:21:07,678
say X X square plus log of X and then

1752
01:21:04,710 --> 01:21:09,480
I'll say what is bit of wine that is

1753
01:21:07,679 --> 01:21:14,370
differential and I'll say what is in

1754
01:21:09,480 --> 01:21:18,299
tough white and I'll say F plot of what

1755
01:21:14,370 --> 01:21:26,849
I'll put it in a thing saying plot F of

1756
01:21:18,300 --> 01:21:29,580
Y and okay so far is it easy then let me

1757
01:21:26,850 --> 01:21:33,600
run this particular section and if you

1758
01:21:29,580 --> 01:21:36,750
notice it's of course it will take a

1759
01:21:33,600 --> 01:21:41,340
little bit of time so I got my value

1760
01:21:36,750 --> 01:21:43,679
which is say sine X square plus log X

1761
01:21:41,340 --> 01:21:45,780
and the differential is this correct - x

1762
01:21:43,679 --> 01:21:48,330
cos x square plus this one and then the

1763
01:21:45,780 --> 01:21:51,840
integral yeah and it's still evaluating

1764
01:21:48,330 --> 01:21:52,920
what the or how it should be plotting

1765
01:21:51,840 --> 01:21:54,239
because this is a more complex

1766
01:21:52,920 --> 01:21:56,969
expression because it's a friend

1767
01:21:54,239 --> 01:21:58,739
function so you will end up getting a

1768
01:21:56,969 --> 01:22:00,659
plot in some time or it might end up

1769
01:21:58,739 --> 01:22:02,280
showing an error but is this a nice way

1770
01:22:00,659 --> 01:22:04,199
for the students to learn and you know

1771
01:22:02,280 --> 01:22:08,429
you're able to see everything in line

1772
01:22:04,199 --> 01:22:10,259
itself right so this is what is known as

1773
01:22:08,429 --> 01:22:12,630
the live editor which is what I wanted

1774
01:22:10,260 --> 01:22:13,889
to show you in terms of deep learning in

1775
01:22:12,630 --> 01:22:15,929
terms of five lines of code

1776
01:22:13,889 --> 01:22:18,630
yeah and if you notice here how does

1777
01:22:15,929 --> 01:22:21,810
this plot look like is it very nice so

1778
01:22:18,630 --> 01:22:27,719
this blue one essentially shows my input

1779
01:22:21,810 --> 01:22:29,100
right this one here is my derivative the

1780
01:22:27,719 --> 01:22:31,800
sorry the yellow one is my derivative

1781
01:22:29,100 --> 01:22:34,800
and the orange one is my integral I can

1782
01:22:31,800 --> 01:22:41,700
put in legend if I want and I can say

1783
01:22:34,800 --> 01:22:51,720
um you know I can see why put in this

1784
01:22:41,700 --> 01:23:01,910
offline and then I can put in and I just

1785
01:22:51,720 --> 01:23:01,910
run or IRA I just run that any questions

1786
01:23:06,920 --> 01:23:12,270
we'll we'll wait for that to do it but

1787
01:23:09,330 --> 01:23:14,970
we'll go back to another example what I

1788
01:23:12,270 --> 01:23:18,600
would like to show you next is how do

1789
01:23:14,970 --> 01:23:20,670
you do it for multiple images okay so

1790
01:23:18,600 --> 01:23:24,990
let's let's take a look at that so I'm

1791
01:23:20,670 --> 01:23:27,240
gonna take a look at an example which

1792
01:23:24,990 --> 01:23:31,019
you will see in just a little bit okay

1793
01:23:27,240 --> 01:23:41,480
it's D it's not opening for me because

1794
01:23:31,020 --> 01:23:41,480
okay we just add two paths that's still

1795
01:23:41,540 --> 01:23:46,050
it's still taking a little while but you

1796
01:23:44,490 --> 01:23:47,820
know what I want what I wanted to show

1797
01:23:46,050 --> 01:23:49,850
you okay it did that

1798
01:23:47,820 --> 01:23:52,559
so you see here I'm able to get the

1799
01:23:49,850 --> 01:23:54,450
different things here yeah the

1800
01:23:52,560 --> 01:23:57,720
differential and the integral component

1801
01:23:54,450 --> 01:24:01,380
yeah now what I'm gonna do is I'm going

1802
01:23:57,720 --> 01:24:03,480
to take into account what happens when

1803
01:24:01,380 --> 01:24:06,990
there's a pre-trained Network I'm gonna

1804
01:24:03,480 --> 01:24:09,839
explore expose it a little more okay so

1805
01:24:06,990 --> 01:24:11,490
this may not work now because I see an

1806
01:24:09,840 --> 01:24:14,040
error but what I want to show you is the

1807
01:24:11,490 --> 01:24:15,690
overall workflow okay so I'm clear thing

1808
01:24:14,040 --> 01:24:17,670
and what is the first step I'm doing

1809
01:24:15,690 --> 01:24:23,610
here can someone tell me net equals Alex

1810
01:24:17,670 --> 01:24:26,280
net what does it do load the network and

1811
01:24:23,610 --> 01:24:27,690
then next component I'm inspecting the

1812
01:24:26,280 --> 01:24:30,450
network I will see what the network

1813
01:24:27,690 --> 01:24:32,610
layers are okay and then I'll also see

1814
01:24:30,450 --> 01:24:35,429
what the last part is what is there in

1815
01:24:32,610 --> 01:24:37,700
terms of classes okay I have a thousand

1816
01:24:35,430 --> 01:24:40,740
by one cell array so it essentially has

1817
01:24:37,700 --> 01:24:43,230
thousand different classification okay

1818
01:24:40,740 --> 01:24:44,670
that is my output layer then what am I

1819
01:24:43,230 --> 01:24:48,949
going to perform I'm not going to take

1820
01:24:44,670 --> 01:24:52,770
what just one image what do I have here

1821
01:24:48,949 --> 01:24:55,799
like one particular folder so here if

1822
01:24:52,770 --> 01:24:57,480
you go here in MATLAB I have a folder

1823
01:24:55,800 --> 01:24:59,909
with a lot of images around twelve

1824
01:24:57,480 --> 01:25:02,669
images over here okay and I don't want

1825
01:24:59,909 --> 01:25:04,920
to keep up doing a lot doing it over and

1826
01:25:02,670 --> 01:25:06,389
over again or put write a different for

1827
01:25:04,920 --> 01:25:09,350
loop and all that there's a smarter way

1828
01:25:06,389 --> 01:25:13,139
of doing it okay so what I will do is I

1829
01:25:09,350 --> 01:25:15,510
will essentially end up taking this

1830
01:25:13,139 --> 01:25:18,090
image what is the path of the image

1831
01:25:15,510 --> 01:25:21,690
okay and I will show what the image is

1832
01:25:18,090 --> 01:25:23,639
yeah but I won't do this just once what

1833
01:25:21,690 --> 01:25:26,610
I will do is I will run this in a loop

1834
01:25:23,639 --> 01:25:29,310
using what is known as beep restore it's

1835
01:25:26,610 --> 01:25:32,759
like a pointer okay it will essentially

1836
01:25:29,310 --> 01:25:35,820
take into account all the files that are

1837
01:25:32,760 --> 01:25:38,550
there in this particular folder okay and

1838
01:25:35,820 --> 01:25:40,679
this will be the pointer and whatever I

1839
01:25:38,550 --> 01:25:42,630
get here whatever I say image data

1840
01:25:40,679 --> 01:25:45,389
stores that as image path this it will

1841
01:25:42,630 --> 01:25:47,280
essentially store it like a pointer it

1842
01:25:45,389 --> 01:25:49,440
won't load all the data into the

1843
01:25:47,280 --> 01:25:51,840
workspace but it will end up saying I

1844
01:25:49,440 --> 01:25:54,870
have IDs I have three images plus nine

1845
01:25:51,840 --> 01:25:57,600
more so a total of twelve images but I'm

1846
01:25:54,870 --> 01:26:00,809
I'm only having it as a pointer is that

1847
01:25:57,600 --> 01:26:03,210
okay yeah and I'm displaying that next

1848
01:26:00,810 --> 01:26:06,540
what I will do is while I'm doing that

1849
01:26:03,210 --> 01:26:08,670
while I ensure that I end up getting all

1850
01:26:06,540 --> 01:26:10,610
the images I'm performing the same thing

1851
01:26:08,670 --> 01:26:15,330
I read the image sorry I read the data

1852
01:26:10,610 --> 01:26:19,110
okay and I resize it and then I classify

1853
01:26:15,330 --> 01:26:22,199
the image okay but I'm running it over

1854
01:26:19,110 --> 01:26:24,059
thousand samples yeah and you know what

1855
01:26:22,199 --> 01:26:27,330
is the max score that I get and show me

1856
01:26:24,060 --> 01:26:32,310
what it is okay so is this a seashore

1857
01:26:27,330 --> 01:26:35,460
here yeah but what about this this is a

1858
01:26:32,310 --> 01:26:39,449
breakwater this is okay but then is this

1859
01:26:35,460 --> 01:26:41,340
a hot dog No so you're getting a value

1860
01:26:39,449 --> 01:26:44,699
that's point 1 1 so it means to be

1861
01:26:41,340 --> 01:26:46,050
trained better is this a bakery it's a

1862
01:26:44,699 --> 01:26:49,440
part of a bakery but it should have been

1863
01:26:46,050 --> 01:26:54,150
cupcakes is that right this is a sheep

1864
01:26:49,440 --> 01:26:55,469
no it's not is this a lakeside maybe but

1865
01:26:54,150 --> 01:26:58,138
a frozen lake site but it took the

1866
01:26:55,469 --> 01:27:00,239
sunset into account and then is this a

1867
01:26:58,139 --> 01:27:02,430
bucket no of course there's a kid inside

1868
01:27:00,239 --> 01:27:04,980
a bucket it forgot the kid but it showed

1869
01:27:02,430 --> 01:27:07,620
did the bucket this is a couch of course

1870
01:27:04,980 --> 01:27:10,469
it is then is this a cardigan perhaps

1871
01:27:07,620 --> 01:27:13,010
not and then is this a Japanese spaniels

1872
01:27:10,470 --> 01:27:17,310
so tidal no this is a fire extinguisher

1873
01:27:13,010 --> 01:27:19,560
sand bar no its beach sand yeah and you

1874
01:27:17,310 --> 01:27:22,500
can puff up the same thing by you know

1875
01:27:19,560 --> 01:27:24,930
reading something from let me not try to

1876
01:27:22,500 --> 01:27:27,780
run this section you can read any image

1877
01:27:24,930 --> 01:27:31,010
and then you're able to classify it and

1878
01:27:27,780 --> 01:27:34,050
you can do that for F web image as well

1879
01:27:31,010 --> 01:27:36,570
is it very easy like what is the only

1880
01:27:34,050 --> 01:27:40,800
key thing that we changed here we ended

1881
01:27:36,570 --> 01:27:44,820
up we ended up storing the whole data

1882
01:27:40,800 --> 01:27:46,860
path in what is known as IDs we provided

1883
01:27:44,820 --> 01:27:50,849
it with a pointer and we were able to

1884
01:27:46,860 --> 01:27:52,950
show what the overall thing is for all

1885
01:27:50,850 --> 01:27:56,670
the images in that folder is that

1886
01:27:52,950 --> 01:28:02,660
correct yeah so an extension of what you

1887
01:27:56,670 --> 01:28:05,870
did in this case ok so let's go back to

1888
01:28:02,660 --> 01:28:08,670
the slides

1889
01:28:05,870 --> 01:28:10,620
so pre-trained networks have a

1890
01:28:08,670 --> 01:28:12,180
predetermined layer order which makes

1891
01:28:10,620 --> 01:28:13,559
them effective for classifying images

1892
01:28:12,180 --> 01:28:15,810
they're typically trained to classify a

1893
01:28:13,560 --> 01:28:17,580
lot of images different networks so say

1894
01:28:15,810 --> 01:28:19,170
Google net will be different they are a

1895
01:28:17,580 --> 01:28:22,140
great starting point but not

1896
01:28:19,170 --> 01:28:23,640
consistently accurate ok we will discuss

1897
01:28:22,140 --> 01:28:26,760
what transfer learning is tomorrow if

1898
01:28:23,640 --> 01:28:30,060
there's a way you can add more layers to

1899
01:28:26,760 --> 01:28:31,320
it and things like that yeah so what I

1900
01:28:30,060 --> 01:28:33,360
want to say is deep learning is

1901
01:28:31,320 --> 01:28:37,200
definitely not complicated it can be

1902
01:28:33,360 --> 01:28:38,910
very easy ok if you notice it's simple

1903
01:28:37,200 --> 01:28:40,530
the only thing is you need to understand

1904
01:28:38,910 --> 01:28:45,570
the concept of neural networks pretty

1905
01:28:40,530 --> 01:28:47,519
well I would actually want to take a

1906
01:28:45,570 --> 01:28:49,500
little bit of time and explaining the

1907
01:28:47,520 --> 01:28:51,720
background of a convolutional neural

1908
01:28:49,500 --> 01:28:55,250
network which is there so if you notice

1909
01:28:51,720 --> 01:28:57,510
I have an input what is this layer here

1910
01:28:55,250 --> 01:28:59,040
feature learning and then the

1911
01:28:57,510 --> 01:29:01,560
classification I want to spend a little

1912
01:28:59,040 --> 01:29:05,250
time on feature layer so assume that you

1913
01:29:01,560 --> 01:29:06,930
want to detect numerals okay so the

1914
01:29:05,250 --> 01:29:08,640
convolutional neural network has three

1915
01:29:06,930 --> 01:29:10,890
components what's the first layer your

1916
01:29:08,640 --> 01:29:12,900
convolution layer the second layer is

1917
01:29:10,890 --> 01:29:15,270
the max pooling layer third layer is

1918
01:29:12,900 --> 01:29:16,199
called the ALU layer okay I'll tell you

1919
01:29:15,270 --> 01:29:18,209
what each of

1920
01:29:16,199 --> 01:29:20,009
leaders are and slob for questions and

1921
01:29:18,209 --> 01:29:22,920
we can continue tomorrow so the

1922
01:29:20,010 --> 01:29:26,010
convolution layer what it does is it

1923
01:29:22,920 --> 01:29:29,309
takes the input and searches for

1924
01:29:26,010 --> 01:29:31,380
multiple patterns okay in this case it's

1925
01:29:29,309 --> 01:29:33,570
a three by three matrix how many

1926
01:29:31,380 --> 01:29:35,729
patterns can we have we can have 27

1927
01:29:33,570 --> 01:29:38,820
patterns is that correct this pattern

1928
01:29:35,729 --> 01:29:41,039
here is there in this yeah this straight

1929
01:29:38,820 --> 01:29:44,039
pattern can be there in this here or

1930
01:29:41,039 --> 01:29:46,440
this one is there here so what it does

1931
01:29:44,039 --> 01:29:50,249
is it takes each of these patterns and

1932
01:29:46,440 --> 01:29:53,759
slides it through the image and performs

1933
01:29:50,249 --> 01:29:56,099
correlation okay and sees which patterns

1934
01:29:53,760 --> 01:29:57,840
give the most score so it performs

1935
01:29:56,099 --> 01:29:59,699
correlation and tries to see what the

1936
01:29:57,840 --> 01:30:02,610
patterns that give the most score are

1937
01:29:59,699 --> 01:30:05,058
and chooses that particular pattern okay

1938
01:30:02,610 --> 01:30:08,308
so any questions so far

1939
01:30:05,059 --> 01:30:10,590
okay these are of course binary images

1940
01:30:08,309 --> 01:30:12,659
the next part is all the patterns are

1941
01:30:10,590 --> 01:30:15,420
compared to the patterns on a new image

1942
01:30:12,659 --> 01:30:17,339
okay it starts at a left corner slide

1943
01:30:15,420 --> 01:30:18,749
over one by one pixel and repeat for the

1944
01:30:17,340 --> 01:30:22,440
next pattern it's a greedy search

1945
01:30:18,749 --> 01:30:24,389
algorithm next a good pattern matching

1946
01:30:22,440 --> 01:30:26,129
what would happen is so this one will

1947
01:30:24,389 --> 01:30:28,019
only end up doing good against this

1948
01:30:26,130 --> 01:30:29,760
pattern it won't do well against the

1949
01:30:28,019 --> 01:30:33,780
other pattern so we'll see what the best

1950
01:30:29,760 --> 01:30:36,030
pattern is okay so what we try what we

1951
01:30:33,780 --> 01:30:38,159
are going to get here is in this case

1952
01:30:36,030 --> 01:30:41,369
will we end up getting a score for all

1953
01:30:38,159 --> 01:30:43,018
of these patterns is that right yeah but

1954
01:30:41,369 --> 01:30:43,499
it will occur in various parts of the

1955
01:30:43,019 --> 01:30:45,719
image

1956
01:30:43,499 --> 01:30:47,789
you may end up getting a matrix like

1957
01:30:45,719 --> 01:30:49,260
this you understand you will end up

1958
01:30:47,789 --> 01:30:52,409
getting a matrix like this which takes

1959
01:30:49,260 --> 01:30:54,749
what the score is and you say in this

1960
01:30:52,409 --> 01:30:58,558
case this was a pattern that was the

1961
01:30:54,749 --> 01:31:00,840
best so let me have for this pattern was

1962
01:30:58,559 --> 01:31:03,360
the best let me have eight in this case

1963
01:31:00,840 --> 01:31:05,760
again five and then six this is like

1964
01:31:03,360 --> 01:31:07,709
down sampling we shrink the large images

1965
01:31:05,760 --> 01:31:09,869
while performing them but preserving the

1966
01:31:07,709 --> 01:31:11,309
important information that is a second

1967
01:31:09,869 --> 01:31:13,619
layer which is called the max pooling

1968
01:31:11,309 --> 01:31:15,389
the third thing the rectified linear

1969
01:31:13,619 --> 01:31:18,389
units layer is if you have a negative

1970
01:31:15,389 --> 01:31:21,599
value in your image they are all set to

1971
01:31:18,389 --> 01:31:23,729
zero mean they're all set to zero and

1972
01:31:21,599 --> 01:31:25,380
which is where we have the three

1973
01:31:23,729 --> 01:31:28,170
components so I just want to go back to

1974
01:31:25,380 --> 01:31:29,690
the same thing so what do we have here

1975
01:31:28,170 --> 01:31:32,420
the convolution layer

1976
01:31:29,690 --> 01:31:34,610
forms pattern search over multiple

1977
01:31:32,420 --> 01:31:36,590
different types of patterns and you have

1978
01:31:34,610 --> 01:31:38,870
the rail U which is essentially sets the

1979
01:31:36,590 --> 01:31:41,330
con sets of a correlation value to zero

1980
01:31:38,870 --> 01:31:43,610
what this pooling do it will essentially

1981
01:31:41,330 --> 01:31:46,190
perform down sampling yeah and that is

1982
01:31:43,610 --> 01:31:48,559
repeated over multiple things and then

1983
01:31:46,190 --> 01:31:51,739
once you get that value it is flattened

1984
01:31:48,560 --> 01:31:53,600
out okay and then there's one thing

1985
01:31:51,739 --> 01:31:55,459
called the fully connected thing and

1986
01:31:53,600 --> 01:31:57,199
then we perform the softmax each of

1987
01:31:55,460 --> 01:31:59,570
these details will be sent to you

1988
01:31:57,199 --> 01:32:01,219
shortly and these are mathematical

1989
01:31:59,570 --> 01:32:03,049
operations we essentially try to instead

1990
01:32:01,219 --> 01:32:04,880
of a vector we try to ensure that it's a

1991
01:32:03,050 --> 01:32:07,250
flattened layer and then we ensure that

1992
01:32:04,880 --> 01:32:09,530
this connection between the image pixels

1993
01:32:07,250 --> 01:32:11,660
and then you end up saying what is the

1994
01:32:09,530 --> 01:32:15,050
best correlation that's over there is

1995
01:32:11,660 --> 01:32:20,809
that clear yeah so this is what you

1996
01:32:15,050 --> 01:32:24,080
might have seen in your MATLAB command

1997
01:32:20,810 --> 01:32:26,870
window here so in the first case what

1998
01:32:24,080 --> 01:32:28,150
was the first layer what was the first

1999
01:32:26,870 --> 01:32:31,250
layer here

2000
01:32:28,150 --> 01:32:34,549
imagine put the second layer convolution

2001
01:32:31,250 --> 01:32:37,010
value max pooling again convolution

2002
01:32:34,550 --> 01:32:38,960
value max pooling convolution value

2003
01:32:37,010 --> 01:32:41,890
convolution value multiple times and

2004
01:32:38,960 --> 01:32:44,870
then finally what is the last layer

2005
01:32:41,890 --> 01:32:48,650
classification output yeah so is that in

2006
01:32:44,870 --> 01:32:51,650
sync with what you saw as um not here

2007
01:32:48,650 --> 01:32:58,670
sorry what you saw as the different

2008
01:32:51,650 --> 01:33:00,469
layers so okay so this is where your

2009
01:32:58,670 --> 01:33:02,449
neural network background will come off

2010
01:33:00,469 --> 01:33:04,550
help in deep learning it's more the

2011
01:33:02,449 --> 01:33:06,980
number of hidden layers better the

2012
01:33:04,550 --> 01:33:08,870
classification and which is why deep

2013
01:33:06,980 --> 01:33:10,968
learning has become a bigger phenomena

2014
01:33:08,870 --> 01:33:12,530
and why has that become because earlier

2015
01:33:10,969 --> 01:33:14,780
you were neural networks were very

2016
01:33:12,530 --> 01:33:16,790
computationally intensive but at this

2017
01:33:14,780 --> 01:33:20,300
point of time they are not and we are

2018
01:33:16,790 --> 01:33:23,120
able to you know use GPUs in order to do

2019
01:33:20,300 --> 01:33:27,020
that ok so what I would like to touch

2020
01:33:23,120 --> 01:33:31,160
upon 30 minutes tomorrow is what are the

2021
01:33:27,020 --> 01:33:33,469
best cases where you can do um this form

2022
01:33:31,160 --> 01:33:35,210
of an approach or in a few cases where

2023
01:33:33,469 --> 01:33:38,239
you might need to introduce additional

2024
01:33:35,210 --> 01:33:40,010
layers and perform the lay learning ok

2025
01:33:38,239 --> 01:33:42,530
so those are the things that we will be

2026
01:33:40,010 --> 01:33:43,309
talking about so just to summarize what

2027
01:33:42,530 --> 01:33:46,369
I want to show you

2028
01:33:43,309 --> 01:33:47,780
is of course you have the flattened

2029
01:33:46,369 --> 01:33:49,459
layer which flattens in neurons in a

2030
01:33:47,780 --> 01:33:51,199
previous layer and then fully connected

2031
01:33:49,459 --> 01:33:52,999
which it says which are the high level

2032
01:33:51,199 --> 01:33:55,669
layers and softmax it's on the scores

2033
01:33:52,999 --> 01:33:58,909
and the probabilities what I want to

2034
01:33:55,669 --> 01:34:01,639
take tomorrow is when do you end up or

2035
01:33:58,909 --> 01:34:03,379
sorry when you end up doing two

2036
01:34:01,639 --> 01:34:06,829
different approaches we will talk about

2037
01:34:03,379 --> 01:34:09,559
it tomorrow but for today the workflow

2038
01:34:06,829 --> 01:34:11,659
is you perform the pre-processing define

2039
01:34:09,559 --> 01:34:13,609
the layers in CNN said the training

2040
01:34:11,659 --> 01:34:15,228
options train the networks and test and

2041
01:34:13,609 --> 01:34:17,649
deploy the Train networks same as a

2042
01:34:15,229 --> 01:34:20,389
neural network workflow access images

2043
01:34:17,649 --> 01:34:22,909
define the hidden layer define the

2044
01:34:20,389 --> 01:34:25,550
weights define of a few other things

2045
01:34:22,909 --> 01:34:27,829
train the network and then by validated

2046
01:34:25,550 --> 01:34:30,769
and then classify it is that correct and

2047
01:34:27,829 --> 01:34:33,799
then repeat these orange steps till it

2048
01:34:30,769 --> 01:34:37,879
reaches a desired level of accuracy is

2049
01:34:33,800 --> 01:34:40,550
that clear clear okay so I'm gonna be

2050
01:34:37,879 --> 01:34:41,209
stopping today at this point if that's

2051
01:34:40,550 --> 01:34:43,849
okay

2052
01:34:41,209 --> 01:34:45,409
and I'm happy to take a few questions I

2053
01:34:43,849 --> 01:34:47,809
know I was told that I have time until

2054
01:34:45,409 --> 01:34:48,169
4:00 so I wanted to spend around five

2055
01:34:47,809 --> 01:34:50,479
minutes

2056
01:34:48,169 --> 01:34:52,280
on questions and then what I would like

2057
01:34:50,479 --> 01:34:57,619
you to do as a homework and come

2058
01:34:52,280 --> 01:35:00,079
tomorrow if possible yes yeah okay while

2059
01:34:57,619 --> 01:35:02,179
I'm checking what I should give you as a

2060
01:35:00,079 --> 01:35:03,858
homework if you cannot ask me any

2061
01:35:02,179 --> 01:35:05,780
questions that would be very helpful

2062
01:35:03,859 --> 01:35:07,629
well yeah while you think of questions

2063
01:35:05,780 --> 01:35:10,188
what I wanted to talk to you about is

2064
01:35:07,629 --> 01:35:13,280
please Google what is known as deep

2065
01:35:10,189 --> 01:35:16,129
learning MATLAB on RAM it's a free

2066
01:35:13,280 --> 01:35:19,188
two-hour training online training anyone

2067
01:35:16,129 --> 01:35:21,949
can access okay what it will end up

2068
01:35:19,189 --> 01:35:23,119
showing you is what the convolutional

2069
01:35:21,949 --> 01:35:24,709
neural networks are how do you

2070
01:35:23,119 --> 01:35:26,329
pre-process image how do you use pre

2071
01:35:24,709 --> 01:35:28,489
trained networks you don't need matlab

2072
01:35:26,329 --> 01:35:31,518
installed on your computer you can

2073
01:35:28,489 --> 01:35:34,188
essentially end up like you know are

2074
01:35:31,519 --> 01:35:36,139
doing it in your own computer with where

2075
01:35:34,189 --> 01:35:39,260
there's no MATLAB installed and you can

2076
01:35:36,139 --> 01:35:42,879
say you know launch the course and I'll

2077
01:35:39,260 --> 01:35:47,510
just show you what the course looks like

2078
01:35:42,879 --> 01:35:50,419
again so let me um let me try to log in

2079
01:35:47,510 --> 01:35:54,619
to mine it's called the deep learning

2080
01:35:50,419 --> 01:35:55,639
on-ramp and MATLAB okay when you finish

2081
01:35:54,619 --> 01:35:56,449
this course of course you get a

2082
01:35:55,639 --> 01:35:56,840
certificate

2083
01:35:56,449 --> 01:35:58,489
I'm all

2084
01:35:56,840 --> 01:36:00,349
happy to issue certificates from math

2085
01:35:58,489 --> 01:36:02,598
works for people who attended this if

2086
01:36:00,349 --> 01:36:07,250
needed in addition to this what what I

2087
01:36:02,599 --> 01:36:08,570
can do is I can say click on you know

2088
01:36:07,250 --> 01:36:11,000
I'll just essentially take the first

2089
01:36:08,570 --> 01:36:12,559
example first example that's there here

2090
01:36:11,000 --> 01:36:15,530
it shows a green bar because I've

2091
01:36:12,560 --> 01:36:17,599
completed it it it runs a little bit of

2092
01:36:15,530 --> 01:36:20,570
video that's there and then I can say

2093
01:36:17,599 --> 01:36:23,540
next and then again the course overview

2094
01:36:20,570 --> 01:36:26,330
it gives me and next now it shows me an

2095
01:36:23,540 --> 01:36:28,760
example on how to identify a few objects

2096
01:36:26,330 --> 01:36:30,830
in some images so as it is by clearing

2097
01:36:28,760 --> 01:36:34,369
those who have completed 5% of the

2098
01:36:30,830 --> 01:36:36,170
course now it says you will be

2099
01:36:34,369 --> 01:36:38,989
performing this this is the same example

2100
01:36:36,170 --> 01:36:41,300
that you performed ok now what you can

2101
01:36:38,989 --> 01:36:45,530
do is it will say use the I am read

2102
01:36:41,300 --> 01:36:47,810
function to or you know read the file

2103
01:36:45,530 --> 01:36:54,679
name so all I need to go and do over

2104
01:36:47,810 --> 01:36:58,010
here is I equals I am read and I can say

2105
01:36:54,679 --> 01:37:00,110
you know I can I can use some what I

2106
01:36:58,010 --> 01:37:00,830
what do I have as a file name I will

2107
01:37:00,110 --> 01:37:04,130
save file

2108
01:37:00,830 --> 01:37:05,780
oh one dot jpg and what do I need to

2109
01:37:04,130 --> 01:37:10,340
store it into I need to store it in a

2110
01:37:05,780 --> 01:37:13,849
variable called I image one yeah and I

2111
01:37:10,340 --> 01:37:16,099
can say submit and once I've submitted

2112
01:37:13,849 --> 01:37:16,960
it it's gonna run it and it says it is

2113
01:37:16,099 --> 01:37:20,630
correct

2114
01:37:16,960 --> 01:37:24,260
ok next it will take me to the next task

2115
01:37:20,630 --> 01:37:26,810
what was my next task I need to view the

2116
01:37:24,260 --> 01:37:33,050
image yeah so all I need to do is I am

2117
01:37:26,810 --> 01:37:36,830
show off I am G 1 and I'll submit it

2118
01:37:33,050 --> 01:37:38,750
we'll run it again and it's saw it and

2119
01:37:36,830 --> 01:37:41,330
it is correct ok

2120
01:37:38,750 --> 01:37:46,090
then I go to the next task so will you

2121
01:37:41,330 --> 01:37:49,070
be able to do things section 1 and 2

2122
01:37:46,090 --> 01:37:51,020
before tomorrow it's very easy but it

2123
01:37:49,070 --> 01:37:53,179
would be great if you can do section 1

2124
01:37:51,020 --> 01:37:54,800
and 2 in the deep learning on-ramp

2125
01:37:53,179 --> 01:37:56,750
before tomorrow it would be very nice

2126
01:37:54,800 --> 01:38:01,099
and I'd like to show you how this

2127
01:37:56,750 --> 01:38:04,239
certificate will look like um let's see

2128
01:38:01,099 --> 01:38:04,239
if I'm able to

2129
01:38:05,260 --> 01:38:10,190
so I can say download the progress

2130
01:38:07,519 --> 01:38:13,940
report I actually will end up getting a

2131
01:38:10,190 --> 01:38:16,549
nice PDF which says you know I have

2132
01:38:13,940 --> 01:38:18,558
completed at 5% I've done introduction

2133
01:38:16,550 --> 01:38:20,479
100% but I'm not completed the others

2134
01:38:18,559 --> 01:38:25,610
it's good for you to show that you get

2135
01:38:20,479 --> 01:38:27,579
us a certificate like this okay so I'm

2136
01:38:25,610 --> 01:38:29,690
also happy to give you a certificate for

2137
01:38:27,579 --> 01:38:31,429
indicating the fact that you attended

2138
01:38:29,690 --> 01:38:33,018
our today's session and or tomorrow

2139
01:38:31,429 --> 01:38:36,559
session but this is something that's

2140
01:38:33,019 --> 01:38:39,440
generated automatically for you molls so

2141
01:38:36,559 --> 01:38:41,420
will you be able to do the two tasks two

2142
01:38:39,440 --> 01:38:43,820
chapters that is introduction and using

2143
01:38:41,420 --> 01:38:46,489
pre train networks before tomorrow yes

2144
01:38:43,820 --> 01:38:51,170
it would be helpful so go to matlab

2145
01:38:46,489 --> 01:38:55,249
on-ramp which just say google matlab

2146
01:38:51,170 --> 01:38:56,900
deep learning learning on-ramp you just

2147
01:38:55,249 --> 01:38:58,909
need to create a math works account and

2148
01:38:56,900 --> 01:39:04,429
you can click on that and it will help

2149
01:38:58,909 --> 01:39:06,739
you launch it's a free course okay okay

2150
01:39:04,429 --> 01:39:08,630
so I took more time than that I promise

2151
01:39:06,739 --> 01:39:10,459
I said I would give you some time for

2152
01:39:08,630 --> 01:39:12,229
questions any questions at this point of

2153
01:39:10,460 --> 01:39:14,220
time or anything that you would like me

2154
01:39:12,229 --> 01:39:27,100
to cover tomorrow

2155
01:39:14,220 --> 01:39:27,100
[Music]