1
00:00:04,839 --> 00:00:08,359
welcome to section seven acha streams in

2
00:00:08,359 --> 00:00:10,849
the previous section we introduced you

3
00:00:10,849 --> 00:00:14,929
on how to test actors in this section we

4
00:00:14,929 --> 00:00:16,730
will introduce the new module in Acker

5
00:00:16,730 --> 00:00:18,019
that is

6
00:00:18,019 --> 00:00:21,140
akka streams at the moment of recording

7
00:00:21,140 --> 00:00:24,199
streams are in version 1.0 experimental

8
00:00:24,199 --> 00:00:27,018
and slight API changes may still happen

9
00:00:27,018 --> 00:00:29,570
however it's stable enough to be used

10
00:00:29,570 --> 00:00:32,840
already so let's take a look here we

11
00:00:32,840 --> 00:00:34,789
will take a quick look at intro to akka

12
00:00:34,789 --> 00:00:37,609
stream and its terminologies then we

13
00:00:37,609 --> 00:00:39,439
will implement our stream application

14
00:00:39,439 --> 00:00:42,140
reactive tweets and test the reactive

15
00:00:42,140 --> 00:00:44,420
tweets application we will then work

16
00:00:44,420 --> 00:00:48,738
with graphs and with stream IO let's

17
00:00:48,738 --> 00:00:51,070
start our trip in the stream world in

18
00:00:51,070 --> 00:00:53,988
the first video we will take a look at

19
00:00:53,988 --> 00:00:56,268
the problems of fast data and streaming

20
00:00:56,268 --> 00:00:58,488
processing world and while we should be

21
00:00:58,488 --> 00:01:01,399
using occur streams next we will

22
00:01:01,399 --> 00:01:03,588
introduce the basic terminology in akka

23
00:01:03,588 --> 00:01:05,868
streams and implement our first stream

24
00:01:05,868 --> 00:01:10,219
up let's start then big data has been

25
00:01:10,219 --> 00:01:12,140
around for quite a while and it's

26
00:01:12,140 --> 00:01:14,810
usually batch processing in the current

27
00:01:14,810 --> 00:01:16,400
world though we're starting to talk

28
00:01:16,400 --> 00:01:18,950
about fast data which means we want to

29
00:01:18,950 --> 00:01:21,140
process the data right away when it hits

30
00:01:21,140 --> 00:01:23,659
our service for example akka streams are

31
00:01:23,659 --> 00:01:25,700
a library which allows implementing

32
00:01:25,700 --> 00:01:27,260
either style of application and

33
00:01:27,260 --> 00:01:28,849
guarantees that it will execute

34
00:01:28,849 --> 00:01:31,579
unbounded buffer sizes will not cause

35
00:01:31,579 --> 00:01:33,829
out of memory errors the two main

36
00:01:33,829 --> 00:01:35,689
problems streaming data solutions need

37
00:01:35,689 --> 00:01:38,359
to tackle are blocking and back pressure

38
00:01:38,359 --> 00:01:41,900
to survive high load scenarios blocking

39
00:01:41,900 --> 00:01:44,420
can happen in a pull based system this

40
00:01:44,420 --> 00:01:46,459
means a consumer pulls data from the

41
00:01:46,459 --> 00:01:49,010
producer blocking takes place when there

42
00:01:49,010 --> 00:01:51,829
is no data to pull this is safe for the

43
00:01:51,829 --> 00:01:53,989
consumer it will not run out of memory

44
00:01:53,989 --> 00:01:57,228
however it is also very slow to build

45
00:01:57,228 --> 00:02:00,769
systems based purely on pulling data the

46
00:02:00,769 --> 00:02:03,579
other problem is back pressuring to fast

47
00:02:03,579 --> 00:02:05,750
producers of data when working with

48
00:02:05,750 --> 00:02:08,959
pushing data in pull systems it's not as

49
00:02:08,959 --> 00:02:11,959
visibly needed however they are slow and

50
00:02:11,959 --> 00:02:15,229
we want our systems to be fast the need

51
00:02:15,229 --> 00:02:16,580
for back pressure

52
00:02:16,580 --> 00:02:18,349
when the producer wants to create more

53
00:02:18,349 --> 00:02:20,689
data than the consumer can handle which

54
00:02:20,689 --> 00:02:23,629
can cause the consumer to crash let's

55
00:02:23,629 --> 00:02:25,550
take a real-life example to understand

56
00:02:25,550 --> 00:02:27,680
these problems let's imagine we have a

57
00:02:27,680 --> 00:02:30,020
source of water and a sink which we want

58
00:02:30,020 --> 00:02:31,969
to fill and we have pipes between them

59
00:02:31,969 --> 00:02:34,460
if we decided to put a pump at the end

60
00:02:34,460 --> 00:02:36,860
of the pipes to pull the water the pump

61
00:02:36,860 --> 00:02:38,900
will still work even if there is no

62
00:02:38,900 --> 00:02:41,659
water on the source and if we move the

63
00:02:41,659 --> 00:02:43,550
pump to start off the pipes to push the

64
00:02:43,550 --> 00:02:46,370
water the pump will still work even if

65
00:02:46,370 --> 00:02:48,199
the sink is filled and the water just

66
00:02:48,199 --> 00:02:51,409
overflows to fix our problems we need to

67
00:02:51,409 --> 00:02:53,539
put a pump on the source and cap on the

68
00:02:53,539 --> 00:02:53,960
sink

69
00:02:53,960 --> 00:02:56,000
this means the pump will work when a

70
00:02:56,000 --> 00:02:58,639
source contains water and when the sink

71
00:02:58,639 --> 00:03:00,919
fills the cap will create back pressure

72
00:03:00,919 --> 00:03:03,159
and trigger the pump to stop pumping

73
00:03:03,159 --> 00:03:05,990
this is exactly what akka streams do

74
00:03:05,990 --> 00:03:07,909
when it comes to solve the main problems

75
00:03:07,909 --> 00:03:10,219
on streaming data processing akka

76
00:03:10,219 --> 00:03:12,289
streams describe the solution as a

77
00:03:12,289 --> 00:03:14,840
synchronous dynamic push-pull back

78
00:03:14,840 --> 00:03:18,020
pressure now let's discuss the main

79
00:03:18,020 --> 00:03:21,019
parts of akka streams the main elements

80
00:03:21,019 --> 00:03:23,629
of akka streams our source and sink the

81
00:03:23,629 --> 00:03:25,519
sources where data begins to flow from

82
00:03:25,519 --> 00:03:28,639
it has the output port and no input port

83
00:03:28,639 --> 00:03:31,250
the source in our example may be a river

84
00:03:31,250 --> 00:03:34,250
in applications it may be a query from

85
00:03:34,250 --> 00:03:37,069
the database or HTTP requests requests

86
00:03:37,069 --> 00:03:39,889
bytes the sink is the end of the stream

87
00:03:39,889 --> 00:03:43,069
it has the input port and no output port

88
00:03:43,069 --> 00:03:45,650
in our example we were directing the

89
00:03:45,650 --> 00:03:47,300
flowing water that we have collected in

90
00:03:47,300 --> 00:03:50,449
a sink in real-world applications it may

91
00:03:50,449 --> 00:03:52,669
be the stored data in the database or

92
00:03:52,669 --> 00:03:55,340
data written on a file so we'd have a

93
00:03:55,340 --> 00:03:58,519
file sink as streams can have multiple

94
00:03:58,519 --> 00:04:00,680
processing stages between a source and

95
00:04:00,680 --> 00:04:02,659
the sink we also need to name this

96
00:04:02,659 --> 00:04:04,550
concept in akka

97
00:04:04,550 --> 00:04:07,069
it's called a flow and one can flow the

98
00:04:07,069 --> 00:04:10,280
data via a flow from source by a flow

99
00:04:10,280 --> 00:04:13,009
and finally to a sink if you want to

100
00:04:13,009 --> 00:04:14,330
make things more interesting on the

101
00:04:14,330 --> 00:04:16,519
stream you need to add a flow between

102
00:04:16,519 --> 00:04:19,160
the source and sink a flow can be used

103
00:04:19,160 --> 00:04:20,930
to apply transformations to the data

104
00:04:20,930 --> 00:04:23,329
coming out of a source before putting it

105
00:04:23,329 --> 00:04:26,120
into a sink when connecting the source

106
00:04:26,120 --> 00:04:28,639
to the sink we get a runnable flow a

107
00:04:28,639 --> 00:04:30,500
runnable flow is something we

108
00:04:30,500 --> 00:04:32,779
can start in other words it's a flow

109
00:04:32,779 --> 00:04:34,879
with the sink and a source attached a

110
00:04:34,879 --> 00:04:37,759
complete stream pipeline this is the

111
00:04:37,759 --> 00:04:39,769
most basic complete form you can make in

112
00:04:39,769 --> 00:04:42,649
akka streams this means you must connect

113
00:04:42,649 --> 00:04:44,360
your source with a sink to have a

114
00:04:44,360 --> 00:04:46,639
complete system and your data can flow

115
00:04:46,639 --> 00:04:49,399
through it now let's implement all this

116
00:04:49,399 --> 00:04:52,759
into a simple app akka streams provides

117
00:04:52,759 --> 00:04:55,459
this nice typesafe DSL to build streams

118
00:04:55,459 --> 00:04:58,160
from the actual execution of it is

119
00:04:58,160 --> 00:05:00,310
actually handled by actors underneath

120
00:05:00,310 --> 00:05:02,750
here's how we can build and run a simple

121
00:05:02,750 --> 00:05:05,149
stream we will talk about them in the

122
00:05:05,149 --> 00:05:08,379
next video first let's define our source

123
00:05:08,379 --> 00:05:11,000
let's say we will generate iterators

124
00:05:11,000 --> 00:05:15,250
from 1 to 100 next let's define a flow

125
00:05:15,250 --> 00:05:18,339
let's say we want to multiply it by 2

126
00:05:18,339 --> 00:05:21,889
then let's create our sync in our sync

127
00:05:21,889 --> 00:05:25,100
we will print the elements now let's run

128
00:05:25,100 --> 00:05:27,420
our app

129
00:05:27,420 --> 00:05:31,019
voila Austria map is working this is a

130
00:05:31,019 --> 00:05:33,240
simple application but we have used all

131
00:05:33,240 --> 00:05:34,855
the basic elements on it