- [Instructor] Stream processing is an alternative
to batch processing.
Now, stream processing analyzes data
in real time as it arrives
rather than waiting to process it in batches.
So it's really designed specifically to minimize latency.
So then unlike batch processing, it handles each data point
or maybe small micro batches individually
as those data points or micro batches
flow through the system.
Now this allows AI systems to react to events instantly
rather than waiting for, like,
a scheduled processing time to arrive.
Stream processing has really become essential
for applications where small delays could really adversely
impact business outcomes or our user experience.
Stream processing works with continuous data flows,
which don't have really well-defined start or endpoints.
Now, the technology that deliver
these really extremely fast response time
are often measured in milliseconds
rather than seconds, minutes, or hours.
Now, modern stream processing platforms help maintain
context and state across multiple data points,
so that allows us to do more sophisticated analysis
than we might otherwise be able to do.
The architectures of stream processing systems
are fundamentally event-driven,
which means the processing is triggered
by when data arrives rather than,
you know, when the clock hits a certain time
or when it's time to schedule a particular job.
Now these systems are built for continuous operation,
so they're really designed for application use cases
that have high availability requirements.
We see stream processing
in a number of different kinds of AI applications.
For example, recommendation engines use stream processing
for instantly updating suggestions based on,
say, your most recent clicks or your most recent views.
We also need low latency for fraud detection, so we can,
for example, flag suspicious transactions immediately
before the transaction is actually processed.
Now, streaming is also useful,
say, in manufacturing operations,
where you can use stream processing with equipment sensors
to help predict maintenance needs
before a failure actually occurs.
Now, self-driving cars and autonomous vehicles
are systems that depend on stream processing
because they really have very short periods of time
to make decisions.
And then maybe for slightly longer timeframes,
we have things like conversational AI applications.
You know, it's not as time sensitive
as, say, an autonomous vehicle,
but we still want to use stream processing
to analyze and respond to natural language inputs
without long delays.
Now, when we think about the technical components
that go into making up a stream processing system,
we want to have an event streaming platform.
And some of the most popular
are known as Kafka, Kinesis, and Pulsar.
Processing frameworks such as Spark Streaming,
Flink, and Samza are sort of the computational engines
that transform raw data into transform data.
We'll also want to make use of in-memory computing techniques
for maintaining state information.
And this is important because we don't want to slow down
operations by accessing data on a disc.
We need to have it available in very fast access formats.
Now sometimes we have to send messages between components.
In that case, we want to use low-latency messaging systems.
Those can ensure minimal delays
between the times we are able to do processing
in the different components.
And then we also want distributed
fault-tolerant processing clusters that can provide
all of the necessary basically computational power
that we're going to need for the amount of data
that's actually streaming in.
Now we also want to think about how data flows
through these systems,
and so we want to think about functionally,
what are the components that we need
from a data flow perspective?
Well, we need things that are essentially operated
as source connectors that capture events
from various source systems like APIs or IoT devices,
and then transform those data into standardized streams.
We also want to have stream processors
that handle the core work.
So these are the computational platforms.
And they do things like executing AI inference
and applying business logic to incoming data in real time.
We also may want to access online feature stores
to provide fast access to pre-computed features
that we need for model inference.
Sometimes we need state back ends,
and this is used for maintaining
contextual information between events.
And this is really important
when we have more sophisticated processing logic
that we need to implement.
And then finally, we need something
known as sink connectors.
These are the components that will basically
deliver the process results and predictions
to whatever downstream applications
are consuming the output of our stream.
Now, the benefits of stream processing include
the ability to deliver immediate response
to changing conditions,
and this rapid insight generation
can create competitive advantages
when we're dealing with time-sensitive operations
or applications.
It's also useful because models can continuously learn
and adapt from streaming data,
and this can help with improving accuracy
without long drawn out retraining cycles.
Also, we can improve customer experience
by improving real-time personalization
that responds to the current context
rather than just historical patterns
we might have analyzed in a batch mode.
Also, we can detect problems
and address them more efficiently,
oftentimes before they significantly adversely impact,
say, an operation or a customer experience.
When you're implementing stream processing,
you want to start by identifying specific use cases
where real-time processing can deliver,
like, clear business value.
We also want to implement comprehensive monitoring here.
I know I keep mentioning monitoring,
but that is just so fundamentally important.
If we're not able to track the health of our system,
especially with stream processing, we're not going to be able
to deliver the benefits of stream processing.
We also want to make sure that we design for failure.
Our architecture should be designed
to degrade gracefully under heavy loads
rather than completely falling over because we have,
like, an unusually or unexpected spike in traffic.
We also want to make sure we have robust error handling
using things like dead-letter queues
to catch and manage problematic data
without clogging up the entire pipeline.
And then finally, we want to make sure
that we're balancing processing complexity
and latency requirements.
The more complex our processing,
the longer we'll have to take
to actually perform that computation,
and that can drive up the latency.
So, again, we're dealing with a balancing act here
of how much complexity we want to implement
versus what's the latency that we can tolerate
before we have to have an answer
and deliver some data to the downstream systems.