‫Hello everyone!
‫Welcome to the backend master class!
‫Continue with the background worker topic,
‫Today, I’ll show you why it is important
‫to add a bit of delay to your async task.
‫If you still remember, in lecture 56 of the ‫course,
‫we’ve implemented this CreateUser transaction
‫with a callback function: AfterCreate.
‫This function will be called after a new user ‫is created,
‫to send an async task for email verification ‫to Redis.
‫And here, you can see that we have a 10-second ‫delay,
‫This means that the task will only be
‫picked up by the worker 10 seconds after it ‫is created.
‫Now, to understand why this delay is important,
‫let’s see what will happen if I comment ‫out this line.
‫Then, let’s open the terminal and do some ‫tests.
‫First, let’s make sure both Postgres and ‫Redis servers are up and running.
‫OK, they are!
‫So we can start our web server.
‫Next, I’m gonna open Postman,
‫And send a request to create a new user: alice10
‫Alright, the request is successful.
‫Now, let’s look at the web server’s logs.
‫It received a request, and enqueued a verify ‫email task
‫And the task is processed immediately.
‫There was no problem at all.
‫So does that mean we don’t need to delay ‫the task?
‫Well, not really!
‫It is OK this time, but there might be issues ‫another time.
‫Let’s take a look at the implementation ‫of the CreateUser transaction
‫Here, the AfterCreate function is called at ‫the end of the transaction,
‫But since it’s written inside the execTx ‫function,
‫This function is actually run before the transaction ‫is committed.
‫As you can see here,
‫The function is called,
‫and if it returns an error, the transaction ‫will be rolled back
‫Otherwise, it will be committed to the database.
‫So what if there’s high traffic on the db
‫that makes this commit statement take longer ‫time to complete?
‫For example, we can add a time.Sleep 2 seconds ‫here to simulate that scenario.
‫Let’s see what will happen in this case.
‫I’m gonna restart the web server.
‫And go back to postman to send a create-new-user ‫request: alice 11.
‫It will take about 2 seconds to complete because ‫of the sleep statement.
‫But now, if we look at the logs,
‫We will see that the task has been enqueued,
‫And it has already been processed,
‫But failed because the user couldn’t be ‫found in the database.
‫Do you know why?
‫Well, that’s because it took the transaction ‫2 seconds to be committed,
‫But the send email task was enqueued without ‫delays,
‫So it was processed by the worker immediately.
‫And because of this,
‫when the worker tried to look for the user ‫in the database,
‫It could not find the record, since the transaction ‫was not committed yet.
‫Also, in the code, if the user couldn’t ‫be found,
‫we return the asynq.SkipRetry error,
‫so the task won’t be retried in this case.
‫And it’s a big issue because user will never ‫receive the verification email.
‫So now you know the importance of this 10 ‫seconds delay.
‫It will make room for the db transaction to ‫be fully committed
‫before the async task is picked up by the ‫worker.
‫Alright, let’s restart the server and give ‫it a try!
‫I’m gonna change the username and email ‫to alice12
‫And resend the create user request.
‫It will take 2 seconds to finish, just like ‫before.
‫But this time, in the server log, we don’t ‫see any errors
‫That’s because the task has been delayed ‫for 10 seconds.
‫So it’s only been enqueued for now.
‫And will only be processed by the worker after ‫10 seconds.
‫At that time, the create user transaction ‫has already been committed,
‫So the task will be successfully processed, ‫without any issues.
‫And that’s how a bit of delay can help us ‫easily solve this problem.
‫But make sure you set the right amount of ‫delay for your task,
‫Because for example, here if we just delay ‫the task for 1 second,
‫Which is lower than the 2 second that the ‫transaction takes to commit,
‫Then the problem will still occur.
‫Normally, I would say 10 seconds should be ‫quite enough
‫for the transaction to commit,
‫But we never know for sure,
‫In some rare cases, anything could happen.
‫So, one way to deal with that, is to always ‫allow the task to retry.
‫We can simply comment out this skip retry ‫statement.
‫So even if the user is not found,
‫the task will be retried again at another ‫time,
‫And hopefully at that time, it will be successful,
‫when the transaction has been committed.
‫Of course, if the transaction fail to commit,
‫then the user will never be created in the ‫DB,
‫So the task will always fail.
‫But it’s not a big deal, because it will ‫run out of retries eventually,
‫Since we have set the maximum number of retries ‫to 10 times in this case.
‫Alright, let’s restart the server and do ‫1 more test.
‫In postman, I’m gonna change the username ‫and email to alice13,
‫And resend the request!
‫The request is successful after 2 seconds.
‫But this time, in the log,
‫the task has been processed 1 time, and it ‫failed.
‫That’s because we set the delay to just ‫1 second.
‫But note that, we have removed the SkipRetry ‫statement,
‫So this task will be retry again later.
‫OK, as you can see,
‫After a while, the task is retried,
‫And it is successfully processed.
‫Exactly as we wanted.
‫Before we finish, I’m gonna revert this ‫delay time to 10 seconds,
‫And remove this time.Sleep statement in the ‫execTx function.
‫I’ll keep this SkipRetry here, but commented ‫out, as a reference.
‫And that wraps up this lecture
‫about the importance of delay when processing ‫async tasks.
‫I hope it was interesting and useful for you.
‫Thanks a lot for watching,
‫Happy learning, and see you in the next lecture!