1

00:00:01.01 --> 00:00:03.05

- Before we finish up with all of our discussions



2

00:00:03.05 --> 00:00:04.06

of S3 in general,



3

00:00:04.06 --> 00:00:07.01

I want to talk about some enhanced features,



4

00:00:07.01 --> 00:00:10.03

of the S3 solution within AWS.



5

00:00:10.03 --> 00:00:11.06

These enhanced features give you



6

00:00:11.06 --> 00:00:13.03

the ability to do things like,



7

00:00:13.03 --> 00:00:16.00

locked down files, so they can't be modified,



8

00:00:16.00 --> 00:00:17.06

even though people can view them.



9

00:00:17.06 --> 00:00:20.02

The ability to perform automated operations



10

00:00:20.02 --> 00:00:22.08

against objects that are stored in S3 buckets,



11

00:00:22.08 --> 00:00:24.05

and, even the ability



12

00:00:24.05 --> 00:00:27.09

to implement such things as,



13

00:00:27.09 --> 00:00:30.03

automatic intelligent adjustment



14

00:00:30.03 --> 00:00:32.07

of tiering or a class of storage.



15

00:00:32.07 --> 00:00:34.04

So that you can save money in the end.



16

00:00:34.04 --> 00:00:37.00

So these are all enhanced features of S3,



17

00:00:37.00 --> 00:00:38.08

that AWS has added over the years



18

00:00:38.08 --> 00:00:39.08

and we want to take a look at them now.



19

00:00:39.08 --> 00:00:41.08

The first one, we're going to look at,



20

00:00:41.08 --> 00:00:44.09

is the ability to do intelligent tiering.



21

00:00:44.09 --> 00:00:48.07

Now if we go into the AWS Management Console,



22

00:00:48.07 --> 00:00:53.01

and then choose S3,



23

00:00:53.01 --> 00:00:54.07

you'll see if you go into any bucket



24

00:00:54.07 --> 00:00:56.09

where you can do this intelligent tiering.



25

00:00:56.09 --> 00:00:59.05

So we'll open up one of the buckets that we have,



26

00:00:59.05 --> 00:01:00.08

and notice that,



27

00:01:00.08 --> 00:01:03.07

when I go into the management of this bucket,



28

00:01:03.07 --> 00:01:05.01

you come to this thing called,



29

00:01:05.01 --> 00:01:07.05

lifecycle rules by default.



30

00:01:07.05 --> 00:01:10.00

This is where you get into intelligent tiering.



31

00:01:10.00 --> 00:01:11.09

So you use lifecycle rules,



32

00:01:11.09 --> 00:01:15.00

in order to implement intelligent tiering.



33

00:01:15.00 --> 00:01:16.08

So you see this concept intelligent tiering,



34

00:01:16.08 --> 00:01:17.09

you might think you're going to come in here,



35

00:01:17.09 --> 00:01:20.00

you're going to find something called that,



36

00:01:20.00 --> 00:01:22.05

but it's actually the lifecycle management.



37

00:01:22.05 --> 00:01:25.01

And with this, you will notice



38

00:01:25.01 --> 00:01:27.00

that you can get automated transition,



39

00:01:27.00 --> 00:01:28.04

to tiered storage.



40

00:01:28.04 --> 00:01:32.02

Which is effectively what intelligent tiering promises us.



41

00:01:32.02 --> 00:01:34.07

So if we simply click on get started,



42

00:01:34.07 --> 00:01:36.08

then you can see that you can create a rule,



43

00:01:36.08 --> 00:01:37.09

so if I say



44

00:01:37.09 --> 00:01:42.08

my intelligent tier,



45

00:01:42.08 --> 00:01:44.08

and we could give it a prefix as well if we want to,



46

00:01:44.08 --> 00:01:46.07

we'll click on Next,



47

00:01:46.07 --> 00:01:49.07

and then you choose your storage class transition.



48

00:01:49.07 --> 00:01:54.00

That is to say, what are you wanting to transition



49

00:01:54.00 --> 00:01:55.00

into a different tier?



50

00:01:55.00 --> 00:01:57.05

Is it current versions or previous versions?



51

00:01:57.05 --> 00:01:59.04

Or both, for that matter.



52

00:01:59.04 --> 00:02:01.08

So we'll just go ahead and say the current version,



53

00:02:01.08 --> 00:02:05.00

and then we add a transition.



54

00:02:05.00 --> 00:02:09.04

We'll say that we want to transition to Glacier,



55

00:02:09.04 --> 00:02:13.01

after, 90 days of creation.



56

00:02:13.01 --> 00:02:14.09

So whatever we put in this bucket,



57

00:02:14.09 --> 00:02:17.09

it's assumed that for the first 90 days it's used,



58

00:02:17.09 --> 00:02:19.01

and then after 90 days,



59

00:02:19.01 --> 00:02:20.06

it's just rarely rarely used.



60

00:02:20.06 --> 00:02:21.08

So we can get by with putting it



61

00:02:21.08 --> 00:02:24.00

in a lower cost storage tier.



62

00:02:24.00 --> 00:02:26.05

Now notice it says transitioning small objects



63

00:02:26.05 --> 00:02:29.00

to Glacier or Glacier Deep Archive,



64

00:02:29.00 --> 00:02:30.06

will increase costs,



65

00:02:30.06 --> 00:02:33.04

because there's a transaction cost,



66

00:02:33.04 --> 00:02:36.00

and a cost based on the size of the file.



67

00:02:36.00 --> 00:02:37.08

So when it's a small object,



68

00:02:37.08 --> 00:02:39.06

every time you transition it,



69

00:02:39.06 --> 00:02:42.08

you're actually incurring the transaction cost.



70

00:02:42.08 --> 00:02:44.07

So that's something to keep in mind.



71

00:02:44.07 --> 00:02:45.07

You'll want to watch out for that,



72

00:02:45.07 --> 00:02:48.06

you can click learn more to learn more about it.



73

00:02:48.06 --> 00:02:50.09

We can just go and acknowledge that we understand that



74

00:02:50.09 --> 00:02:52.03

and click on Next.



75

00:02:52.03 --> 00:02:54.07

And then you configure expiration if you wish.



76

00:02:54.07 --> 00:02:56.07

Expiration basically says



77

00:02:56.07 --> 00:02:59.05

that the file is going to expire after some point in time.



78

00:02:59.05 --> 00:03:00.03

We really don't care



79

00:03:00.03 --> 00:03:02.03

about how we're storing it at that point.



80

00:03:02.03 --> 00:03:03.09

We'll go ahead and click Next and not do that,



81

00:03:03.09 --> 00:03:04.08

because that's not really part



82

00:03:04.08 --> 00:03:07.01

of intelligent tier transition.



83

00:03:07.01 --> 00:03:08.08

And then we've got a review page



84

00:03:08.08 --> 00:03:10.01

of what we're configuring.



85

00:03:10.01 --> 00:03:12.07

We click on Save,



86

00:03:12.07 --> 00:03:14.08

and I have just implemented intelligent tiering.



87

00:03:14.08 --> 00:03:17.01

So it's really pretty straightforward to do that,



88

00:03:17.01 --> 00:03:18.07

within an S3 bucket.



89

00:03:18.07 --> 00:03:20.02

Now the next thing I want to talk to you about,



90

00:03:20.02 --> 00:03:23.02

is the concept of Object locking.



91

00:03:23.02 --> 00:03:24.09

So this allows you to do



92

00:03:24.09 --> 00:03:27.08

what's called write-once-read-many



93

00:03:27.08 --> 00:03:31.07

or WORM, as a file storage method.



94

00:03:31.07 --> 00:03:32.08

So it means you write it,



95

00:03:32.08 --> 00:03:35.04

you store it in the storage location,



96

00:03:35.04 --> 00:03:37.05

and that's the only time it's modified



97

00:03:37.05 --> 00:03:38.09

at the time of creation.



98

00:03:38.09 --> 00:03:40.06

Then you read it many times,



99

00:03:40.06 --> 00:03:42.04

write-once-read-many,



100

00:03:42.04 --> 00:03:43.06

WORM, okay?



101

00:03:43.06 --> 00:03:45.07

So to do that, within an S3 bucket,



102

00:03:45.07 --> 00:03:47.09

you need to be able to do Object locking.



103

00:03:47.09 --> 00:03:50.01

By default, you can only enable,



104

00:03:50.01 --> 00:03:52.00

Object locking for an S3 bucket



105

00:03:52.00 --> 00:03:53.07

during the creation of the bucket.



106

00:03:53.07 --> 00:03:54.05

If you don't do it then,



107

00:03:54.05 --> 00:03:56.03

you have to contact customer service,



108

00:03:56.03 --> 00:03:57.07

they've got to go in behind the scenes



109

00:03:57.07 --> 00:03:58.09

and set it up for you,



110

00:03:58.09 --> 00:04:01.01

so make sure you think about Object locking



111

00:04:01.01 --> 00:04:02.08

before you create the bucket.



112

00:04:02.08 --> 00:04:05.02

Let's take a look at it.



113

00:04:05.02 --> 00:04:09.06

So we will go back to our Amazon S3 console,



114

00:04:09.06 --> 00:04:11.02

click on Create bucket.



115

00:04:11.02 --> 00:04:12.05

I'll call it,



116

00:04:12.05 --> 00:04:19.00

Object locking example,



117

00:04:19.00 --> 00:04:21.05

Dash 1009.



118

00:04:21.05 --> 00:04:22.07

Just to make sure remember,



119

00:04:22.07 --> 00:04:23.08

it's a unique name,



120

00:04:23.08 --> 00:04:26.02

all S3 buckets need a unique name,



121

00:04:26.02 --> 00:04:27.07

the US East region is fine



122

00:04:27.07 --> 00:04:30.01

and we're not going to copy settings from any other bucket.



123

00:04:30.01 --> 00:04:31.06

I'll click on Next,



124

00:04:31.06 --> 00:04:34.03

and the key here, is when you scroll down,



125

00:04:34.03 --> 00:04:37.00

expand Advanced Settings.



126

00:04:37.00 --> 00:04:39.08

And notice it says here, Object Lock.



127

00:04:39.08 --> 00:04:43.05

So this is a feature that was later added to S3 buckets,



128

00:04:43.05 --> 00:04:45.09

and I can say permanently allow objects



129

00:04:45.09 --> 00:04:48.02

in this bucket to be locked.



130

00:04:48.02 --> 00:04:50.09

Object Lock requires bucket versioning enabled as well



131

00:04:50.09 --> 00:04:51.08

so we need to come up here,



132

00:04:51.08 --> 00:04:54.06

and make sure we're going to keep versioning.



133

00:04:54.06 --> 00:04:56.05

And we'll check it,



134

00:04:56.05 --> 00:04:59.04

and it does permanently enable object Lock.



135

00:04:59.04 --> 00:05:02.02

So, only if you want to prevent objects from being deleted,



136

00:05:02.02 --> 00:05:03.06

you do this well that's our goal, alright,



137

00:05:03.06 --> 00:05:05.06

so we're going to go ahead and do this.



138

00:05:05.06 --> 00:05:08.00

And then we'll click on Next.



139

00:05:08.00 --> 00:05:11.01

We can set permissions,



140

00:05:11.01 --> 00:05:13.05

and other settings are reviewed here.



141

00:05:13.05 --> 00:05:15.02

Now we're not going to worry about those other settings.



142

00:05:15.02 --> 00:05:17.02

We've talked about those elsewhere with S3,



143

00:05:17.02 --> 00:05:20.01

So I'll click on Create bucket.



144

00:05:20.01 --> 00:05:22.08

I now have my Object locking example bucket.



145

00:05:22.08 --> 00:05:24.00

If I go into that bucket,



146

00:05:24.00 --> 00:05:30.00

I can upload a file,



147

00:05:30.00 --> 00:05:34.06

And we'll click on Next.



148

00:05:34.06 --> 00:05:39.04

And we can manage the permissions for this file as well.



149

00:05:39.04 --> 00:05:41.01

We can set properties for the file,



150

00:05:41.01 --> 00:05:44.03

so storage properties, metadata properties,



151

00:05:44.03 --> 00:05:47.02

all of this can be configured here.



152

00:05:47.02 --> 00:05:49.05

And then we click on Upload.



153

00:05:49.05 --> 00:05:52.00

Now when we click on the file,



154

00:05:52.00 --> 00:05:55.08

you can see, that the file has an owner,



155

00:05:55.08 --> 00:05:58.02

and the object URL is set,



156

00:05:58.02 --> 00:06:00.05

if we go to properties of the file,



157

00:06:00.05 --> 00:06:02.08

down here it says object lock.



158

00:06:02.08 --> 00:06:06.06

When we click it, notice, I can now enable it.



159

00:06:06.06 --> 00:06:08.06

So. here's the key.



160

00:06:08.06 --> 00:06:12.06

If you do not enable Object locking on the S3 bucket,



161

00:06:12.06 --> 00:06:15.00

when you go into an object in that bucket,



162

00:06:15.00 --> 00:06:16.08

and you go to the properties like we have here,



163

00:06:16.08 --> 00:06:20.01

you won't see the ability to actually lock the object.



164

00:06:20.01 --> 00:06:22.01

So this screen is only available now



165

00:06:22.01 --> 00:06:24.02

because I've locked it at the bucket level.



166

00:06:24.02 --> 00:06:26.04

Okay, one more thing to talk about.



167

00:06:26.04 --> 00:06:28.04

And that thing is going to be



168

00:06:28.04 --> 00:06:30.06

batch processing for S3 buckets.



169

00:06:30.06 --> 00:06:32.00

To make things happen automatically,



170

00:06:32.00 --> 00:06:33.00

wouldn't that'd be nice?



171

00:06:33.00 --> 00:06:34.07

Let me quickly show you how it works.



172

00:06:34.07 --> 00:06:37.00

So here we are back at our S3 bucket again,



173

00:06:37.00 --> 00:06:37.08

and what we're going to do,



174

00:06:37.08 --> 00:06:39.03

is look at this section over here



175

00:06:39.03 --> 00:06:42.01

that says, batch operations.



176

00:06:42.01 --> 00:06:43.07

When we click on that,



177

00:06:43.07 --> 00:06:46.03

we can see that there are no jobs



178

00:06:46.03 --> 00:06:47.03

in this case for this one,



179

00:06:47.03 --> 00:06:50.03

but we can click on create job and get started.



180

00:06:50.03 --> 00:06:54.07

And what you'll do, is you'll create a job,



181

00:06:54.07 --> 00:06:55.08

based on a manifest.



182

00:06:55.08 --> 00:06:59.02

So a manifest is just a collection of all the objects



183

00:06:59.02 --> 00:07:00.03

that are in the S3 bucket



184

00:07:00.03 --> 00:07:01.02

a listing of them



185

00:07:01.02 --> 00:07:04.05

either in CSV format or in JSON format.



186

00:07:04.05 --> 00:07:06.03

So that's all a manifest is.



187

00:07:06.03 --> 00:07:07.07

So you'll have your manifest,



188

00:07:07.07 --> 00:07:09.07

and then you use that manifest,



189

00:07:09.07 --> 00:07:13.03

to go against the list of objects in there to take action.



190

00:07:13.03 --> 00:07:14.06

So what actions could I take



191

00:07:14.06 --> 00:07:16.04

with these types of batch operations?



192

00:07:16.04 --> 00:07:19.06

Well, you could move the objects to a different S3 bucket,



193

00:07:19.06 --> 00:07:21.03

you could delete objects,



194

00:07:21.03 --> 00:07:23.09

you could change properties of objects,



195

00:07:23.09 --> 00:07:25.08

you could change objects to be stored



196

00:07:25.08 --> 00:07:27.07

in a different class or tier.



197

00:07:27.07 --> 00:07:29.01

So there are a lot of different things you could do



198

00:07:29.01 --> 00:07:31.06

with batch operations to really speed up,



199

00:07:31.06 --> 00:07:33.07

the management of your S3 buckets.



200

00:07:33.07 --> 00:07:34.07

So that gives you an idea



201

00:07:34.07 --> 00:07:36.04

of some of the enhanced features,



202

00:07:36.04 --> 00:07:38.00

that are available for S3 buckets,



203

00:07:38.00 --> 00:07:40.07

when you need those, more advanced management techniques



204

00:07:40.07 --> 00:07:42.05

that you might not have been aware of.