Hello everyone.
Thank you very much to the few of you who are here.
The time is pretty bad.
But thank you very much for being here and thank you very much to the organization for accepting the job.
There were many, many jobs that were left out and with a lot of potential and they have chosen us and I hope you like it.
Both me and Jorge have spent a lot of time trying to make it as bearable as possible, to learn a little bit, even if it's the core of what augmented reality is, but we also want to show you a little bit of what happens when you put it together with security.
First of all, who are we?
My partner, Jorge Cuadrado.
As they have commented, he is from the BWA Next Technologies laboratory, just like me.
And we work in the security area, in the development of secure products for the company.
And we do a little bit of research.
And this is one example of what augmented reality can bring to the company.
In my case, what I was telling you...
It's going to go a little slow.
Miguel Hernández has already been here in the morning.
But well, I work with him in the security laboratory.
And we will be here since we both participate in the hackathon.
So if you have any questions, you can come and ask us.
What is the talk about?
You don't need technical knowledge of augmented reality.
We are going to go little by little.
We are going to introduce what augmented reality is.
How it has evolved.
What use cases may exist.
And then the core is going to be the security part.
How we can apply augmented reality security.
With cases that have already existed.
Jobs that have been done.
And the proposals we make on how to implement what would be safe augmented reality.
Or cases where augmented reality could help us improve security in our organizations.
Jorge will take all the practical part.
I will tell you a little bit about the theoretical part.
And if you have a computer and you can follow it, it would be much better.
Because there will be a lot of practical part.
It will be with the computer showing you the code.
Doing tests with the camera, etc.
But you can keep doing it if you have brought the computer.
And we show you some POCs that are all uploaded to GitHub.
You will see it.
And some proposals that we put for you to see how you can help us.
How we can improve or that other ideas have not occurred to us and we could carry them out.
I'm going to start with what would be augmented reality.
The theoretical part.
What is augmented reality?
Basically, it is all the technologies that are giving you more information than reality offers you.
You need a technological device that offers you that information inclusion.
It can be of any type and, believe it or not, it has been among us for a long time.
It appeared for the first time more than 100 years ago.
And as you can see, the device is quite large.
Because the components that existed before were also much larger.
And the capabilities they had were much more limited.
But the first one already appeared in 1901.
What they did was, from a camera, they introduced me more information with images than you were seeing to improve.
In what would be the accessibility of that information.
It is used in many areas, as we will see.
How has this evolved?
As I said years ago, devices are much larger, much rougher.
They were more about education, universities.
It was not so easy for someone to have it at home.
And they were used a lot, above all, in medicine or in much more university research.
Not in everyday life.
From the 80s to the 90s, it was improving in what would be the size, above all.
The components were smaller and they were able to generate these glasses much easier.
All you need is to superimpose information.
But they needed a very large processing power and it was complicated for those devices to be really usable.
From the 2000s, very interesting applications began to be seen.
For example, those of you who play classic aviation video games.
The typical camera that showed you where the horizon was, the height, the latitude.
That seems silly, but it was a big leap since it is important to know where the ground is.
But until you realize these things, before, how should they do it?
Or the glasses began to be smaller.
And all this issue of taking it to the general public is where the most money has been invested.
The biggest powers, like Google, are the ones who are making the HoloLens, much more ergonomic glasses.
But basically what they are trying to do is not to bother you as little as possible, and you can see that information added to improve visibility.
And companies also appeared that saw it very interesting.
For example, IKEA, when they have a problem with refunds.
Whenever you buy something, if you don't like how it looks, you have to return it.
And that causes a cost.
IKEA says, I offer you the possibility of putting it in your house with augmented reality, and you avoid that purchase and return.
You can see how it will look, reducing the cost of refunds a lot.
It is a clear case of this augmented reality, without any security problem or anything.
This would be in general.
And from this, how did it really explode?
When did people start talking about augmented reality?
It was with the case of Pokémon GO.
The video is a little long, and we will cut it in half.
The video is from 2015, which seems to be yesterday.
Almost everyone knows the world of Pokémon GO.
It has evolved, there have been more versions.
But they achieved something important, which was to get you out of your house.
They simply used the GPS position to apply augmented reality within the GPS position.
Basically , your screen appeared Pokémon in the random part of where it was.
I don't know if anyone has proposed the privacy problem that this had.
I think I have been quite clear with this example.
The privacy problem that this had is that basically Google knew where you were at all times, just like with the GPS.
But here more massively.
We will see in that case how the issue of privacy can be addressed.
This was from 2015, from Niantic.
They made a joke for the anniversary, they proposed it, and suddenly they saw that this had a future.
And from there, a lot of companies started to see that people use it, it's easy, I can have it on my cell phone.
And this is the case that came out a little while ago, which we will also see.
I think it's great because you can collaborate.
Fill the backyard with muddy pigs.
Host the perfect pixel party.
And craft some compact cuteness.
Or build something a bit more clever and metallic.
And they can share and play like they did with Pokemon Go.
You don't have to leave home anymore, you have your house and do the same.
Do your farm, do your caves, etc.
And this came out a little while ago.
Discover new dimensions to your creativity with Minecraft Earth.
It's a shame because it's a waste of energy.
But we can still play.
So you can see how it has evolved.
The videogame is still there as something important.
The internet has increased, the app is still available in other areas.
But we can see that videogames are getting better.
It is also used, it appears in the media, to give more impact and visibility to events.
The increased network allows you to see a dragon in the finals of League of Legends in Korea.
And it appears in a stadium and there you can see a dragon.
Or in the stadium of Estudiantes de la Plata where they put a fire lion.
So this really causes a lot of impression in the networks and you can reach more people.
Companies are starting to use it for their 3.0 business processes.
And it's starting to appear a little more than just Pokémon Go.
Or when Minecraft appears in the news, it will be everywhere.
What other examples do we have?
One thing is to know how it works in general.
But the technology behind it is almost always JavaScript.
And we are going to use 3JS.
Jorge will explain it in detail.
But there are others so you don't close us to a technology.
Here you can choose the one you want.
You can use Google's.
AR is also used with 3JS.
There are toolkits.
One dedicated to Android.
They are open repositories.
There is no problem in that you can develop in an open source and collaborative way.
But we will focus on the first of all.
Otherwise it is unbearable.
What applications?
We have seen video games.
We have seen that it is for social impact.
But there are cases that I had not realized that in my day to day I am seeing them.
When parking, the typical lines that appear have been something new with the rear cameras of the cars.
And it really is augmented reality.
It is giving you more information when parking.
With those lines to see how you have to turn, how much you have to turn.
The sound part also adds that information to what you are really seeing.
It is very interesting.
And maybe we had not realized that it was being applied here.
As in the cases to do time issues.
Because people see the TV less and less.
They try to make it more attractive with the programs of time.
Putting tornadoes.
In sports applications.
In cycling it already seemed that to know the distances they put the cyclists, they could move them.
Football alignments.
Many times they asked me if when you put the TV you have a game.
The advertising that appeared.
Did it really appear in the field or were they putting it on top of it?
People even doubt if what is really happening is augmented reality.
Or they do it really well.
There are also companies.
They have seen that there is a business.
They have started to inject money.
And they say, hey, maybe this is a market niche that needs to be explored.
They start giving courses on augmented reality.
You do not know what to do.
You like development.
You can come here.
Companies are starting to appear.
They offer projects for home automation.
To assemble your house.
The furniture, etc.
And before you buy them, you can see them in your house.
Project consultancy.
As a service.
Or game themes, etc.
It seems to me that there are a lot of industries.
To avoid the protection of labor risks.
So that the person does not have to get their hands in there.
Etc.
It seems that the world is also moving towards these issues.
There is also a very important question.
Where does the line between augmented reality and virtual reality come from?
It is clear that the further away from reality you are, the more you are going to virtual reality.
The problem with this is that it has to isolate you from reality.
It is a process in which the human being has dizziness problems.
Because he does not quite understand where he is.
You can lose your orientation.
And you also need a very expensive equipment.
When you get used to augmented reality, you do not need a big camera.
You do not need very powerful glasses.
You can use HoloLens.
They are the most expensive.
But you can even do it with your mobile phone.
And instead of these systems, you need dedicated hardware.
And it is quite expensive.
The most novel thing I've seen, which was recently released.
I do not know if you know it.
It's this video game.
You put on some gloves and grab each of the phalanges to grab the fingers.
Capture all the movement.
And basically it allows you to reload, pick up weapons.
It's amazing.
But of course, I do not know how the body will behave when you really have all the movement there.
You have to be coordinated with the head, where you look, etc.
How is it going to behave?
But this is the most powerful thing there is in virtual reality.
I think that when it comes out it will be quite interesting.
The thing is that the cost of this is about 800 euros.
600 euros is not so easy to have in your house for the few games there are.
There are also cases of use that not only apply to companies and that can be beneficial for everyone.
In awareness campaigns, especially blood donation, it is usually used.
There is even a project in UNICEF of augmented reality.
To bring education to countries where it is very difficult to carry material.
And maybe with these systems it allows you to help those who have it more complicated with this technology.
They do not need to touch things.
You can help them to learn, to develop in this and to improve their standard of living.
And what I say, for companies it usually comes in handy.
It gives much more impact than seeing it.
And what all this would be is the introduction.
I did not want to spoil myself any more.
The grace of this is not how I can apply augmented reality and security.
In the case of Pokémon Go, they give you an idea that you are exposing yourself.
Every time you catch a Pokémon, it is registered.
Then you are at that point.
We see that there are some problems by default in augmented reality.
And we will see it.
First, there is a clear case that has nothing to do with augmented reality.
They are the copies of these applications.
When something explodes, it is very famous.
Almost always there are attackers who are dedicated to cloning these applications to try to deceive the victim.
Steal the data, etc.
And there are cases where thousands of similar applications came out with Pokémon Go.
People downloaded them out of ignorance that they were copies.
And you could charge your phone.
Or with the issue of malware, etc.
You have to be a little careful with what you are downloading.
Always see that it is official, etc.
But we see that this has nothing to do with augmented reality.
But it is true that applications have the same problems as any other application.
In the case of security, there are already works that are quite interesting on this topic.
There is a part that is before the information reaches the augmented reality application.
There are models of entrances of confidence.
What they tell you is that in the first case, everything that registers the camera will be saved.
And there may be privacy problems with what you are seeing.
If what is sent goes not only to the application, but to a server, all your activity can be monitored.
And what they call live logging.
Just as you log in with the keyboard, here what they tell you directly is live logging, which is everything you are doing that has been registered.
They try to see, especially with privacy policies, that it is not all the time, that it depends on what you are recording.
Or, for example, if you are trying to see an AR, but suddenly you put a document or a screen that should not be what you should be reading, you should not be recording that recording.
Because you may have information leaks.
And in the case of the Whatapps, something similar happens.
They go to privacy models based on whether there is a man-in-the-middle within it.
The information they are putting in is different.
If you are not processing the input well, they are trying to trick me.
They have several modules.
You have the links there.
They are quite interesting jobs.
They are from a couple of years ago.
One year, I think, is the last one.
And it is a good base to grow.
In what would be the previous part to which the application arrives.
But when it comes out of the application, we can also have problems.
The clearest one here, and they usually give you as an example, I am with the car and there are applications that tell you, hey, you are too close to a car.
Please, skip an alert or it will notify you.
The fun of this is that you can get in the middle and say, no, no, do not notify him, not at that distance, but much less.
So that the car does not get to see the warning.
It thinks, well, when it notifies me, I brake.
It will not notify you until you are too close.
These jobs also try to ensure that the output that the application is giving you is really the one you have to see.
And here we can also see many problems of physical security.
When you are working with the application, especially in vehicles, is where these jobs come out the most.
Be careful with the signals.
It is something similar to what happens with the issues of signals and computer vision.
The image processing and that they can deceive you with the signals.
The issue of privacy, as we have said, is key in all these jobs.
Because basically you expose yourself, expose both the user who has the application and is using the markers that exist.
You are exposing yourself to know when it is being read.
If, for example, as happened with Pokémon Go, you were on the street, you could know when you were reading each Pokémon, etc.
Even with other cases of exposure that you want to put an element in augmented reality and someone can modify it.
But all these jobs, I have put the references, and I guess the slides will be released and I will have them available for you to take a look at.
But these jobs are quite interesting.
From here, some very cool cases come out.
Because when it comes to hiding information, I could use it to do this.
For example, I can hide information, my own information, and not be able to know until I have the way to get it out.
What you do is, in some way, you can encrypt it, you can hide it in the AR itself, and so you can search for information or you can make a cover channel, whatever you want.
There are also works on this, on how to hide information in augmented reality.
What else can we do?
This is very interesting because in some countries, as I said, you have the lion of fire, you have a way that a company really wants to make an impact with its event, and there are people who are dedicated to trying to do vandalism on it.
I don't know how the law is typified that you do vandalism on a monument but in AR.
It's complicated where the limit reaches and where you can say, well, you can put that on a Columbus statue.
The funny thing about this is that there are open source libraries to make graffitis in AR components.
You can say, okay, I'll download it and I can do it at home quietly.
I don't need to have a lot of knowledge.
And it's free and you can use it.
This is a rather complicated case to solve, since you are the one who can protect each monument from each place so that no one gets there.
Mount the marker so that anyone can see it with AR.
Since you don't block the markers, you'll tell me how you're going to protect these things.
For now, there haven't been many cases in Spain, but it's something we don't know how to solve.
What other problems could there be?
Especially following the cases of you going down the street and seeing messages, etc.
How do you control that someone doesn't spam you massively in your application?
That someone says, okay, I know there are going to be people playing whatever.
Or working with machines and I can put all the AR content I want there.
Spamming.
It's very complicated to control because there are no authentication systems in AR.
To say, hey, I just want my content to come out.
Or shoulder surfing.
It's another case where I can see it on my phone and other people can't.
But if I have it behind me, I'll see it.
How do I protect myself against that?
Just like the issue of vandalism, although I publish something in AR and someone can modify it.
Not putting it on top of something real, but something that has been used in AR.
Being able to edit it.
These cases are more complex and don't have much of a solution.
What has been seen the most, what happens is that it depends on HoloLens.
You have to buy them.
It's this library that you talk about privacy and security in the development toolkit.
Until now, all the toolkits we have taught at the beginning, above all, do not provide it.
All you do is develop, release the markers, program the model you want to appear on your device.
But it doesn't tell you at any time that it can only be seen by one person, that it's safe, that someone can modify it and who can't.
And in these cases you start to see that it can be a problem.
Especially if you develop for a company that wants to have a product that you don't want anyone to touch.
Here you start to see some solutions.
What I'm telling you is that it's not completely free because it depends on Microsoft.
So there's a little bit of a problem.
And the proposals we have thought about and that we are going to see in the practical part are the following.
We see that there is a problem in which I want to show something, but I don't want everyone to see it.
How can I do that?
Can I put some kind of encryption in that information?
Is it necessary that there is a team that I control to be able to read those markers?
Can I make sure that an AR document has not been modified?
Has there been anyone who has put a marker on it to deceive me and that when I read it, I modify the state of my device?
All this is what we are going to try to see with some examples of the pods that we are going to see, although we will go little by little.
To see with some small examples how we can solve these things.
I already told you that there are things that are more reachable, others not.
But I think that at least the first ones we will be able to see.
And well, without further ado, I leave you with my partner Jorge, who is going to tell you everything.
Well, once we have told a little about this more theoretical context, we are going to move on to the practical part.
What we are going to do is to contextualize the technology first.
I am going to explain a little how it works, in detail, and what are the dynamics that are usually followed with this type of technologies.
And we are going to go over the same example little by little, evolving it until we do something a little more complicated.
So we are going to go from 0 to... not 100, but well, as far as we get.
So, the library that we are going to use is called ARJS.
This library is a library that allows us, through JavaScript, to model augmented reality environments.
When handling it, what this library does is, on 3JS and, in turn, on A-Frame, it will allow us, in a very simple way, in the browser, to create our own augmented reality application.
This has quite a few positive points, in the sense that, normally, libraries are designed for you to develop an application with Android or in a mobile environment.
However, now that we are doing it with web technologies, this will allow us to run this same application on our phone or on our computer.
And when making the application, we are going to play a lot with that, because, normally, we would need, perhaps, a physical marker to be able to place that augmented reality object in reality.
So, thanks to this, what we are going to do is, we are going to use the laptop camera instead of using the mobile camera, which is the one we would normally use, and it will allow us, in a quick and fluid way, to do the tests and to see the changes as we do the code.
Well, as I have told you, here, what we are going to do is, we are going to use these two technologies, which are the ones that, above, have ARJS and are based on markers.
The AR markers will allow us to place these augmented reality entities in reality.
Because, in the end, augmented reality simply serves us to show content in a visual way, but we are not going to process data with it.
Data, encryption or anything else, we are going to do it in a traditional way, with a client-server model.
Here, the only thing we do with AR is to receive the data and show them in what is the reality.
But, how does our application know where it is going to show them?
Here there are different approximations.
The library that we use, uses markers.
In this case, the best known are Giro, but we also have, for example, the patterns, we have QR codes or we have bar codes.
There are even other libraries that would allow you to use images as markers.
So, what these codes are going to do is simply say, I want you to place this image here.
The library will understand it and will draw what we ask of it.
However, as Miguel mentioned before, we also have the case of Pokémon Go.
What Pokémon Go is doing is placing these elements, which are the Pokémon, using GPS coordinates.
This is, instead of saying to the application, I want that when you recognize this element, you draw an augmented reality object on top of it.
It is, I want that when you record in this geographical position, I want you to place this element at this exact point.
We are not going to use them, however, the library we use does allow us to do this, which is quite interesting.
Let's take a look at the concept tests.
So, let's start with Nolamundo.
Let's go to the simplest, what we are going to try to do is draw a cube.
For this workshop, we have a GitHub repository released.
So, we are going to use a code that will be in this repository.
So, we are going to gain speed and if you want to follow it while I explain it, you will be able to do it.
For this first test, and probably for the next two, we are going to use, on the one hand, GitHub.
On the other hand, we are going to use this augmented reality library and we are also going to use Netlify.
GitHub is a service that will allow us to host our code and host versions of our code online.
And what Netlify is going to do is take that code and show it to us on a website.
This is not going to be completely necessary because we are also going to be able to do it from the terminal, but we want to show you the whole cycle, from when you create the website until you can release it on the Internet and you can also use it from your own phone.
When it comes to a project with this library, it is practically the same as a website.
We are going to have an HTML document in which we store the content.
We are going to have JavaScript scripts in which we model how our application behaves.
And we are going to have CSS styles to be able to place that content in a visual way.
So, to get started, when we want to make an application with ARJS, we are going to have to include in our file these two libraries, which are the AR ones.
So, we are going to go to our code, and what we are going to do is, if you have downloaded the repository, you will have three folders, inside the exercises folder, in which you will find the proof of concept 0, which is what we are going to do now, the proof of concept 1,
which is what we will do later, and the proof of concept 2, in which we will make watermarks for documents.
So, in this first proof of concept, we have the solution and the exercise.
We are going to start with the exercise, ok?
We go here, and what we have is the index of our application already prepared.
We have here the comments, where they are going to indicate where we should place those lines of code.
So, as we have said, the first thing we have to do is include these two lines of code.
You have them inside the readme in the exercise.
So, you can also copy them directly from there without taking too long.
Here is the repository.
We go to exercises.
The initial proof.
Here we have the lines of code, ok?
So, the first thing we are going to do, as we have said, this is like a web application of a lifetime.
Here we have the doctype, we have the html, we have the head and we have the body.
Inside the head, what we are going to do is include these libraries.
On the other hand, this is not necessary, but in these proofs of concept we have also wanted to include some styles to make it look a little more aesthetic and more visual.
Now you will see it, but we are going to add it later, we are also going to do the change, so you can see how it is with styles and without styles, the same application.
So, in an augmented reality application, with this library, we have several tags that are explained here.
When we go to model an augmented reality application, with this library, what we are going to have are certain html tags that the library provides us and we can use them in a very simple way to create the application.
The first one we have is the scene.
With this, what we are going to do is model the scenario in which we are going to place our objects within reality.
We have the assets, which are going to be images, videos, music, in general, multimedia content that we want to load in that augmented reality application to use it later.
For example, when we are focusing on a marker, which, as we have said, is what is used to know where to position that element in augmented reality, instead of putting a cube, it will put the image that we decide.
On the other hand, the marker, as I said, is what is going to be used to position the elements in our application.
We are always going to use the turn marker in this proof of concept because it is the most standard and so we use a single marker.
And on the other hand, there is the camera.
What we are going to do is model a camera that is going to be the viewing angle that we have with respect to that scenario that we have created.
Because it is not the same in this type of augmented reality objects to see them from the front than from the back, since they are objects in three dimensions.
So, we have imported the libraries.
Now what we are going to do is create our application.
As we have said, we have the HTML and we have the HEAD.
In the HEAD we are going to have the two scripts that we have already imported and we are going to have our own scripts if we want to do bigger things.
And inside what is the body of our application, which is a web application, we are going to have the scene, we are going to have the camera, we are going to have the marker and we are going to have the drawings or the multimedia that we want to add to our application.
So, if we see the body in more detail, we are going to have, on the one hand, the scene as seen here.
This is going to have some configuration options.
In the repository they are already set, but the most important is the one of ARJS, which is the one that marks.
This is an ARJS scene and not A-Frame, which is the library that is below.
Inside we are going to have the marker and the camera.
So, if we remove these blank frames and unravel the code, we are going to see this.
We have the marker and inside this element of the marker we are going to put the things that we want to be displayed when the camera sees that marker.
In this case, for this first proof of concept, what we are going to do is put a cube, which we can directly create with this library without the need to use JavaScript.
And we are going to make that cube rotate.
And then, finally, let's not forget the fact of putting the camera.
If we do not put the camera, it will not work.
So, as I said, here we have the code in the repository.
So, inside the body we are going to add our scene.
We are going to have a marker.
And inside the marker we have to define our cube.
So, the element that is used for this inside A-Frame is called A-Box.
What it is going to create is a box that, as you can see, is going to have a red color, it is going to have these dimensions and we are going to put an animation.
This animation is going to have certain properties in which, in a loop, it is going to keep rotating according to these positions that are 360 degrees.
And finally, we are going to add the camera so that this works.
So, we put the camera here and save.
To run this, we are going to have several options.
Right now, I'm going to run it from my computer, but in the next steps, what I'm going to show you is how to publish this application on the Internet and that you can use it from your mobile or wherever.
So, I have it here.
Here we have the exercises and the solution.
Let's go to exercises.
And as we said, here we have the index, styles and static, which are the images that we are going to use.
We do not have JavaScript because, as I said in this first test, it will not be necessary.
What are we going to do to run it?
We are going to create a local server with Python, simply.
Python-m, we use the SimpleHttpServer and we are going to open it in port 8080.
So, we have this here and we are going to enter.
As you can see, here what we can see is our application.
This would be our first augmented reality application.
But, as I said, right now this is a camera.
What it is doing is trying to find where to position those elements, because right now it does not know, it is not distinguishing where.
I have here on the mobile prepared a marker, as you can see, once I show it to the camera.
I'm going to see if, giving it a little reflection, it does not show it to me.
Well, no, it's going to show it to me anyway.
So, look, there it is.
We have a marker and what this marker is telling our camera is, hey, position this cube at this point.
And as we have said in the animation, what we have said is, we want this cube to be positioned like this and rotate.
As you can see, this is a 3D object.
So , if I rotate it, the object is going to rotate because the perspective is not the same.
Here we would have this first proof of concept and, as I said, this is without styles, without putting any kind of aesthetics to this application.
What we are going to do for that, do you remember that line that I told you that we were not going to put yet?
Well, let's put it now.
We go to the repository and we are going to put the styles here, in the header.
We click.
We reload this first page.
And it is simply so that you can see how the effect of our application changes by simply putting this line.
So, now, if
the mobile wants to pay attention to me, it is a little different.
It just looks a little better now.
And we, within this application, are going to be able to add other content, accessories, about, for example, the instructions on how to use this.
We have put the hashtag there in case you have any questions, especially remotely.
Now, this is a little test for you to understand how to develop in AR.
Very basic.
But the next ones are a little more complex.
And here you have the hashtag.
If you ask, I will be attentive with the mobile to be able to answer you.
But here you can also use it.
So, if you see it, if it does not look good, hashtag CyberCampMariMorena.
With the CyberCamp19 too.
So, here we have this first test.
We will go little by little.
We will leave a little time in case you want to follow us now live.
So, let's explain a little now.
Now, if you want to access this, you will not be able to.
Because it is on my computer.
So, what we are going to explain to you is how, for free, using services that are available to everyone, we can make this page be uploaded online and that there are people from all over the world who can access it.
So, for that, let's go back to the app.
Well, of course, as we are using Git, we are going to upload it to GitHub.
So, you have two options.
You can make a fork of our repository.
So, if you make a fork, you will already have the code there.
Or you can create your own repository for this proof of concept.
And you would simply have to make a TINIT from this terminal, add all the changes and commit it.
And we make a push to the repository.
What this is going to do is that we have our code here in GitHub.
And that we can now synchronize it with Netlify.
So, we already have the GitHub repository.
We are going to create an account in this service.
Netlify is a service that will allow us, through a code repository such as GitHub or others similar, to synchronize the repository with this service and it will automatically upload it to the website.
So, if you want to create it, once you create the account, we are simply going to have to click on this button.
We are going to do it live.
We have here our account of Mari Morena.
I think it looks good, but since we have a lot of screen, we are going to enlarge it a little bit.
And what we are going to do is create a new site from GitHub.
We could also use Bitbucket or GitLab.
In our case, we have it in GitHub.
So, it is going to ask for authorization.
And what we are going to do is...
I already have these repositories synchronized, but we have to give permission to our repository so that it shows it to us here.
We are going to take the account.
And what we are going to do is select the repository.
In this case, it is this one.
Cybercam 2019 Security AR Workshop.
We save it.
And here it is going to show us our repository.
We are going to enter it.
And here it is as simple as simply next, next, next.
And we already have the repository up.
So, we are going to change the name, in case you want to test it later.
And we are going to put...
No, here it is not.
So, right now, if you go to marimorena.com, you are going to have the repository here.
What we are going to have to do is navigate to it.
So, let's see where it is.
It is going to be exercises.
It is going to be the zero concept test.
And, well, in our case, we are going to go to the solution.
And
, as you can see, we have it up on the Internet.
Well, it is going to ask for permissions to be able to activate the camera.
And we have the application up here.
So, right now, you, from the phone itself, if you go to marimorena.netlify.com, exercises, POC0, first project, solution, it is also going to open this web page for you.
Because it is public.
So, well, the only thing that is going to be missing is this Código Giro, which you can find on the Internet if you type in Código Giro.
And , if not, in the repository.
Not in this first POC, but in the next ones.
We are going to use some special markers.
You are also going to have them there, in case you want to do the tests.
So, with this, it would be like our first project.
We have seen how, in a very short time, we are going to be able to create a simple augmented reality application.
What it is going to do is show us a spinning cube.
This is what I have told you about how to create this web page.
And this is what you could see.
In this case, we did the test with a paper and a blue cube.
So, what we have seen so far, the examples we have seen, we realize that, so far, they have focused a lot on leisure.
They have focused a lot on video games.
It is true that there are other examples, such as the one Miguel was talking about, of that rear camera of the car that helps us to park.
But the proposal we made about this workshop is that we are going to see if we can take a look at this technology, which we think is very visual, very impactful, and which can be very useful for a user, for cybersecurity aspects, and bring them to this technology.
We are going to try to bring cybersecurity to the physical level.
We are going to bring it closer to people and we are going to make it easier to use.
So, it was a bit of a crazy idea.
But we have decided to carry it out a bit, we have decided to give it a spin, and, in the end, we have come to quite a few examples in which augmented reality can help us in security aspects.
The first one, the simplest one that occurred to us, was the fact that we have all seen, in offices, in school, in public spaces, the Wi-Fi code.
Hello, welcome, this is our Wi-Fi.
In many cases, there are public Wi-Fis and there is no problem with that.
But in other cases, as it can be an office, this can lead to a visitor entering our network and taking advantage of it.
So, what we have decided is to try to secure all aspects of the passwords that are here, or in a post-it, or simply on a blackboard.
For this, what we are going to do is, we are going to use a Giro code, and, within that Giro code, we are also going to put a QR code with information.
Do we have a question?
Ah, ok, sorry, we are going to take a break, so the people who are still programming can catch up.
Kikelo tells us that we have used Excel.
Ok, this is the iTerm of Mac, and we have added ZSell with a couple of plugins, because they are more comfortable for me.
Maybe you have noticed that it does self-completed, it has memory, it's pretty cool.
If you want to use it, it's ZSell.
Is there any more out there?
No, not now.
Ok, let's continue.
So, I'm going to put here the link of the Wi-Fi repository, and we are going to see all this example of passwords.
What are we going to do?
On the one hand, we are going to make the simple version.
However, we tell you that this is a proof of concept, so it has some flaws that we should, if we really want to use it, we should correct.
On the one hand, what we are going to do is, we are going to use a marker, which is exactly the same as I showed in the previous example, because it is valid for both examples.
That marker inside will have a small QR that will store our password.
Ok, in this proof of concept, that QR stores the password in clear.
That means that it is encrypted and that with any application that reads QR codes, you will be able to read it.
So, if we wanted to secure that, we would have to encrypt that code and that it would only be possible to decrypt it from our application.
For that, the idea that we have is, in a corporate environment, that you use an application from the company that has some special keys that allow you to decrypt that information with the Wi-Fi key.
In the second proof, we are going to use encryption and we are going to use cryptography.
In this first proof, for simplicity, as we want it to be incremental, we are going to put the concepts of how to add images and how to show content.
So, let's go step by step, ok?
Another very cool thing that this has is that, as we are going to use an application to see this QR code, we can modify that key.
So, we could give different RedWiFi credentials to a guest than to someone who has the corporate application and who is an employee.
That is something that allows us to do it this way, which is very cool.
For this, if we see the structure of our application, it is very similar to the previous one.
We are going to have the scripts in frame, here we are going to add our own javascripts and we are going to have the same thing in the scene.
We are going to have a camera, we are going to have a QR code and we are going to have here what we are going to draw, which in this case is what we are going to add to majors, ok?
So, in this case, the top part is going to be the same.
What we are going to change here is this second line at the bottom, in which we are going to add our QR library.
This will allow us to simply read the QR.
This is a library that you can find in github and it is open source.
So, let's go to the code part.
I'm going to stop the camera for a moment.
We have another question here, if you want.
Do you recommend any IDE or console to run and program all the augmented reality in Windows?
I think it's independent.
In principle, I'm using Atom, but you could use Sublime or you could use any other IDE.
I prefer this type of IDEs that are less closed than Eclipse or some known ones.
In principle, there is no problem.
The only thing, the part of github, I mean, the part of git and the part of github, I don't know very well how it will work in Windows.
I think there are IDEs that, as they already include it, you can make your own IDE.
And if not, it would be to use an external program, but I can't tell you there either.
So, let's go to the second POC, to the second proof of concept, which is the Wi-Fi.
Let's go to the exercise.
And as you can see here, we are going to have our CSS.
Here we are going to have our libraries with the JavaScript that we are going to use to handle the QR codes.
Here I leave you the marker, in case you want to use it.
And we are also going to have a series of images, which are mainly from the top headers, and that aesthetic that we have added with the CSS.
So, let's go to the exercise.
We open the index, which is what we are interested in.
And the first thing we are going to do, as we have said.
Now that we want to read QR codes, we are going to make a differentiation first.
The marker is not a QR code, okay?
If you look at the proof of concept 2, we have here a marker in PNG.
What we have here, what I point with the mouse, this is a marker, okay?
This is what the library is going to recognize us to position augmented reality elements.
What we have on this other side, this is already a QR code that is storing us information, okay?
They are not the same, although it is true that we could use the QR code as a marker, if we train the library for it.
So, I mean, nuance aside, we are going to continue.
And what we are going to do is, in the index, we are going to add our library to read QR codes.
As before, here we are also going to have the code fragments that we need, okay?
We are going to go to the solution, which I forgot to add here one, which is this, the script.
We take our script.
And here we are going to paste it, okay?
Here what we are saying is, we are going to use this script, which is going to be in this location, because if you notice, it is here in the lib folder, and it is this javascript file here, okay?
So, we already have a library that will allow us to read QR codes.
But first of all, we have to tell that library how we want it to read our QR code, okay?
So, for that, here in the repository we have included our own, okay?
We have this program, which what it is going to do is, it is going to take the camera of our laptop and it is going to create a reading loop in which it is going to try to locate QR codes.
If it finds a QR code, what it is going to do is, it is going to pass to this library that we have imported the data of that image in which the QR code appears, and what it is going to do is, those data in this case, which are a password, are a password,
it is going to show us on the screen.
Now we will see where.
So, first of all, we are going to run this library, which is this one here.
So, now, in our project, what we have...
So, now what we have is, we have an augmented reality application, if you notice, the code is very similar to the one before, and what we have added the most so far, a library that is going to allow us to interpret the information of the QR codes, and a small script that we have made,
which what it is going to do is, using that library, it is going to return us the information of that QR code so that we can use it, so that we can show it to the user.
So, the first thing, now we are not going to use a cube, we are not going to use that cube that we used before, which the library already gave us.
Now we are going to use an image.
What we are going to do is, we are going to show the user a poster, as it could be stuck on a wall, in which the Wi-Fi password appears.
So, for that, we are going to have to create an asset.
I was telling you before, in one of the slides, the elements that there are.
The assets are those elements that allow us to show in the application this multimedia content, such as images or videos.
Well, let's go here.
And here we have an example of this type of elements.
We take it here.
When creating this element, what we have is an asset, and inside the asset, what we are telling it is, we want it to be an image.
We are going to put this ID on the image, so that we can later locate it and use it in other parts of the code.
And this image is going to be located in the static folder, and it's going to be called passwordpanel.png.
Let's see it.
It's here.
This is going to be that Wi-Fi poster that we show to someone who has our application and is able to read these augmented reality codes.
Okay, so, this is the same as before.
We are going to need a marker of type GIF.
Here, as we have already added before, we have it.
So, what we are going to change is, instead of showing it that cube, we are going to tell our application that, instead of a cube, it is an asset.
It is this image that we have declared before.
In this case, let's see if we put it here.
Okay.
Here, this is already an element of the A-Frame library.
And what we are going to tell it is to place it with this rotation, so that when we focus it with the phone or with the laptop, it is seen correctly and it is not seen upside down or upside down.
We are going to scale it to 1.1.
This means that the size of our image is going to be the same as the size of the marker.
And, if you look here, we have the source, the source of the content.
And we are going to put here the ID of this asset that we have declared before.
Okay.
So, if we run this now, let's see what happens.
Here we have our project.
And we are going to do the same as before.
We are going to run this local server on our computer.
We go to our browser.
And we enter 8080.
Okay.
So, as you can see, we have a very similar presentation to the previous one.
You already know that if you have any questions, you can use the hashtag.
Okay.
So, we are going to show you this code.
We show... let's see...
Let the mobile pay attention.
Now it does not want.
Okay.
Let's see what is failing.
We have here our asset.
We have our image.
And probably what is happening is that we have here a script that is asking us for one more thing.
I had not wanted to add it yet.
We are going to add it and now we are going to try again.
So, we have to show our user the password.
If it is able to decrypt that QR code, what we are going to do is show it.
Okay.
But for that, how do we do it?
We are talking about augmented reality.
How do we show a text?
Okay.
Well, inside this library they also provide us with the option of how to do it.
For this, we are going to use what is an entity.
We have here too.
And I tell you.
This entity is going to have an id that is password.
It is going to have the same rotation as our image.
We are going to position it inside our image so that it is where we want it to be.
Not that the text is on one side and on the other side of the room we have the marker.
We are going to scale it so that it is a little bigger than the size of the QR.
And initially we are going to put this value that is going to be password not found.
So now I will show you the JavaScript script that what it is doing is initially it is going to look for this entity to add the content.
Let's save it.
Let's see if it wants it
now .
Okay, now it is showing it to me.
So now instead of showing a cube, what it is going to show us is this Wi-Fi panel.
Maybe now you see it a little weird because you say, well, but you are recording it with the computer.
The point is that the moment we publish this on the Internet, what we are going to do is not use the camera of our computer, it is to use the camera of the mobile.
But well, for simplicity, as we have said before, for these tests it is more comfortable to do it from the computer and so that it can be seen on this screen.
So what we have seen is that at the moment we do not get the text.
If we show the query, nothing is showing up.
Let's see how the script works.
We go to the query reader, this is, we remember, the code that we have made to tell the query library what it has to do with this query.
And what it is going to do is, it is going to look for the password element, it is going to add an attribute, which is the text one.
And it is going to put a value, which in this case is going to be the data that is inside our QR code.
So, we have already raised this before from Netlify.
Let's try.
So now it's a little easier, we just have to change this.
We have another question, although it is not about augmented reality, but they ask you what MacBook you use to program and if it also works for graphic design.
It's not AR, but hey.
Brisblue.
Well, I'm using a MacBook Pro.
I use it indifferently from any Linux distribution.
In my house I use Linux or this Mac and it works to program perfectly.
For graphic design issues, the same, although I also usually use a tablet to compensate for it.
So, we have it here on the screen.
We see the poster.
We are going to make it read this QR code, which is here in Peñajo.
If it wants.
And it doesn't want.
Let's see.
We have raised it with something that is not right.
The demo effect completely.
But well, inside a workshop there is a lot of space for a demo effect.
We have the camera here.
We have, just in case, here.
This was some mistake I made during the effect.
So, we have this raised here too.
If you want to try it, it is also here.
The only thing is that we are going to get a slightly different image, okay?
Because it is an old version.
You see?
Here it appears that it is not finding the password.
We are going to bring the QR a little closer.
This is very small.
And look how it appears here.
It has read that QR code.
And our password appears, which in this case is mypasswordkey.
We are going to repeat it again, in case you have not seen it well.
We allow access to the camera.
You see?
Here it appears that it is not detecting the password.
And when we bring the camera a little closer, what would be the password of our Wi-Fi appears.
Sometimes, with the reflections of the screen, it does not detect well.
Okay, so this would be this second proof of concept.
As we have shown you, here we have the assets, as we have been including them.
And here we have the example that, as you can see in the slide, we have the old image.
So, so far we have been doing an incremental development of what our code has been.
We have started simply doing that first project, in which we have added our red cube, which rotates.
About that same code, what we have done is...
Let's turn it around, let's use this for cybersecurity.
And what we have done is we have removed that cube, we have added that image of a Wi-Fi panel, and we have added to our marker a QR code, and to the application the possibility of reading that information in the QR code.
In this way, when someone comes to our office, he will not be so clear about where that Wi-Fi password is, but he will have an application through which he will have to use to be able to obtain that code.
It will no longer be anyone who enters or looks out the window and sees our code.
We have put this as an example.
However, we consider that it can be used for many types of information, and that it has an interest in protecting the keys or the information of our physical environments.
Turning it around, what we said is, well, this can also be useful for other things, of course.
And if we use it to verify, for example, documents, we will see if, by bringing this digital world to reality, we will also be able to show users more information about those documents that may be printed on paper.
Then, by means of a marker or a QR code, we will print them together with contracts or together with an exam or whatever, and we will be able to verify, for example, that that contract or that document has not been modified.
Or, for example, to whom that document has been delivered.
And on the other hand, what is the level of sensitivity of the information that is reflected in it.
So, for this, here we start to complicate things a little more.
We have the repository, and you already know the hashtag, MarimorenaCybercamp, CybercampMarimorena, sorry.
And for this, we go to the POC 2.
With this, what we said, we will also be able to see metadata, we will be able to trace this document, we will be able to know who it was delivered to in the first instance.
So, if we have a confidential document, if this document appears at some point in the networks, we will know where that leak of information has come from, which is very interesting.
And we will also know if this document has a category of confidential, reserved or simply public information.
For this, here we are going to use encryption.
We are going to encrypt the information that we store in that QR code, and we are only going to be able to decrypt it.
Here we are adding, as you can see, little by little, steps.
I show you the structure again so that you can see that even with this, what has been our augmented reality application changes little.
What really changes is what goes behind it.
But showing the information is still just as simple.
For this, we are going to use this reading library of QR codes again, and we are going to use this library that we are going to bring from a CDN, which is cryptography.
Here we have two parts.
On the one hand, we have to encrypt, and we are going to do it from Python, and we have to decrypt, which we are going to do in JavaScript.
So that brings complexity, because many cryptography libraries are designed to encrypt and decrypt with that same library.
So there is a lot of complexity that is abstracted.
However, the moment you have to use a encryption in two different environments, we already have to understand a little how that encryption works, and it will bring us a little more complexity.
However, here in the resolution I will explain later how we have done it so that you can see it.
And well, the same.
We have a scene, we have a camera, we have a marker, and we have that information that we want to show you.
Here we are going to add a little more code in the body, in the body of our document.
And we have the scene.
We are going to have several assets that are going to model the three states that our application has, which is going to be the document is valid, the document is not valid, and we do not find a qr to get information.
Be careful.
On the other hand, we are also going to use a turn-type marker.
In this case, our marker, I showed you before, is combined.
We are going to mix two markers.
As you can see, we are going to combine this double marker.
As we are storing a lot of information, our qr has grown, so now it is not so easy to put it inside that turn-type marker.
We are going to put it outside.
However, in this library it does not allow you, but in other libraries you could use this qr as a marker directly.
However, as this is the simplest to develop on the web, we have made this balance between what it offered us and what it did not.
So, we are going to have that marker.
In this case, it is going to have the combined image, as we said.
And we are going to have the camera, as before.
Ok, for this, we are going to see how this qr code is created.
We are going to take a pdf, a digital document from our computer.
And what we would do with it is...
We take the file binary.
We take its metadata.
We take those data and what we are going to do is a HMAC.
We are going to make a hash of those data to check that they are not modified later.
We are going to take the metadata list also to be able to show it.
In this proof of concept, in the panel we do not show it, ok?
But we could also put the metadata that, for example, include the title of the document, when it was created, when it was modified, etc.
The level of confidentiality.
Well, if we, this document that we are creating the qr, is going to be private, or is going to be public, or is going to be extremely confidential.
We are going to put the owner of the file, which in this case is going to be to whom it is given, of this file.
And we are also going to put a document, the entire document, or a URL to the document in digital format on a server.
To then be able to contrast that what is the content of the document has not changed compared to the original that we have stored.
All this we are going to do, on the one hand, the creation of the qr in a python script.
And on the other hand, deciphering that information that is in the qr from the javascript script, ok?
So, well, this is what I was telling you, that on the one hand we have this part of the computer in which we create the qr, and on the other hand we have this part, which can be any browser, in which we decipher it.
And this is how it will look at the end, ok?
So, let's get to it.
Let's go to the exercise.
And the first thing, we are going to add that new library that we have to import in which it will bring us the cryptographic functions.
Let's go to the exercises.
Let's go to 2.
Let's go to the solution.
And let's go to the index.
Ok.
With this, as we are importing it from a cdn, we are going to need internet for it to bring us these functions.
But, well, we could download this library and put it in our lib folder, ok?
So, here, in the qr code reading script, where we have made most of the changes.
Let's take a look at how that deciphering is being done.
Let's go to lib.
Let's go to the qr reader.
And as you can see, the code is quite similar to before.
However, here we are already starting to have this type of functions that will allow us to process the information in the qr.
As you can see, this is the tick function, this is the loop in which we are looking at the camera and we are constantly reading the image it brings us until we find our qr, ok?
At the moment I find the qr, this function will stop.
But later, we will have the deciphering function.
This is a proof of concept.
So , here we are not using the server.
What we are going to do is to store the key in this script.
But it should not be done like this, ok?
This is a lack of security.
And this should be stored safely.
However, for simplicity, at the moment we store it here.
We are going to call our encryption library, which is the one of CryptoJS.
And we are going to decipher all that encrypted text that the qr library has read to us.
To do this, we are going to take this library.
We know that the encryption is an AES and we know that it is in cfb mode.
So, simply calling the library here, a string will pass to us.
And in that string will be all the information, ok?
This library, this string, we are going to use it here.
And what we are going to do is that when the qr code is read, we are going to decipher it with this function, which is the one I have mentioned here briefly.
And those data that has been deciphered, we are going to add them to the image that we are showing.
Initially, an image will be shown that the state of the document is not known because there is no qr code.
But later, if there is a qr code and it is able to decipher it, that means that the encryption is valid, that the person who has encrypted this document had our keys.
So, what we are going to do is add the information that we get from that qr.
In this case, in this test, what we are going to add is, on the one hand, we are going to show if it is valid or not, and we are going to add the owner and the sensitivity of the information, ok?
And then, on the other hand, if it is not valid, we are going to show a different image in which we tell the user that this code is not valid, and we are going to tell him that it is invalid and that he should be careful, of course.
Ok?
This is briefly this code script.
So, here is what we have just imported, ok?
As we said, we imported the encryption library to be able to use it from our script.
And then, we have taken a look at what has changed compared to the previous version of the proof of concept, ok?
We go back to the scene, we go back to our content.
As we have said before, now we not only have one image, we are going to have several in which we model all the states of our application.
Ok?
So, we are going to take those assets and we are going to add them.
I am going to show them to you now.
They are in the static folder, then.
We go to the static folder.
We have this one, in which it is shown that the document is correct.
Here the owner is going to write to us, and here the sensitivity level is going to write to us.
We have this one, which is going to tell us that the QR code is not valid, be careful.
And we have, on the other hand, the unknown one, which is...