hey everyone it's Daniel from VMware GTIX PGE team.
I'd like to present an idea which is called
an intelligent screenshot-based change identification.
So we're starting with our journey with the two general questions.
Why do UI change matter?
And who do they matter to?
Nowadays, SaaS has enabled 24x7 self-service ability
which requires the continuous improvement of
the functionality and the user experience
with the high frequent iteration.
Some changes are expected and the same are not
so the awareness has become very important for the unexpected changes.
when you refresh a webpage anytime
you might be experiencing the display problem
Here's an sample showing the unexpected display changes.
right here, that shown an untranslated string.
or it could be an inappropriate alignment of button.
Those are the very common problems when the web pages are published,
and they need to be identified and fixed as early as possible
before they are presented to end users.
Those changes or what is called issue or defect,
That matters to a particular number of people
like document writers.
Who needs to just reflect the latest the user interface
to the documents and manuals
Project managers need to review the UI
to see if they're expected as design.
Quality assurance engineers care about the any inappropriate display
all the time
But there will be a disaster for them to go through
the big volume of UI within a short period of time.
Especially the work has to be repeated quite often.
Now we have the screenshot-based solution to help them out
You're presented the basic idea
on how we intelligently detect the changes in the general way
with the less or more details or only for the non-text change.
The diagram just presented that
we had two screenshots from the same URL.
which may contain the change from the one to another.
We highlight the difference on the one against another
Alternatively we can remove the text from the each of them
before we start detecting the difference, the changes
A notification will be sent out if any difference found
between the two screenshots, like this.
So the this like just the present the pub line of the of the difference identification from the Pier of the screenshot.
It started with the essential difference from the first one to a second one, and then we divided, to the second screenshot into the medical regions.
So we'd call the aspirin so division and then we identify the regions, they are just contain the now black just pixels and on the second screenshot and then we highlight the region we identified so hey here's the technical purple line for the identification.
So here's the animation to demonstrate how to works.
So, first of all, so we do the rgb.
subtraction.
Between the two peers and the subtraction will output, the non zero.
Value indicating the pixels with color.
and
If.
They are pixels or exactly saying a word just the brain down a pixel with the with a nail color like here.
And then, with the wanting to the second screenshot into two regions.
The upper one and the lower one.
And we'll check it out.
The two regions, see if we can find out and enough black regions.
Among those.
Okay, now we highlighted the region with those difference and we just bring attention to user.
On this region.
OK, we have the options to make the user have more focus on the changes in a smaller region.
Again, so we started with the difference, the screenshots are these self to action.
And then difference as we this time we evaluate the region, the screenshot into the four regions.
And then we started to to search and find out if any regions contain the non player.
websites.
And now we have the two oranges.
Which is contain the changes and which is what we call it.
And that's region so you were so you can see that the alden SMS, which is the.
move to here.
And this one is what's changed or his position.
or sometimes a task changes are navigable are acceptable, like the user names and times so they changes every time the different user login and they are expected, but those arts every the noise that is bothering people telling the real display issue.
So now we have the advanced identification they ignored their task change.
First of all, so we use the ocr technology to identify the town's region from the screenshots.
And then.
We have something tricky in we filled out the region with the tax find the best price per gram color so literally so we erase the tax from the sprint Shell, or we take off from them, will take the test, for we take the time off from them.
And then we'll be able to go through just the the person as we mentioned it in the previous discussion to identify the difference between the here.
So let's see just how to works to identify the non tax only change.
First of all, so we locate those just the arm tax regions by ocr.
On the.
Each of these financial.
And then we do the subtraction again.
Now you can see, the attacks are just be raised from the screenshot by filling out the bag haulers.
and
With do the region thing.
And we just a search about each region and trying to find out.
Just the some of them that contain the non black color.
Now we have this region identified, which is the container change with alignment.
And this this bottom only changed with the caption so that's something we don't care and we intend to ignore them so that's why it was not highlighted.
With a solution so we see how the House people in their day work and they're just some comments and errors gas adjusting the change detection, to be used for web pages, so we have like change change detection, based on the time sequence and environment, even for the whole way of saying.
Like like given any.
URL or webpage we captured we kept the screenshots by the time sequence and were able to just the monitor the change.
In between the latest one and two, the second last one at any particular moment so once the changes is detector so we're be so user will be able to to receive the notification to verify if the change is expected.
In the company for the corporate a web contents or it's published through the number of the varmints like touting ones stage environments production and production so.
They sing web pages.
don't necessarily have the same URL in the different form, for instance, the Google homepage might not have the URL like google.com so it could be like Google dash.
tasks.com and before he is published.
So in that case so we're able to just be a group, the number of the URLs referring to the same web page and i've down if I change this amount them.
So we identify the changes to see if anything just the happened when they just published through the whole publication purple line from the wireman through the one to another.
So let's see so our thing just that look the X back on the production, but just the something happened when they publish to the production and we are able to identify those changes as early as possible.
Ultimately, so we are able to monitor a change on the web on the web site level, so we use the word calling which allowed us to travel through the whole website by the sitemap, so we are able to just applying the model of the whole website.
Whatever it's based on the time sequence and vironment things morning.
So any just the have a brand.
or children, or just a ground web page will be just the trial through.
Just the by just given their homepage of the way our other website.
So let's talk about so.
The benefits musical just they have, by using by using the solution.
So we support any type of the changes.
And so, with the very extremely high speed and precision and we have the complex.
And to an automation put the the automation process.
As long as the changes present on you on, so it is reflecting on the pixel level, no matter what type of changes they are.
So they are not, they are technically be detectable and all the top of changes could be a separate as the two groups attack the pace of change or a non tab pace of change in were able to design.
to identify the both or one of them.
The precision and their physician always matter and Anna solution.
A skillful reviewer could look for the changes from the hundreds of specials within an hour, that the provenance, the more he revealed and more potential mistake, he were make us going.
To human error is now available.
But i'll solution richest 100% of the no change missed what the tam times process and body and all of this, all of this just accomplished within a few minutes.
And amanda what changed mode applying like a tiny squares or just the barman base or just the whole website email notification would arrive, right after the change detected and user only have to verify the changes the mansion in the email.
Instead of go through the all the screenshots they are could be thousand times of them, then the what they really have to reveal.
So let's have a very quick review on what this social offering users benefit, so we support any type of the display changes on where you are.
And 100% of no change and Miss with the 10 times per cents and volume and the third 100 and the first 100 of the timing days the human being, which is really incredible.
And we have the company automation personality which just freeze users hands for the something more important.
And it's save up like they might have 5% of the Labor costs, so the high frequently interesting work.
What the Bob so we're running received a a great feedback from the vmware internal user and different roles so it's changing the game in the fields of the documentation production management and a quality insurance.
So let's talk about a little bit about the future.
So we were be focusing on the following fields in the coming future.
So we'll be empowering the change of attachment theory to enable us to attack change, they only care in the thick of them and the field out the others.
So that means, so the the change detection all very costume on style.
Different user have their different purpose and focus where they doing the work when they do the the detection.
Someone just the.
Some of them are just care about the attacks, because they are charged up via the web content and some of them just.
Just a particular of the UFC as a space to change because they just the arm took care of you wanted to sign the layouts so we enable user to cast my theory change detections in the future.
And now we're comfortable to handle the thousand out screenshot in advance and now we're targeting in the millions, with the same time it's very important in the.
Proper the level service so we're going to build the arm the big system they are support at a component level user to just the process, they are all detection online asking time and keep the very good performers.
In or so with a lot of sheer the solution with the open source community, because we like to allow users to cast their their on.
Change detections ask the manager early detection is very customized so user are just on the one know their their perverse bath so they can just the.
APP developer there on just the pipeline to to plug in our platform to to to build up a their own customized the change detection system, whether on with a very short period of time.
So those are just the big progress we're going to make in the coming future.
So thank you for just the listening the DEMO is Daniel Kim.
Thank you.
well.