- [Instructor] Hello and welcome back
to the AI Security Practitioner course.
We are in Domain 1.
We're talking a lot about ethics
and human and AI interaction in this domain,
setting the foundation as we go through, you know,
incidents and incident response and things like that,
and other domain areas of this course.
But in this one, we're going to talk
about analyzing ethical considerations
in various aspects and in AI systems,
as well as traditional software systems,
and how we can build and use AI
in traditional software systems as well.
So let's talk about accessibility
and traditional AI systems.
So when we're working with our AI and building AI tools
for specific generative AI uses
within cloud-based, within traditional systems,
tying it into and making our new systems more effective,
inclusive design involve is very important.
And it involves designing the products
and services that can be used for a wide range of abilities
by our stakeholders, by our users, by our customers.
And it requires taking into account
the various needs of our users,
which could be, you know, disabilities.
Could be somebody that is visually impaired
or hearing impaired, as well as other impairments
in terms of using mobile devices.
Maybe, you know, the difficulty of using newer technology.
How do we make that easier and more accessible to others?
And so AI systems pose some unique challenges
when it comes to accessibility,
such as natural language processing systems
may struggle to understand the speech of certain users,
certain accents, certain potential speech impairments,
somebody that might have a stutter.
And also facial recognition may have difficulty
when people have less accurate and less,
less accurate of skin tones, you know,
makes it a little bit harder
for certain physical characteristics
to be noticed in facial recognition.
So those are some challenges
and some ways that we need to adjust
and make sure that we are utilizing
the ability to use AI for everyone.
And that's a key aspect to all of that.
And a good practice is that when developing these softwares,
that we are following accessibility
that we have with both AI
and traditional software systems.
We may have within certain systems,
they may be designed for keyboard-only navigation.
We may need to provide alternate texts
for various images, ensuring color contrast for users
with various visual impairments.
We can't ask somebody to, you know,
select the green button if they're visually impaired
and can't identify colors.
We need to make sure that if somebody can't see an image,
that that alternative text explains that image.
Those are all things that we need to understand
and need to build into our AI
as well as even traditional models.
So part of that is having human oversight.
I always say when talking
about artificial intelligence applications
is that AI is still an application
and it's still built by human beings.
So even though AI relies on algorithms
and data to make decisions,
those decisions are very specific based on that data.
So they can't differentiate what's fair and ethical
and that may perpetuate biases.
And that's why human oversight
and having steering committees,
having advisory groups from different geographic areas,
different cultures, different languages,
are very important to work through these decisions
and make sure that our AI systems are fair and unbiased.
Traditional software systems also, again, any system,
any application, obviously it has a level
of human development, human oversight.
And we need to make sure that any decisions
that those systems are making are fair and ethical.
They don't learn.
They don't have the large, you know,
the large neural networks to build,
and build and train models to learn.
And they're very static in terms of,
"this is my role and this is what I'm going to do."
But the level of oversights is there,
and depending on the complexity of the system,
we need to make sure that the impacts
and the decisions made by that system
are also fair and ethical.
As we kind of tie together and take traditional systems
and those traditional roles of those systems
and adopt potentially AI technology into them,
if we've got, you know, a system that is our web server
and we start building in chatbots
and building in prompt engineering
to find information on our website,
those also become areas of decision making
that we need to also make sure those decisions
are fair and unbiased as well.
And balancing the aspect of automation
and the aspect of human intervention
is a important consideration for any system,
for any AI development,
for any aspect of how we're working with data.
And if we have too much automation built in
without human oversight, that could lead
to more and more unfair and unethical decisions.
While on the flip side, too much human interaction
can slow down the decision making process,
and remove the effectiveness of our application,
of our system, and create a poor user experience.
So we need to have a delicate balance around that
and make sure that we're monitoring what's taking place
within our AI systems and our traditional systems,
testing those on various levels.
And a key aspect to all of that too
is not just fairness and unbiased, but also safety.
And AI poses some unique safety risks as well,
you know, when we're thinking about healthcare,
we're thinking about autonomous vehicles and transportation.
If an autonomous vehicle makes a wrong decision,
it could cause a serious accident,
just like from a human being perspective.
If we're using AI and we're using it in healthcare,
and we are testing various models
and testing various blood work and results,
if the model is not trained appropriately,
it could make a bad healthcare decision
and that could also risk somebody's treatment.
Traditional software, again,
also poses various safety risks.
Think about aerospace, nuclear power.
You know, those systems have been around for a long time.
We need to mitigate and do rigorous testing,
quality assurance processes, continuous monitoring.
Even though we're using
and talking about artificial intelligence
and we're talking about, you know,
moving towards generative AI and co-pilots and agents
and, you know, AI agents that are helping us,
you know, that are acting as our like digital assistants
and scheduling things for us and making appointments
and all of those things.
They're no different than traditional systems, ultimately.
The only thing that they're doing
is they're learning our behaviors
and adjusting based on the algorithms
and the models to make our lives
more efficient and effective.
But they still are built by human beings.
So making sure that we have oversight,
making sure that we're monitoring, we're testing,
and we're implementing various safety protocols,
standards, following regulations, guidelines,
good practices are going to help us
prevent accidents and errors.
We go back to the previous section
where we were talking about governance.
There has to be governance involved in any system,
whether it's artificial intelligence or AI systems,
and we need to make sure that the decisions
are not going to cause any serious consequences.
So therefore, you know,
we need to make sure we're assigning accountability
for these decisions, and we need to make sure
we have that determined, who is responsible
for a decision made by an AI system,
which is a little bit more difficult
than traditional systems.
You know, we have system owners a lot of times
with traditional systems that are, you know,
and developers that develop that application.
Within AI, we develop those AI models, those AI algorithms,
but then we step away from really ownership of that system.
But we need to make sure we are maintaining accountability,
you know, and we need to handle it in a very similar manner
that we do with traditional systems,
where it is assigned to the individual
that is creating and maintaining that system.
And we need to make sure, you know,
if it is a complex system with multiple layers,
multiple applications,
that we have somebody who is responsible.
There needs to be only one person that's really accountable.
There may be multiple people who are responsible,
but somebody ultimately has to be held accountable
if something takes place that is a compliance violation,
a legal violation, a safety violation.
These legal and ethical implications of accountability may,
you know, again, are complex,
but we need to make sure that we have considered
the issues of liability, transparency and fairness
when assigning that accountability.
Who is accountable for those various areas?
You know, it could be legal.
It could be people within our systems,
it could be somebody within our governance group,
within our risk, you know, our risk executives.
But there needs to be a RACI in terms of who's responsible,
accountable, communicated with and informed
within our overall applications.
whether they are AI or traditional.
AI systems, again, may perpetuate biases
if they're not designed to be inclusive.
We need to make sure that we promote
and work towards diversity
and inclusiveness within the design,
within the development of an AI system.
It's not just at the, you know,
at the monitoring and governance perspective.
It is a continuous use and continuous improvement aspect.
Traditional software, again, also same thing.
If this section doesn't provide you with information
to understand that AI and traditional systems
aren't that different in terms of how we need to promote
inclusivity in these systems,
that should be your big takeaway here
within this this particular video
is that what we've been doing
and what has been our development
in terms of continuous improvement,
continuous delivery within traditional systems,
traditional applications, should also be applied.
And any of those secure development life cycles
and frameworks that we have for traditional systems
should be applied to our AI systems as well
to build a level of governance, risk, compliance,
cyber resilience to our AI systems.
We need to make sure we're promoting,
you know, the diversity.
Having training and education for our developers,
for our architects, for our engineers,
for our administrators,
as well as all of our organization and our stakeholders
should have those policies and practices in place
to continue and to foster a culture
of inclusion within our technology.
Securing our software is a particular challenge as well
when we think about AI systems
and think about even traditional software challenges.
AI systems have different layers
of vulnerabilities and security threats,
but still have the typical threats like data breaches,
and people that want to gain access
and hack these systems.
And we need to make sure
that we are protecting these systems.
One of the common new risks that we have
and threats that we have is prompt injection attacks,
where attackers are trying to prompt a Copilot
or an ChatGPT and trying to gather information
that can help them gain access to organizations.
And so when we're talking about healthcare,
we're talking about finance,
we're talking about critical infrastructure,
utilizing AI, those become potential issues
that we need to look at.
And it's no different than how we would treat
traditional software security
and the particular challenges around,
you know, holes, patches, gaps within code,
and those particular vulnerabilities within software.
Malware and phishing attacks.
Obviously, encryption, maintaining,
having encryption and access controls
is going to be key to mitigate those.
But if we have weak encryption or weak access controls,
those also can be challenges
and can be security issues as well.
We need to enhance both security in AI
as well as traditional software.
We are working in a digital world
where everybody is accessing everything over the internet,
so it's important to implement good practices
such as vulnerability testing,
doing regular security audits.
Even, you know, third party
as well as internal assessments and audits.
Having proper incident response plans.
And incident response planning, incident response teams,
incident response executions are always all going to help us
with having good practices to enhance our security.
Balancing autonomy and control
within our systems is important.
So AI systems may have varying levels of autonomy.
We might be, you know, have full autonomous systems,
but we might also have systems that require
some level of human oversight.
As we stated before, having some layer
of at least human oversight
in terms of governing over and reviewing
how that model is making decisions,
doing periodic audits and assessments
are all going to be important
in terms of ethical and safety considerations,
as well as security considerations within our system.
When we train our model, we talk about supervised
and unsupervised data to train that model, you know,
using supervised data that's labeled data.
That has a level of human oversight
in training the decisions that our AI system might use.
But we want to make sure that we are considering all that
and balancing the autonomy.
Because putting too much, again, human interaction,
human governance into an AI system
is going to deter and render levels of ineffectiveness
to that AI system,
and kind of go against everything that our AI,
you know, our AI solutions are trying to do.
So we want to make sure that we are maintaining
and monitoring them, but not putting too much guardrails
to the point where it's not doing
what it's expected to do.
And same thing with traditional software.
Obviously, we want to have autonomy.
We want to have automation
within how we're using our software,
but we want to make sure
that we have a proper amount of oversight.
Again, autonomous systems,
back to kind of the ethical considerations
and ethical challenges.
If we are too autonomous
and we aren't looking at and evaluating
and auditing our models on a regular basis,
auditing our AI systems,
we lose that level of accountability.
You know, we need to make sure
that those systems have accountability,
and that we have levels of oversight,
and making sure that critical applications within healthcare
and transportation have the safety considerations.
We need to maintain transparency and the safety.
When designing those autonomous systems,
we need to have rigorous testing
when that all takes place as well.
Informed consent is a key aspect to our AI system.
All AI, as well as traditional systems and software,
we are, in most cases, there's some level
of information on there that is personal,
that has privacy needs and privacy considerations.
So we need to get informed consent from users,
as I stated before,
where somebody goes into a healthcare organization
and they are given the HIPAA attestation to say,
"Okay, this is how this healthcare organization
handles our personal health information."
That's really that informed consent,
and we need to use that for AI usage
as well as traditional systems.
Easier communication with traditional systems,
because it's not a moving target.
That information is where it is
and is used how it is in traditional systems.
AI, it's grabbing information from everywhere.
So making sure that we are building proper access control,
segmenting that private data
is important aspects to things as well
in terms of the ethical challenges, again,
transparency, fairness.
Making sure that the information
does not have any exposure to coercion
that can alter that data
and create a potential issue as well.
And ultimately, we also have to have transparency
within our system to provide trust
and have those ethical considerations in place
in understanding how decisions are made within our systems.
It's very important to ensure that trust,
so that we can continue to,
so that those AI systems and processors are continued
and are used appropriately.
Again, traditional software, it's a little bit easier
to make that decision,
because how those decisions are made
are pretty straightforward.
But we ultimately, with all AI
as well as traditional systems,
need to make sure that we have communication
with the people that are interacting with the systems,
with our users, with our stakeholders.
Having proper manuals, system documentation.
Terms of service is a very important aspect
of here's how we're using your data and things like that.
Environmental impacts,
AI systems require energy to train and operate.
There's a lot of energy.
I wonder how a lot of these cloud providers
that are building these big AI, you know,
Azure OpenAI, Azure AI Foundry,
you know, Google and their AI,
and Amazon and all their AI tools,
need all of these large language models.
These neuro neural networks need a huge amount
of compute power and are using a huge amount of energy,
and all of these cloud providers
want to get zero carbon footprint
back to when their companies were founded.
So I haven't read too much on this,
but I'm really interested
in how these environmental considerations,
how this energy consumption for these AI systems
and the large increase versus traditional systems for these,
you know, large processing,
these large aspects of energy consumption,
and how we're building that sustainability,
how we're building and lowering that carbon footprint.
And like I said, using renewable energy sources,
reducing energy consumption as well as recycling,
is going to help to build and eliminate
some of those aspects in terms of footprint.
So that brings us to the end of this section.
So we talked a lot about ethical considerations
and safety and oversight,
and key things around ethical systems.
So hopefully this information gave you
some things to think about.
So thank you very much for attending this video.
I look forward to seeing you in the next one.
Thank you very much.