my name is David Santiago I'm a
principal technical artist at insomniac
games working with procedural systems in
pipeline and there been many talks by
insomniacs at GDC this year and if you
missed some of the ones earlier in the
week I'm sure they're recorded and I
will be mentioning some of the upcoming
talks during my presentation now before
I get started and run out of time I want
to make sure to thank everyone in
insomniac games for their support in
preparing this talk and I want to thank
and Samak for being such a great place
to work and insomniac E is hiring so
visit our booth on the show floor to
talk to people at openings here are some
of the current openings I can show this
slide later and we do I believe still
have openings for a rigging in turn in
both studios this summer and I'll be on
the at the booth on and off the next
couple days so I'm gonna talk about a
game that some of you might not have
Platinum to yet and I might reveal some
secrets of the game that have already
been published in every language on the
online so is show of hands so I do
really want this information of who
hasn't seen any spider-man open-world
content at all okay as well so I will be
sure to play the video even though it
stopped working I will just get out and
play it how many people have not played
the game alright well that's pretty much
all the information I need so so if
everybody else playing the game so how
did it start and how do we plan on
making this game let's jump back a few
years there was a meeting where Brian
Andujar of creative director revealed
the next game than insomniac would be
making for the PlayStation 4 and I
actually wasn't at that meeting but I
heard that people freaked out I remember
the moment I was told you're gonna be
working on spider-man and I was like
what what what we're doing spider-man
you're kidding and it was hard to
believe but I was very quickly convinced
that's what we were going to do so I was
being tasked to help build the cities
and the technology I first developed on
sunset overdrive creating ground
buildings and markup but for Manhattan
and now that's a little more real
state to cover but but I had a plan I
reviewed Brian's announcement
presentation which set the game's
pillars and goals for every department
and it was a mind-blowing amount of
content to create in a relatively short
amount of time and then in 2016 the
public rumors started that oh there's a
spider-man game coming and fans started
sharing their expectations online so III
2016 we opened on a shot of the city
transition to this street with traffic
and pedestrians and hot dog carts and we
cut to this card and people got excited
and while I'm on this card I'm remind to
thank Marvel games and Sony interactive
entertainment for their support and most
importantly I want to thank my family
and the family of micro families and my
co-workers for all their support and
enthusiasm as we threw ourselves into
making this game so the 2016 III crowd
they really erupted when we cut to the
shot of spider-man there and then he
started swinging down the city but now
for all of us at insomniac
this was real and the quality bar the
minimum quality bar was said so from the
start we work to make sure that all of
our demos ran real time on ps4 hardware
and from the beginning we had a strong
plan which actually got us to this 2016
III reveal but that plan wasn't yet
strong enough for the challenges of the
next two years and speaking of that
Brian in ihara the creative director has
a talk pretty much right after this
11:30 in the South Hall so you might
want to catch that but my talk today is
about the procedural systems and how we
integrated them into the part pipeline
as part of that plan to meet the
challenges so this is that video that
last week didn't play and then it
stopped started playing and now it's not
playing again so let me play this real
quick if you are able to see it do I
have to put it over somewhere else
do I need to quit the other presentation
maybe let's close that alright so since
it's not so for those of you who hadn't
seen anything so here is spider-man
swinging through the open world don't
need to play too much of this I think
you'll uh you'll get it alright back to
the presentation it's great when with
PowerPoint ah five all right so let me
give you an overview of my talk today so
we're gonna talk about what the open
world planned content was just for a
procedural content what the challenges
would be I'll give an overview of the
procedural systems that we were mostly
ones some of the ones we used on
spider-man I'll describe our original
pipeline plan and how the project
evolved as production happens and how we
updated the pipeline to handle it like
I'll mention some late additional
challenges we met to finish the game and
then most importantly what we learned so
here is our marvel island of Manhattan
but what is spider-man what is what is
in this game so it's an open-world game
of approximately six kilometers by three
kilometers of designed playable space
and we have the nine districts of Harlem
Upper East Side Upper West Side Hell's
Kitchen Midtown Chinatown Greenwich
financial district and the raft and
Rikers Island prisons and Central Park
there are 544 Rhodes and 1202 alleys
those are mostly fictitious alleys
either actually very few of them in
Manhattan over 8,300 distinct buildings
in 3,250 edifice prefabs and an edifice
it contains the contiguous buildings or
structures within a block more than 350
different types of storefronts which are
uniquely customizable more than three
thousand crimes more than three thousand
vignettes and I think there's a story
missions and a hero
so how will we do it well to make such a
large game we new automation and
procedural systems would be used on a
really large scale and in some net games
is not a big company I mean I think
during spider-man we're about 250 in the
entire company and we're always working
on multiple titles so we knew that
automation would be key using macros
scripts etc and then procedural systems
so what our procedural systems they are
methods for creating content
algorithmically rather than manually and
so data flows through a set of
operations that manipulate the data
based on parameters attributes and other
input now this is not real-time
procedural I mean there's always some
small
real-time procedural systems but these
are procedural tools to author content
and our goal is as much as possible to
ought to procedurally create a
procedurally authored open world Wow
it's it's a marvel talk you can't have
to do something like that so and I just
thought of that like two weeks ago
so Pam crafting is very important at
insomniac games designers and artists
handcraft the game experience as much as
possible throughout the entire game and
procedural systems are a tool to help
author a lot of content quickly and give
the artists and designers a big head
start so the goal for spider-man was to
procedurally generate an open world
early in production to give the
designers and artists the rest of the
remaining of our production time to
polish the game and our target was 80%
useable procedural content and I'll have
more on that later so we started with a
foundation of systems and tools from
sunset overdrive and we had to augment
that foundation to create the basic
tools and systems for spider-man sunset
overdrive we built roads on terrain and
placed a relatively small set of props
and traversal markup and there was a
building system to help block out areas
of the game but the building Sun sense
that were so unique that it was just
more efficient to constructed by hand at
that time so we had a plan for what the
main procedural systems we needed so I'm
going to be using this east up
orientation for the island throughout
the presentation so it fits our
quick ratio better and so hopefully you
get used to it it took me a while to get
used to it cuz I we weren't working this
way either so at the time in
pre-production the planned procedural
systems included ground system for
terrain streets sidewalks some decor and
integrated markup buildings with markup
the traffic system for pedestrians and
vehicles and a first pass of propping
the open worlds really
spamming the props out there so step one
for the ground system our roads and
alleys roads and alleys define this city
everything is dependent on their
placement size and lane configuration
now a big hint for this talk dependency
is very important so from these lines
and curves and editor we will generate
the streets sidewalks grey spaces and
the seawall around the island ground
modifiers affect how the environment is
constructed so this yellow overlay is
the contents of our ground modifier zone
now these are additional curves polygons
or volumes that cut deform modify add
materials etc to a particular areas of
the game there was also an analogous
design modifier zone this is the ground
modifier zone a design modifier zone to
affect design content such as crimes
traffic bots navigation etc so we're not
going to generate a giant model for the
ground we're gonna make models for each
streaming region so our open world is
cut into square tile streaming regions
128 meters on a side I must say it about
128 meters on a side no it's a 128
meters on a side so if you see here
there's some double lines I mean they're
not important but it's an interesting
detail in our system they indicate a
history of scaling and offset of
streaming reasons regions for various
reasons and we actually preserve that
history so we could validate content to
make sure that it was fitting into it's
appropriate streaming region now there
are very few actually scaled regions in
the final game and mostly they're on the
coastline just to reduce the tile count
you know just make it a little bit
bigger we can fit the content into there
and you'll notice that Rikers Island and
the raft prison do not have streaming
regions but that's as far as procedural
sims are concerned because there is no
procedural content in their final
version tile IDs so each region has a
tile ID which is a map location from a
o1 to x-48 and each tile hierarchically
contains all the zones for its open
world content for that region and you
can notice that lonely tile in the top
right corner there that's X 48 and it's
our test region that was there
throughout the entire game so here is a
procedurally generated ground it's
typically 16 models per tile so that
each is a manageable size for physics
and other systems and this is actually a
debug version of the ground which is why
it is not complete in some areas and you
also see remnants of Rikers and raft up
on the top there which was later
completely handcrafted also note that
Central Park's ground was hand authored
to fit the character of a park in the
area allotted so eventually all of those
manual iterations of Central Park were
procedurally processed in order to
integrate them into the game the last
part of the ground system is kind of
actually the beginning of the prop
system it's where we add some ground
details lane lines crosswalks per
district materials and such and it's
also we also do a little first passive
Street props such as manhole covers and
sidewalk props such as traffic and
streetlights and a few things like trash
cans trees just to fill it out for
designer testing so when they're
swinging through the city up don't hit
that tree onto the building system so
buildings were constructed from
primitives such as boxes cylinders and
polygamy volumes to define the outline
of the floors of a building and then
these simple buildings are then
processed by the procedural system two
instance assets from several building
kits onto the faces of the gray block
and then there are roof models that
could be sized and tiled to fit a roof
or a custom roof model would be created
automatically if the procedural system
couldn't couldn't make it out of pieces
and then some rooftop props were
automatically added to these building
edifices at this time so things like the
HVAC
in those water towers the fizzy stir
system also added all the markup swing
he invaded
ledge paths for traversal and then as we
were laying out all these buildings we
notice that anytime we change the
streets we might have to change the
buildings some talks related to this
Doug's talk concrete jungle gym the
building traversal and spider-man that
is tomorrow morning and tomorrow
afternoon programming talk about our
tools our in-house tools editing with
immutable data by by Ron so our building
tool was originally designed with a
recalculate markup only mode so if a
building was hand modified or if it was
complete built from scratch our system
could automatically read that edifice
prefab generate the markup and that
markup was always optimized by joining
all segments to minimize the number of
curves minimize the markup actors so
that's pretty much the building system
and onto traffic system so from the road
pedestrian and vehicle traffic were
procedurally generated so as far as the
pedestrian traffic system the first
thing we did is we placed
excuse me static pedestrian volumes on
sidewalks and so those volumes just
populated the sidewalk with a certain
density of pedestrians and they were
they would just be standing there to
start with but then they could get
promoted to the pedestrian system to
actually walk around so after you placed
those volumes then we would create a
network of paths that would connect from
corner to corner so corner to corner
along the sidewalks corner to quarter
across the street then we would
integrate any paths from other assets
like bus stops and subways because they
had their paths that where you have to
come up the stairs and around the subway
or around the bus stop around the bus
shelter so those had some built in
curves they would be integrated into the
pedestrian system and finally we'd
integrate any hand authored path so
there was a lot of hand authoring of
paths which pretty much just curves
through parks and plazas and maybe some
other spaces so those were all
integrated together and then they would
generate the pedestrian traffic nodes so
that the actual gameplay system would
use and that would include the
traffic phases for each link so that
when the light turned green they could
walk across and one day it was red they
would hopefully stop actually would turn
yellow and they would speed up and run
across the intersection
well said sometimes you know they just
stopped but so that we had this debug
with the show with the color all the
face is to make sure thing was all
working correctly and the pedestrians as
they II reached each node they could
decide on whether they wanted across the
street or just turn around the corner
and go down the sidewalk again so now on
to the vehicles it's very similar to the
pedestrian traffic system so but first
the parking and driving lanes had to be
chosen from a variety of the lane
template templates from each street size
so each Street would could randomly if
it wasn't assigned a specific template
of number of lanes and and indirection
and then the crosswalks from the
pedestrian system became the stop lines
for the traffic at the intercession so
just like we populated the sidewalks
with pedestrians of the volume we used
volumes to populate the parked cars in
the parking lane the only difference is
we had to create multiple volumes to
avoid bus stops hydrants alleys and
other exclusion areas where you
shouldn't be parking your car we also
created a network of the directional
paths that would take a vehicle from a
intersection intersection and then at
the intersections give them the legal or
desirable turns that they could make
then we would integrate the alleys as
limited paths so if it was a drivable
alley so see it back in the distance
there there are those magenta lines
those are alleys and they come come in
too and they are integrated and I will
talk a little more about their their
limited use later and then we would
create the traffic nodes with the
corresponding phases and links to other
nodes and the lane markings based on the
traffic pattern would then get get
applied in the with the prop system so
here you can see the route choices so
these are two - well they're two two
lane one-way streets that are crossing
each other and that Lane on the Left
which is actually the right lane going
down I don't know if it's south east or
whichever way it's going but they have
the choice of turning right into either
the lanes or going across the
intersection and the other lane has no
choice they're only allowed in this
traffic situation to go straight across
the
intersection now here's a more complex
area with pedestrian vehicle paths and
so it's multi Lane streets so these are
these are avenues and so it gets a
little more complicated as to what
people can do if you look at that lower
intersection in the middle there you can
see that all the lanes that have the
green light so they can turn two lanes
right into that one-way street and the
other ones can go straight across and
then when the traffic phase changes that
left turn lane going from right to left
they're allowed to turn onto that
one-way street and then here the same
the pedestrian traffic for that area you
can see the hand authored paths through
that plaza there and yet pretty much all
they had to do was draw those curves so
they intersected a procedural path and
they would automatically be integrated
into the system so now the dekor system
which is pretty much props and and
decals and then and then some ped
prefabs I'll simply explain in a minute
so we procedurally optimize placement of
the street props such as manholes skid
marks tar asphalt patches and such and
sidewalk props which include street
lights traffic lights mailboxes hydrants
trees and on and on and on now also
pedestrian prefabs were part of the
system so pedestrian proof shows where
things were like bus stops subway
entrances hot dog carts sign spinners
cab hailers so there was our assets with
integrated actors that could be promoted
to bed at pedestrian behavior so say
people are waiting at a hotdog cart they
would wait there and then they could be
promoted to walk along the along the
path or flee from a crime if necessary
so whenever there was an encounter the
pedestrians would go into a flee mode
and then we could also integrate any
hand place pedestrian prefabs of which
they were also many they need to be
integrated into the the pedestrian
prefab system in order to populate the
game appropriately at the proper times
we also did a little bit of procedural
lighting and I'm not going to talk any
more about that because today at 2 p.m.
x-ray is going to be talking about our
procedural lighting tools so also in our
original design our tools already had
the sophistication required for the GAR
and debris propping of act three maybe I
should act how many people made it to
act three of the game where there's
stuff everywhere
alright so the challenge there was to
make it look like there is litter and
debris everywhere
but not block any path needed by
vehicles pedestrians crimes vignettes
missions etc so that was it's a little
tricky and I know there has been much
interest in puddles in the last nine
months or so those blue circles they're
procedurally placed puddles if wetness
reaches a certain threshold in the
material which uses a random seed a
puddle his place in the world and active
during rainy atmospheres and then it
will splash and ripple and such so at
this point we had all the tools to place
the props now Impostors a fairly early
in pre-production it was decided that
imposter generation should be part of
the procedural systems since we
generated a lot of the content where
that would turn into an imposter so
here's an overhead view of the imposter
version of the city and it contains
every structure now imposters are lower
res representations of every structure
in the city and we generate a simplified
model from the high-res geometry in an
entire edifice prefab and then we use
that simple model for our imposter
generation to project textures and make
an atlas and then we populate the
imposter zones with the correct updated
imposters so that we can put them in the
right kind of containers for optimizing
a memory and such and then we can pursue
the regenerate them when ground
structures materials or lighting changes
so pretty much we're generating
imposters over and over again and here
again are all the imposters and showing
that the entire city is always present
whenever you see a shot of the city it's
pretty much all imposters so at most
only twelve zones or ever ever loaded
high-res and even most of those knowns
are going to be in their lowest LOD
which is very similar to the content of
an imposter so that's it we've designed
and implemented all the systems we need
to create open world Manhattan in less
than a year of pre-production and now we
can generate all the procedural assets
let the other teams polish it up and
generate imposters and we're done
so it's party time it's party time we
can just sit back and make the game but
the design theme I don't I don't see the
design team at the at the party at all
what what's going on
well justice we feel like we were going
to be able to handle all the challenges
for the open world the designers wanted
some procedural system love so we had
some new challenges the first set of
challenges were dynamic encounters the
dynamic encounters are crimes and
vignettes now we place 29 different
crimes procedurally throughout the world
and 25 different vignettes for a
procedurally placed using different rule
sets another challenge at that time from
design and gameplay were additional
desired markup now that we were getting
some new hero moves and then bought
navigation so an example of of the rules
required to place a crime or vignette
like a jetpack a jetpack mission no
jetpack crime so what we do is have a
four jetpack crime and have to find a
roof in the city with 64 square meters
of combat space available that was
within 40 meters of two other rooftops
with 64 scuitto square meters of combat
space available which were within 20
meters up and down from that original
rooftop and there was a clear path from
each roof to each roof so I could put in
nav and then also so I could put nav
from each rooftop down to the sidewalk
or down to the alley below so the system
would run through the world find all
those possible places based on the
buildings that were currently in the
game and then based on the density that
was required or neat or desired for each
district it would place all of those
crimes or in this case those jetpack
crimes and create all that markup for
the for the actors in the bots so if
we're going to procedurally author all
that content with all those dependencies
and have it be handcrafted we need to
figure this out and make a pipeline so
we have to think about that crime in
vignettes are dependent on the
environment and changing a road or alley
will affect contiguous blocks defined by
those roads and therefore all about
everything in those tiles so buildings
crimes etc and even changing moving a
crime can
effect nine tiles because of its
depending on its complexity because
crimes drive prop placement we want to
avoid props or static props in the world
where there's going to be a crime and
then changing building will affect
traversal crimes again lighting and an
audio did I mention audio audio wants
procedural help so just to start with
audio we generate some procedural
systems to generate the reverb volumes
for audio and also be able to distribute
audiences at appropriate locations
throughout the city and speaking of that
tomorrow the Otto team designing the
bustling soundscape so Alex and Blake
will be presenting on audio for the game
now back to our story and all of this
stuff that's a lot a lot of procedural
systems a lot of rules a lot of
dependencies so we need a plan and that
is to create a pipeline that completes
procedural content in phases that we can
iterate approve and lock content for all
the handcrafting I mean because we're
going to go from this to this it's just
gonna be a little more complicated so we
got Houdini's so although Houdini from
side effects software is our main
package of choice to develop and manage
our procedural systems in production the
algorithms can be implemented in any
software package and in fact we have
plenty of code for generating procedural
content that is independent of Houdini
or any other third-party DCC software
now here is a circa 2016 implementation
of our open world pipeline involving
procedural systems and it's a very
simplified diagram without every
possible error or box because it's just
unreadable and it's broken the roughly
three phases so in phase one it's the
main world layout we're predominantly
designers place roads rough out
buildings to find Park areas and mission
spaces and so forth and then we move on
to Phase two where the majority of
procedural content is authored and then
finally phase three the polish stage
where the procedural systems will just
help finish the game and let me quickly
point out that Impostors and lighting
data are being calculated during every
iteration of each phase and during phase
two and three audio and UI data are
also being procedurally generated as
well so the game is always playable in
its most updated state so here's the
first phase the first step is designers
layout curves to define the roads
well actually the very first version of
the city was from an illustrator drawing
of the map for our Manhattan and the
procedural system could ingest that
image and then create the curves in
polygon primatives in our editor the
procedural system would also randomly
place rectangular box buildings in
different zone to block up to block out
the city and it would actually read
color data that the designers put down
in order to vary the range of building
sizes for the different areas so once
they got that to a certain state we were
just going to work just in the editor so
the designers would modify curves in the
editor to redefine the roads and they
could specify Road directions or
dominate direction if it's a
multi-directional T Lane bi-directional
Street Road width and the road width was
from outside a sidewalk to outside of
the sidewalk Street width is from curb
to curb they could alter curb height
lane configuration and and so on so
initially only the main avenues were
specified and the procedural system
automatically chose street sizes
directions and Lane layouts for all the
rest of the streets so we just told it
something like we want 70% of streets to
be one-way streets and 70% of those
should be one traffic lane with one
parking Lane and then the system just
went through the city to create workable
traffic patterns and we got those
alternating one-way streets as you might
expect but as time went on the designers
would specifically choose sizes and Lane
configurations for many the streets to
accommodate story and and chases and
such but I would bet that most of the
streets sizes and directions are
determined procedurally so also to start
we had very few alleys there were some
blocks that designers knew they wanted
to break up but until we had buildings
in place it didn't really make sense to
try to place alleys so then using those
road curves just like I kind of talked
earlier we proceeded you generated the
ground
that's just like that
so then the designers place the
parameters to defined the buildings and
in this first phase so the rough
elevations and profiles are created and
procedural systems can modular eyes the
buildings using those basic
architectural sets and then it would any
would apply the markup so now designers
could swing through the city and test
out their space so is swing traversal
possible is desired silhouettes and
changing urban landscape lead you along
the desired paths do we have the correct
variety of density and shapes for the
different districts and neighborhoods so
there was another talk here at GDC that
actually went over all those decisions
but one thing to keep in mind is that
this early in production spider-man did
not have his complete move set of web
zipping and parkour or pork point
launching I mean it was some pretty
basic stuff so we were trying to approve
this layout with some limitations
so as we iterated in this first face
there obviously should be an arrow back
to Road curves so as and then as also as
separate buildings were used to fill a
block the procedural system would
automatically create the alley curves in
between them and you might ask why do we
actually need the alley curves instead
of just saying well the space between
buildings is an alley well again some of
them are drivable used for crimes and
chases and they can be even used by
regular vehicles their street is is
blocked by a crime or a vignette so they
could take a detour down an alley we
would also use those cursed inform the
procedural systems for change in the
ground surface props lighting audio
needs etc so after this first phase we
should have the streets pretty much done
and the rough buildings can drive our
next phase phase 2 is where the bulk of
the people work for our procedural
systems happen and there should be
arrows going from each block to any
previous block because any change based
on a dependency requires some
reauthorization of the pipeline we will
generate everything in this phase based
on all the changes to keep the game
update updated and playable that last
bullet is very important it means that
during this phase artists and designers
should not hand
anything that a procedural system
generates they must use the controllable
parameters and inputs to the procedural
system to affect change if they don't
they're at the risk of losing work so
here is the what we call the art
polyline pass and that's the opportunity
for in the environment artists to
replace the primitives in the designer
zones with multiple polygonal volumes to
further define the shape of a building
and the artifice the artists could also
specify the architectural sets that they
were wanted to use for the building in
this phase the artist could also modify
or delete the alley curves and saying
that they didn't want a drivable alley
there and then we could recalculate the
ground for them and in reality this
diagram this phase two represents two
almost identical phases that we call the
two stages of environment modular zation
passes now iterations of the environment
arts first modules a modularization pass
would yield the structures with their
final layouts and shapes but the
architectural styles and propping could
change in the next stage the next round
of phase two or phase three now after
this face that we're we should have
accurately at the end of this phase I
guess after the end of this first stage
of phase two we should have accurately
placed and approved the pedestrians
traffic vignettes crimes props and
accuracy is within the scope of our
procedural system design so the goal was
80% completion using procedural systems
so that means that 80% of procedural
content would be ready to ship or
completed to 80% so that art and design
would have at most 20 percent of the
work they would traditionally have so
that means it's something that used to
set be a week of artist time should be a
day of artist time five weeks one week
so what we're trying to do is generate
content to rules and metrics and cull to
Ballpark but not the final spec because
the manual polishing passes would take
care of the final adjustments so at this
point phase three design has approved
everything and there's arts sign-off
after the second round of phase two and
every team is hand crafting their parts
of the game and the procedural systems
will just
date things like markup Impostors
lighting audio etc and there should be
plenty of production time to develop
procedural assistance to add any new
traversal or markup or crimes or
vignettes and then we can also start
working on optimization and validation
tools to improve performance and help
find bugs so this pipeline was created
to work within a specific plan and that
plan is to develop the island uniformly
through several stages of completion and
we need to lock the rough map layout and
structures and define the key spaces
before we can work through any detail
and so we got to start with the
island-wide systems so again we're gonna
finalize these roads in the map layout
so oh this was yeah this was a like a
little clip showing the changes
throughout the that the production after
pre-production but it's ok it's not that
interesting the so the generated cond
we then there we do have the generated
grounds which did find the key spaces
and landmarks so we're gonna work one
district at a time adding detail in
iterating on dependencies within that
district to bring it to the next level
of completion and that was fun in our
pipeline design so as long as we worked
through the island district by district
maintaining uniform levels of polish we
could treat each district kind of as its
own island and because of the unique
attributes and establish boundary
conditions for each district we already
were handling the shared edge tiles in a
special way so for example here is
Greenwich district in phase 2 stage 1 so
at this point we've gone through phase 1
so we finalized the road map and the
rough buildings defining the silhouettes
and traversal spaces and they've been
converted into prefabs of instance
architecture for the whole city so now
phase 2 stage 1 art does its modules the
first module ization pass they finalize
the shapes of the buildings artists are
assigned neighborhoods each artist
assigned a neighborhood they do their
stuff they work through the district
then a district is a level completion
where we can run our procedural systems
to place the crimes vignettes and props
and art moves on to the other districts
to do their first stay
phase two but we found as artists work
through the neighborhoods and districts
they needed to adjust roads and alleys
for the character of the space so the
placement direction or size of road or
alley in one turret district will affect
traffic flows from multiple districts so
we might have to change even more roads
so at least to a domino effect of
changes and that brings us all the way
back to block one of the flowchart also
due to different milestone missions
story development items not only were
different districts at different stages
of completion but one tile next to
another tile one could be in rough and
one could be in polish but in order to
place our majority of our content a tile
has to be completed in environments arts
first modular ization Pass and gained
several approvals and that wasn't
happening so there was too much work
here and not iwould enough work here and
that smooth progression was was upset
for our development mm-hmm now it's not
that the procedural systems couldn't
regenerate the content but manual work
could be lost so for example adjusting
an alley just just adjusting the alley
width in one region could affect many
surrounding tiles such as the UV
continuity around the block because a
block could span many tiles proper
placement proper proper placement it's a
tongue-twister proper proper placement
and the placement of crimes and
vignettes in that alley or vicinity
I never messed that up when I was
rehearsing so that created a domino
effect beyond what we imagined now
honestly we or III expected it to happen
at edge or coastal tiles where there was
minimal gameplay content and I knew
those who get neglected and fall behind
so they were just crazy amount of
dependencies every every tiles regions
content is dependent on at least its
neighbors and best case they're in
production in parallel so not only are
districts being developed to the
different levels of Polish but the
individual tiles as well and there was
new content for dioramas missions demos
and other test areas that we're all over
the city and so things like adding a
median
marinas more roof top features they
required at least update of additional
markup for the hero and bot support and
we also relocated several story related
locations and I was personally a little
bit stressed and frustrated that we kind
of weren't stickin to the pipeline that
that we worked on ah so what do we do we
get the leads together and we figure it
all out now it didn't happen in a day
several meetings on different topics
with the right team members over several
weeks at least
and some growing pains were had by all I
mean we needed to know the targets and
goals for art design and gameplay one of
the main systems the ground buildings
has set are going to be at a level of
completion that we can move on to
everything that all the other processes
that depend on them and then when it's
work on a tile neighborhood or district
gonna be at a stage where additional
polishing of the art crimes missions etc
will not affect any downstream processes
because then we can more confidently
hand off work to the next procedural
system with the less fear of having to
go back up the flowchart and then at
some point we'll get to our final
imposters lighting audio and UI and and
we'll be done
speaking about more about how we work as
a team
Jason our environment lead has a talk
tomorrow case study and empowering your
artists so knowing what everyone needs
to finish their work we make a new plan
for our procedural systems and where
possible we branch the dependencies so
that each tile region can be
independently edited as much as possible
then each procedure law ization
operation should be modularized based on
its dependencies so for example traffic
lights traffic lights they're really
only dependent on the roads and the
sidewalk corners once we know a
direction of a road and where the corner
is we know exactly where to place a
traffic light so at that point that we
have all of its dependencies set that's
when traffic lights or anything with the
same dependencies should be placed in
that module mailboxes for example they
have additional dependencies such as
avoiding alleys bus stops crosswalks
things like that
so anything with those dependencies
should be in its own in its own prop
placement model better rules so we need
also need some better rules to catch the
corner cases always lots of special
corner cases we want to get these rules
so we can definitely exclude the invalid
content now we don't need to be 100%
accurate with our placement of content
since we know there will be handcrafting
passes but we do want to make sure that
at least 100% of the planned procedural
content is in the game we will place
things the metrics and then we'll
indicate assets that are in the ballpark
but with which we are less confident of
their proper placement right because
it's easier to try to place these a
hundred and sixty crimes out there and
know that every people have to adjust
them and put them in the right spot
other than just saying oh I can only
place 120 of them and then relying on
the artists and designers to count and
remember that they have 40 more crimes
they need to place and placing them in
the appropriate areas so this system for
verifying confidence of placement it
morph into a validation system that was
applied later to a lot more content
which was whether it was procedurally
generated or not so bringing order to
the city bringing order means getting
the teams on the same page and as many
contiguous tiles to the appropriate
level of completion for our new modular
process so originally each district was
a set of contiguous tiles one now it's
like I don't care just give me some
contiguous tiles so we can work through
this game and the the new procedural
processes that preserve additional
handcrafting would be interleaved with
the artisan designer work now that's
easier said than done but what we got it
done then came a few more challenges so
we had some wall crawl issues and on
some new assets especially some of the
new storefronts so we needed some
additional ledge path markup and we all
call collision added then it had to be
added by the procedural system because
there's particular to which buildings
kits the storefronts were added to also
a little bit later in production a
little bit late in production we
implemented the act system so we knew
that
that Spiderman was gonna have these acts
and different situations and times of
day but later what is late in production
when we created the system of overlay
zones that would control the state of
the world at different times of day or
weather conditions or different acts so
that way stores could be opening the day
closed at night closed during act 3
there could be different groups of
people out and about depending on these
situations and we could modify what was
in the city and act to an actor II such
as all the garbage and in different
vignettes so that solution was proposed
in October 2017 we designed it in 2000
December 2017 I went on holiday break
and then with our implementation was
completed in March of 2018 and then the
content was complete well before we
shipped later in the year so there were
also some late mission space updates
that required reauthorize
with how the new systems handled it and
then that I mentioned we did audio so we
had to generate a bunch of custom audio
collision models to match the visible
textures that were being presented by a
material and that was a significant in
Central Park so lessons learned I've
talked really fast so we already knew
this we the case a strict workflow is
best but we need flexibility for
production we want to lock procedurally
generated elements with their
dependencies as soon as possible as soon
as something's locked
we're done with it we can move on we
also want to hark eclis modular eyes the
procedurally generated content to reduce
dependencies so that's where I was
talking about the traffic light once we
know the the dependencies to place that
traffic light or any other content that
has the same dependencies we want to do
that then we can use that as that might
be a dependency for other props or other
content that would come later so
modularizing it will reduce the number
of dependencies if you have although all
the props in one a module then you've
got every possible depend
going in there you don't want to over
define the goals of our procedural
system modules now you don't you don't
need to create the perfect rules to
place something first a lot it takes a
lot longer to make the perfect system
I'm at I think it supposedly takes in an
infinite amount of time to make the
perfect system and then second if you if
you only play something with strict
criteria you're not going to find that
many locations that meet the meet the
criteria so again I just like I was
saying before it won't be it won't be
out there in the world to be nudged into
place it'll be something that will have
to be hand added later which takes a lot
more work so it's easy to if something's
there to say oh I'm going to just nudge
this to a good location also are the
perfect rules or are a lot of an over to
over defining your goals and is
increased constraints and constraints
for dependencies and we're always trying
to reduce dependencies for each stage of
procedural authoring because we know
things are going to change and any time
in so we want to have a limited number
of dependencies so it's a limited number
of changes now we can create procedural
systems that preserve manual work so
once authoring hand authoring has begun
we want to change to a mode where
procedural ism is a pride applied
appropriately so whether it just
maintains the hand authored work or it
augments it or processes another way
also compound dependencies should be
split into separate processes now Kondo
pound dependency is when a single asset
has distinct sets of rules of
replacement or has a significant level
of branching in its rules so for example
say today we have a crime that can be
placed on it needs me place or it can be
placed on a road and we want place it
like in the middle of a block or in an
intersection and then we just to orient
it with traffic but then also that same
crime it can be placed on top of a
rooftop well we don't want to have all
the rules for placing that we don't want
that all the rules for placing that
crime in one module we should separate
it out so okay the once we have the road
dependencies we'll place those crimes
with that there and then once we get to
the point where we're the building
dependencies we'll place that same crime
based on based on those rules so those
are the compound dependencies and then
again compound dependencies they limit
your options
for a single asset and will will
increase dependencies for all the assets
if you have compound dependencies in you
this big Network then it's going to
create unneeded dependencies for for the
neighbors we want to tag procedural
content well we want to tag everything
but knowing if something is procedural
means you can confidently regenerate it
again and again without destroying any
manual work now this can be a little bit
tricky since an incorrect workflow can
allow an artist to prop out around a
procedurally placed an instance without
creating any indication of the
dependency so for example an artist can
augment a procedurally placed vendor
cart and they put additional items
better there outside of the proper
hierarchy so procedural system has no
knowledge of any of those additions and
so if it decides oh I'm gonna move this
vendor over here it's a better location
well you might end up with floating soda
cans and pretzels in that spot because
they're not within the correct hierarchy
so that particular simple case didn't
happen but there were some similar cases
also knowing if something is completely
procedurally authored means that it
might benefit from Sam hand crafting so
it's like oh that's just in its in its
procedurally generated State let's see
if I can custom customize it too it's
it's part of its specific location we
want to perceive lis validate as much
content as possible now we want to find
content that is not placed as metrics
would dictate so for example a spawn
point is inside of a building that
doesn't have an interior so we want to
delete nudge or modify automatically
only when we're absolutely certain of
the result so if that spawner is no
longer usable in any way we know we can
delete it but if it could be usable
outside of the building we'll try to
adjust its position as defined in our in
our systems rules knowledge and then if
we can't be confident of that procedural
adjustment we need to indicate the
possible issue for the artists and
designers so that they can make the
decision now what we did for many of
those cases for dynamic elements is we
put those assets in a state where they
would not build or spawn unless someone
got around to Hannah
modifying them so ladies later in
production the procedural system was a
lot smarter about placing all the
breakables and throwables for crimes as
we had to do it over and over and over
again so the procedural assistants were
actually doing a lot of that work and we
would put those assets that we couldn't
find a proper location for in this state
so we lawn learned a lot of things that
we can do better and this afternoon
Ellen has a technical post-mortem but
overall I think we were satisfied so
repeating myself the original goal was
80% of procedural content would be ready
to ship or completed to 80% so that art
and design would have at most 20% of the
work they would traditionally have now
we exceeded those 80% goals but all that
additional time went into additional
iterations and additional handcrafting
of content and polished yeah we did it
would we didn't have there was no
leisure time you know trying to get it
get this out the door so now that you
know about the dependencies and what has
to happen when we change the ground I'll
tell you that that road curve zone was
locked in late January 2018 and the last
procedural change to the ground occurred
in late June 2018 a little over two
months before you could get the game in
stores I like to thank you all for
coming I really talked fast so there'll
be lots of times for questions thanks
Sony Interactive Sony Playstation Marvel
games side effects software Jeff Hannah
my advisor and everyone at insomniac
games for having making it such a great
experience a great adventure and now I
will open it up to questions so there's
microphones on either side
I can throw those job openings if you
like hello okay I worked on we happy few
and we have a procedural system a much
smaller system that was similar
structure and one of the problem we met
is communicating with the artists the
consequences of a change into a handmade
tie because they are clearly as you said
dependencies between handmade child and
the procedural generation around it it
was really hard for them to imagine all
the consequences of simple change and so
my question is how did you explain what
could be the possible consequences
because Houdini is not that simple it's
a complex pipeline and so well so at
first you know there's always a little
hesitancy for the artist to give up this
generating of you know authoring their
buildings you know they want to make
their buildings but as soon as they saw
that they could just change the safe and
then boom and it would instance all the
stuff for them they're like oh oh I like
this I like this so they I mean sure
they're there some some bumps in the
road but they started liking that they
could just do their polyline work and
get the structure they wanted and they
could instance the tree they could test
it it would could go through the whole
you know through the whole pipeline and
then later on I mean it took a little
bit but they had to they did have to
learn and and it's kind of a trial and
error as far as what they could change
and what they shouldn't change what
could be preserved and what wouldn't be
preserved different things about how to
it's just I mean it's particular to your
pipeline but how you do overrides so
what things could be override what
things can't be overridden just this all
those little details of the dependencies
and the hierarchy in your in your data
structure it's just it's just things
that have to learn and I mean we had a
small team so a really small team so
getting everybody up to speed
especially up for the open world it was
overall not that not not that
challenging I mean we just had lots and
lots and lots of iterations and and they
learned when to when they should just
stick stay with the procedural systems
and when they're going to cut it off so
sometimes it at some points right they
would just say I'm cutting you this off
from procedural systems at all and I'm
saying I and our system say okay fine
and you you I mean do you have to do
everything and that would be fine okay
thank you there is so much going on in
that city and I can see that from your
talk that used in posters and use lot
systems but I've noticed that you're
building some made of different blocks
with the and as you go to the city they
change Elodie's yeah like small pieces
not the whole building altogether
sometimes yeah and I'm just wondering
how did you manage having so many
objects and what are the decisions you
have to make to make the procedural
system handle so many objects in a
performant way well for the procedural
system every every building there were
only certain building sets building kits
that were would be used by the
procedural system when it was originally
authoring the city and each of those
building kits had required pieces right
and all that stuff that the the artists
environment artist designed the building
kits and then they would export an
entire building kit at a time and the
appropriate pieces would be put into the
procedural system library but then they
had all these auxiliary pieces they
could use later when they wanted to
polish it so as far as the the
procedural system it knew about a
certain sets of pieces with different
architectural styles but it didn't it
had no other knowledge other than their
dimensions and if there were special
offsets or things like that and then it
would just based on metrics you know
place them on so the the procedure
system didn't have have that much
knowledge and then later on when they
did things by hand they could actually
kind of they can quickly replace
architectural styles on their own or put
custom pieces or things in themselves
now as far as all the content in the
game Howie
it was a lot of content in the game I
don't know if there's anything else to
your question about how we managed all
those models I mean everything had LOD
and that was there's all these
requirements through how you would have
to author everything I guess one thing
with the imposter system so the imposter
system could actually if could actually
see an invisible LOD in the GM in the
model if it was available so if there
wasn't a possible lower resolution state
that we didn't want to be in a standard
LOD we created a special one that the
imposter system would see so that when
we took those prefabs of buildings and
simplified them up to make a single
model we would use that even more
simplified geometry thank you hi great
talk great game by the way thank you
very much I wanted to ask about the
iteration time like how long did it take
to to regenerate all the city how did
how long did it take to regenerate like
one one block or one area and were you
able to do it run time for example in
the editor generate and see me need to
change or did you have to reload the
editorial or the game the the artists
had tools in the editor that they could
if they did a pot a new polyline of a
building they could say regenerate this
building and regenerate the building or
regenerate the imposter there were some
things that they could do it wasn't like
real time it would you know we have to
get processed in the background then it
would pop in for them so took the minute
or something but most of the stuff they
would get it to a certain completion and
then it would run through the whole
procedural system at some point just
because there are so many dependencies
you know and then you don't know who has
what checked out so there's and that's a
whole thing with permissions and when
you can run changes and when you can
check things out that's yeah it's what
I'm forgetting I want to forget about
that we could actually regenerate the
entire open world overnight and in the
early in production we were because
there were so many rapid changes let's
see so the the ground system what it
would first process to create a sparse
set of data that would actually describe
worth what the UV continuity need to be
and where the materials were defined I
don't know if you noticed but every
single district has different materials
for its asphalt and for its sidewalks
that's fun and and so all
stuff there's a pre-processing a
processing time of about an hour to
create that sparse set of data and then
each tile itself each about sixteen
models per ground tile would take just
just for doing the ground and and and
creating the Associated data that other
steps would need took about a couple
minutes a tile but if we ran it all
together so that would that's about you
know five hours of generation or so but
overnight it was not then all the other
processes pretty much to process things
around the whole city would like to do a
processing pass up saying a crime
placement or things like that was twenty
five minutes no good to do so every time
you would do something worldwide I mean
I would try to everything has all of his
depends so processing the whole world
for something probably about twenty five
minutes as far as a placement pass okay
thank you
one of the questions is saying sounds
good and question about your tile states
so you have probably I mean I'd say
you'd have different acts and things
like that so that could be in different
states how did you solve it within
posters do you have like multiple in
posters as well for your now now it's
just one one one set of imposters and we
had this thing that we calculated
lighting at a distance so there were
actually really was some lighting out
there for they would light those
imposters based on based on captured
roads and such and memory footprints was
also taken into account when you like
yeah thanks so as we as that's the thing
the imposters were put into the zones in
order to preserve appropriate memory
footprints because we were you know
guess it gets technical we were making
sure that we were stacking these things
tightly next to each other in one
location in memory so we could have them
all in then also you go to you know an
instance interior and they're gone or
but it's a you're an instance interior
that has a view out the window then
certain of those imposters zones that
are visible has to be on right so that
all that stuff was how we how we packed
them and when you had a toilet for a
high lot did you for example had a fixed
budget of memory you could use for a
certain tile to place things or oh so
yeah every every artist knows their tile
budget for stuff I mean they can at the
end of the game they've got it up as
they're doing stuff and they go like
know what am i gonna take out so uh okay
but and then the procedural systems as I
would process things I mean we optimize
lots of things optimizing LEDs
optimizing collision yeah this is a lot
of things we tried to do so everyth all
have fixed budgets every tile has a
fixed budget until we break the rules
for some reason yes because an aisle
next to it has a little bit less you can
take a little bit more now now we didn't
we didn't treat it that way it's only if
this has to be in this tile because a
lot of gameplay actually depends on this
and the open world has to be in this
particular state okay thank you yeah
yeah my question was related to
something that that you just brought up
in the previous question about the file
locking and working between different
artists I you know kind of run into that
as soon as you start to scale you have
these problems between authoring assets
and then as soon as you introduce a
process of procedural generation there
my head's kind of spinning so just my
question around that did you have what
was your process for coordinating all
that together and are you'd also on top
of that do you have kind of like a meta
tool that is over the top of all these
processes and tools that you use to
coordinate between people or was it just
knowledge of the process I messaged them
and say let go of that tile no no no so
well some there was some of that but but
no we we generated all sorts of data
caches based on the state of the world
at different pretty much every day so
people are supposed to check in their
stuff when they go home and and whether
they do it or not because we actually
grab that data per tile we would we've
just this is the latest state of the
tile as it's checked in and then we
would we would create that cache data I
mean so x-rays here but we would create
that you know we could do it every day
but as often as we needed to have an
updated state to run the processes we
needed so but as far as yeah and you
know perforce and and checking
dependencies or having watched you know
having a watch daemon that's checking to
see is this ready from
to go it's just people were supposed to
be done at certain amounts of time and
then that brought the processing would
happen and if they weren't well because
at the end of the pipeline we were able
to sort of run everything in parallel
tiles in parallel then that's how we
would treat it so then that we that we'd
get the that updated tile I would
process that tile and its neighboring
tiles or I have that tile inherit its
neighboring tiles properties right so
the procedural systems would use which
neighbors instead of what was originally
authored in that tile just so we're all
blend in yes does it work yeah hello I
have a two-part question um first part
how did you deal if you had to with
complex intersections like 5y
intersections or very close together
intersections and how did you deal with
if you had to like overpasses and like
underpasses
radius and yeah so the system would if
if it knew about the the street itself
it would try to figure out a solution so
you'd have cars going driving right
through each other and and then we would
they would a dedicate well it was just
when our production support guys who did
a bunch of technical tasks he would go
through and clean up those intersections
so I would I mean it's another part of
this process that you could just go
through and kind of hand author in a
different way in our editor and then it
would get processed by the system then
added to this system and then preserved
so anything that was like so a crazy
intersection where it's like we can't
support those right those left turns
there cuz they're gonna cut in another
traffic we don't have enough track we
might have to traffic faces right east
and east west and north and south so is
there a way we can make it work alright
so we can't we have to exclude this from
being happened so we're gonna delete
that path and they would also the ones
they needed and then you could we knew
it was manually authored and that would
be preserved and then that's how that
stuff was happened happened and the same
thing with overpasses so a Grand Central
Station I mean pretty much to author
author any path you just draw a line and
then you give it the
data which was mostly by naming
convention of what you want it to be and
then my system automatically integrated
it in okay but did you have like roads
going over each other or was it 100% 2d
now there were over there the grand
social station has that raised deck and
of course we also had the bridges where
you so for the bridge pretty much they
authored they clicked on the ground here
and they drew a line they went click
growing up the bridge and then they
clicked and I think made it come back
the cars go up the bridge and then
they've turned around in the middle and
they come back okay thank you very much
the pedestrians actually go down the
subway they go under the street and then
they come out the other side anyway yes
yes thank you for the talk so did you
have any art driven rules for instance
in terms of color palette color
variation or illumination of certain
parts of the town I yes yes art drives
at all so we had a lot of there was a
lot of there's a lot of vertex paint
driven stuff and vertex color so like
something like like the puddle so in the
wetness of the streets so that was all
built with by building the vertex colors
into the models themselves so that was
already static but then artists could go
through in vertex paint and so create
this other cloud of colors that could
modify all that and if it was on
something on the street the optimum
thing to do was actually read their
vertex paint integrate it back into the
street so they could concurrent control
it that way as far as lighting and
lighting directions stuff I mean pretty
much I mean you can see x-rays talk but
pretty much the only thing was that was
any alley alley lighting but pretty much
we were just placing this stuff out
there but yet we're I mean we're always
the the rules include any artistic
artistic choices that we could follow oh
thank you hi hi okay you let's just talk
of a sunset overdrive your colleagues
suggested a hex grid for organizing yes
and the streaming a large procedurally
generated city but
I it's very inspiring and very useful to
us but I noticed that you go back to
regular square grids yeah yeah tell me
the story and the reason behind this
okay
so the hex grids were originally
designed because our procedural system
was going to use a a premade set of
puzzle pieces in order to do all the
roads in layout but once we started
having all the crazy elevations we it
was impossible to have a fixed set of
pieces to do all the work so we use a
procedure assistance to make all custom
custom streets so the hex grid actually
allows you a lot of flexibility when
you're using those pre-created pieces I
mean New York sort of built on a grid so
it I'm gonna close this New York sort of
built on an on a grid so using the grid
system for streaming regions kind of
just made sense but the hexes did not
make sense for for doing New York City
oh I can do this thank you
so we can move to a a wrap up room and
if you could tell me where that would be
tell everybody where that would be