This post is a transcription of 10 for the Chairman: Special Edition, material that is the intellectual property of Cloud Imperium Games (CIG) and it’s subsidiaries. INN is a Star Citizen fansite and is not officially affiliated with CIG but we reprint their materials with permission as a service to the community. Enjoy!
As is with any information on our transcripts and summaries, everything posted is subject to change by CIG and in some cases may not always be 100% accurate at the time. While we strive for accuracy, mistakes do happen so please let us know if you find something amiss that we didn’t catch. Enjoy the show!
10 for the Chairman: Special Edition Transcript
Chris Roberts (CR): Welcome to a special edition of 10 for the Chairman, I’m here with my guest Josh Herman, Character Art Director who’s based here in Los Angeles, but he’s the head of the global character pipeline and we’re doing a slightly different format. We’ve decided to try and switch around the 10 for the … show so that we ask questions specifically about something we want to talk about and then you guys come back with questions and we pick what’s the most popular and the ones that are suitable for the topic. So we have 10, the topic was about character and more specifically heads and facial stuff.
So we’re going to get to that, but first I’d like to say thank you to Davidian who sent these very nice little teddy bears of Squadron 42, mines got a little name on his back “Chris Roberts”, very good. I think there’s quite a few more of these that have showed up. Now I just need to make sure I put this here where he doesn’t collapse onto the table and he can listen to our answers. Sit down Teddy.
[Teddy falls over]
Josh Herman (JH): [Laughs]
CR: Not very good. Alright one more time, come on teddy sit correctly. There you go.
CR: See, not too much abuse of teddy bears as you can see. Alright first question is from…
[1:23] TheDeadlyKenny, Ghazlawl, BaconofWar, Kenzi Snow & Steve Hunter ask: So we know that character heads will be scanned in, however will we be able to manipulate these scanner versions to customize them to create our own character or will we only be able to select a specific set? For example facial features, body morphology etc.
CR: A couple of things. We have two levels of heads: One we scan a whole bunch of our actors that we have while we did the Squadron 42 shoot and also a bunch of Foundry 42 folks and Imaginarium folks. So I think we have over about 160 heads or so already scanned and are in various process of being brought to final quality and we have a plan to modulate and let you change and build your own player head that’s not a preset one.
So our plan isn’t really to say oh you can have one these eight male heads or one of these eight female heads. You pick sort of a base head, maybe combine a few of them and alter the various features on it. Your hairstyle, skin colour, eye colour so the general stuff sort of adapted around.
So I think you’ll be able as a player build any kind of face that you want. I think a little later on there’s another question that’s relative to this that asked this question, but the way that we have it right now is we have 18 sort of I guess player heads? Which are evenly split between male and female.
JH: Male and Female
CR: And so those would form the base, nine type of heads for male and nine types of heads for female and then from there on you would customize it.
JH: From there you could take pieces from the other ones that we scanned and you can create your own custom one based off of that.
CR: But you can’t have Mark Hamil’s head or Gary Oldman’s head.
JH: No, definitely not.
CR: Okay so that’s the first question. Second question comes from…
[3:38] Lt. CDR Draco & Tsukabi ask: We’re both wondering about hairstyles and how customisable they will be?
JH: So we’re gonna have a lot of different variations of hairstyles. We’ve already scanned like we said 150 actors and so each one of those is going to have a lot of different variations. You’re not going to be able customize it in the same way like you’re not going to be able to comb one hairstyle that granular, not going to be able to do that, but you will have a huge selection both in style, parting, lengths, colours.
CR: Yeah, besides specific hairstyles that we have chosen for actors and we have some that are specific to the actors, but we also have again like a set of NPC hairstyles and so we have the various you can be a skinhead, you can have a mohawk, you can have a ceaser, you can have a pony tail and all sorts of different hairstyles. So if you think of a hairstyle out there, I think we’ve got…
JH: We’ll have something from out there.
CR: We have an archetype for it that you can choose and you can choose the colour of your hair and then you can put that on the end, so basically the base would start with an archetype head then choose what kind of style hair you want, what kind of hair colour you want, eye colour, maybe what skin colour you want and on top of that you can start to adjust facial features. So it won’t be too dissimilar to a lot of sophisticated avatar creation setups in other online games or even other single player games.
[5:10] Maligor asks: Will the amount of detail in the character creation let me make my own doppelgänger?
CR: So what do you think Josh? Do you’ll be able to make a doppelganger?
JH: I think with the amount of heads that we’ve scanned you’ll probably be able to find somebody who has eyes like yours or a mouth like yours or different facial features like yours. So with that you could probably like [Finger Quotes] Mr. Potato Head, like put those pieces together and find something that looks kind of like you.
CR: So you should be able to try to get close. I mean I do sometimes when I play games try to make the character look like me. I don’t necessarily succeed all the time, but you never know.
[5:51] Hearts asks: Character heads as in the modeling of the heads? If so, I’m curious about how long the process takes to model an in-game render of someone’s face.
CR: So I guess they’re trying to qualify the question that we asked when we were soliciting these awhile back.
If so, I’m curious about how long the process takes to model an in-game render of someone’s face.
CR: For what we do we always, currently anyway, we don’t model it just from scratch artistically. We scan a person, an actor, fully. We have a whole rig of 50 plus cameras that go around the head, and we, essentially use photogrammetry to digitize and take a 3D snapshot of both the geometry or the sort of 3D mesh of the head as well as what the face textures would be. Then we take that, and we do a lot to it after that. So I don’t know if you want to talk a little bit about that.
JH: Yeah after that we kind of… Especially for the some of the main actors who do full facial capture so they’re getting blend shapes on specific ways that they talk. So that adds a whole another kind of layer. So that’s a really difficult thing to do if you’re just going to be sculpting a head from scratch, ’cause I think our characters have like some other upwards of 400 blend shapes.
CR: Yep. Over 400 blend shapes.
JH: So that would be 400 individual sculpts that we would need to make to match what we’re doing. So that’s really not an approach that we would want to be taking to get the realism that we want, especially for that actress that we have.
CR: We work with a … there’s a company called 3Lateral who are sort of quite well known for their headrig and head animation technology and sort of pushing forward the field of scanning live, real people and bringing them into the 3D world. We’ve been working with them for quite awhile. They’ve also in the past, you know they did Rise, for instance, back for CryoTech which is in the CryEngine, and our headrig, of the headrigs out there, is you know we’ll talk to it in a bit, but the by far the highest fidelity …
CR: … headrig that they’ve done. I’m thinking it’s probably higher fidelity or up there with anyone else’s headrig, but you know so it’s a big process. So if you go in and do a full set, an actor will set in this whole scanning set up and we will take 78 facial poses for them. So you know it’s sort of like things kind of like you [Scrunches Nose] scrunch your nose, or [Raises Eyebrow] raise an eyebrow …
JH: [Makes Faces]
CR: … [Opens Mouth & Sticks Out Tongue] opens mouth. And you have to do that and each one takes about a minute to collect all the data from all the cameras, builds it on the computer, and you then take another one and you do it. So it can take anywhere depending on how quick or slow the person in the seat is, an hour to two hours to do the full 78 pose set, and that builds and that basically gets built down by a lot of the techniques that 3Lateral are doing to this sort of 300 pose, blendshape setup and then of course on top of the 400 pose blend shapes. I think about 440 actually for our higher tier characters. It also has bones underneath that like, I think it’s something like 175 joints.
JH: Yeah, and that also drives specific maps so when they are doing those specific faces, [Wrinkles Forehead] like the forehead, it drives wrinkle maps and animated diffused maps for our higher influence as well.
CR: So it’s not just blend shapes, …
CR: … it’s not just bones, it’s a wrinkle map which is sort of telling the shader kind of what parts of the skin. So that’s almost like …
JH: Yeah. [Wrinkles Forehead]
CR: … like a bump …
JH: Like the wrinkle information you see here.
CR: … yeah almost like it’s a bump map and then the animated diffuse is sort of showing like how your color changes in your skin when you scrunch something up …
JH: Yep. Maybe you’ll get a red or a white. Yep.
CR: … Maybe you’ll get white or maybe you’ll get red or whatever it will be. It’s a pretty long process but after even doing that we bring it in and then we actually do sort of an artistic pass.
JH: Yeah, kind of like a makeup polish, polish or a makeup pass in a way, ’cause a lot of when you see those actors getting scanned it’s, you know. You usually have to scrub their face and so you can get these dots and you get really accurate information of what they look like, but that’s also not what they would look like on the a film set, because they don’t have makeup, and you know when you go and film sometimes. Most of the times you have like a very specific people that just do that. Right?
CR: Yeah, you don’t look like … you don’t look like you woke up in the morning.
CR: Everyone makes you up to look really great in a film …
CR: … light you properly, and so when we actually scan the actors we don’t want any makeup on them, because it’s best for how they sort of build the 3D head.
CR: So then we have this process that we bring, the head in and then we sort of do our own sort of makeup pass on the character and set it up in such a way that they’re going to look like a movie star playing … running around in our 3D world. So that’s part of our process. So we do all of this kind of assembling of the data that we scanned and then we bring it in here and do our own pass on top of it …
CR: … to sort of take it up to the extra level. We really work on things like the eyes, the tongue, the teeth.
JH: Eyes. Yep.
CR: And we still have a whole bunch of additional graphics work to be done. So the eye shader is going to get a lot more work.
JH: The hair shader.
CR: The hair shader is a big one, so that’s our probably weakest point now. So a lot of the images you’ll be seeing during this 10 For the Chairman, we don’t have what our intended hair shader is going to be, but our hair shader is going to be to the level of the best hair shaders you’ve seen out there in real time games, and then we’ll also work a little on the skin shader too. So, our goal is to sort of when the characters, you know when you play Squadron 42 for instance, the level of character tech and visual level will be as good or better than anything out there. So that’s what we’re shooting for. We’re working really hard on it. We’ve got a lot of talented people working on it. We’re not there yet, but I think you’ll be seeing in this particular episode sort of some stuff along the way, and it’s only just going to get better.
[11:27] Perry Hope asks: Wouldn’t it be nice to get some faces from your backers? Is that even possible? Can you tell us what is the hardest part of character creation and why?
CR: We just talked about how… the process we go through or scanning so…
JH: Pretty in depth scan.
CR: It’s pretty in depth. So, we definitely could scan some backers out there who would have to have a full rig setup. Right now we have the rig set up at Ealing Studio, takes awhile to set it up so we have to sort of, kind of semi permanent set up at Ealing so that’s where we do all the Squadron 42 shooting and so when we bring an actor in, we shoot with them and also make sure we scan them. At the same time if we’ve got Foundry 42 crew people or people from over here in LA or people from Imaginarium and they want to appear in the game, we scan them too. So, not out of the realm of possibility to have a set up somewhere and a few lucky backers…
JH: Bring some people in.
CR: Could be scanned and we could put them in the game and there is a reduced sort of scan set up we do if you’re not going to be up front doing lots of talking, more like sort of a background player in a movie, you know.
JH: Kind of like an extra.
CR: Yeah, an extra or something. Which is instead of 78 poses, I think it’s something like 18 or 16 poses… I can’t remember off the top of my head but it’s a much reduced set up because we don’t need to do quite as much because we don’t build the full 400 blend shapes setup for them. So, there is… that could potentially happen although you know requires a lot of work to do it at the fidelity we’re putting in this game because even our background characters have to stand up to…
CR: The quality that we do for our main characters so, you know, we’ve talked about it. Haven’t made any decisions, we may do something like that for fun in the future. We’ve also looked at some standard kind of techniques you could do like you know for instance, you can take a webcam and look into it and go here and go here and probably map your face there.
JH: Your face, yeah.
CR: The question we would have is, is it going to look weird at that level and…
JH: The question you run into with that self, you know, homemade iPhone or whatever photogrammetry is that the lighting condition and fidelity. So, we’re just talking we want all our characters to be able to stand next to each other. So you should be able to stand next to as a player character next to Gary Oldman or any of the top tier characters and if your lighting is weird when you’re taking your pictures, it’s going to give us wrong information and the texture’s might not be as good quality. It just creates the scenario where one can look worse than the other and that’s not really what we want.
CR: Yeah, we don’t want to pull you out of the game because someone looks like they’re shot with a really cheap camera.
CR: Not great lighting and everyone else looks like a movie star. So, you know, I guess to be determined. It is possible but to do it at the level we would like would be difficult.
[14:11] Coopernicus asks: How soon will we begin to see the heads on the npcs or players in the PU? That would make for some nice variety. And how editable will they be once implemented for character creation?
CR: Okay so the next question comes from Cooper … nicus. I guess.
JH: I think it’s Copernicus.
CR: Coperni … Well, but it’s got two o’s.
JH: Cooperni … Coopernicus.
CR: Coo … pernicus [Laughs]
JH: Yeah, I think so.
CR: How soon will we be able to see the heads on the npcs or players in the Persistent Universe? that would make for some nice variety, and how editable will they be once implemented for character creation? We are working on getting a variety of heads into the Persistent Universe.
We’re actually working on some of that stuff now, because we want to mix it up some and obviously we’re, hard at work on Squadron 42, cause as you can appreciate with the … I mean … In Squadron 42 we actually have about 350 speaking parts, which is an insane amount of speaking parts. Like …
JH: That’s a lot.
CR: … In most movies would have maybe 30, 40 speaking parts or 50 speaking parts. You know, it’s a lot, and so each one of those will obviously need their own unique head. So a lot of it’s going to be supplied by the custom actor’s scans, but some of the stuff will be also done sort of background characters where we would almost assemble the heads kind of like how you would assemble a player head.
JH: Much like the character creation that we were talking about earlier, so it is kind of one of those things where once we’ve got that system in place for Squadron for the background and for all the speaking roles, it’s really not much more to get it over to the PU as well, ’cause it’s, basically the same system.
CR: Yep, so, I would say not in 2.5 and, not … we’re going to try and get a few things in for the next release … when the Star Marine one, but you know as we move on to widening out the Persistent Universe with procedural planets and there’ll be some more landing locations we’re going to get more variety in there, and then of course Squadron 42 is going to feel like a movie. Everyone’s going to feel like their own cast member wandering around. It’s all going to be very high quality and very cool. So, but yes we are very focused on that. So we’re sort of feeling like we’re coming over the hump of getting the tech kind of place where we’ll be able to mix and match some stuff. We’ve still got some more stuff to do on the kind of morphing/build-your-own-face stuff, but just generally having a wide set of npcs to be able to use is very close now.
[16:30] L’Mac asks: Will the faces we see on our characters, other players, and regular NPC’s be the fidelity we saw on the character in “Pupil to Planet” (Or better), or more simple like the characters in the Morrow Tour video?
CR: Okay so forget the Morrow Tour, that was a whole combination of things, bad lightning, broken shaders, quick assets that hadn’t gotten through any of our pipeline. So you’d be interested to know the character in Pupil to Planet to a character called Joaquim Steiger in Squadron 42 scan of the actor.
What you saw in Pupil to Planet was an untouched up head and that was a Tier two character. Our rating is at top is tier zero, so that is what we used for Gary Oldman, Mark Hamill, Liam Cunningham, Mark Strong, they’re all tier zero. Then tier one is the next level so actually the actor Steven Bustling who’s playing Marrow is a tier one head and then the next one down is a tier two which is what Joaquim Steiger, and then we have a tier three head and that’s basically our ratings, but all of our heads have pretty much similar fidelity and they all have to stand up next to head other because you could be standing in a conversation with a tier zero, a tier one and a tier two and really the big difference between the tier one and the tier one’s and the two’s is the tier zero just has more facial density in the mesh. So for instance we use 40,000 polygons when you’re up close to a tier zero. Where as a tier one, tier two, and tier three are about 10,000.
JH: Yeah, Much less.
CR: Much less, but 10,000 up until now, the top level of triple AAA, up close, cinematic head character.
JH: Even our bottom level is the top level of most other games.
CR: You’d have to look close to see really that much difference. So we do it for the Gary Oldman and the Mark Hamill’s of the world because we really want to capture every single nuance of their face when their performing and talking in motion and so in those ones for the main characters you’re right up close to them and you want have some emotion to it, we’re just going for a little more fidelity in the character, but you know a tier zero and a tier one almost have the same number of blend shapes, almost the same number of bones, and so outside the mesh itself having just more density of faces, it’s pretty much the same and so the difference is really just… the wrinkle maps also give you a lot of detail, but maybe just a kind of little detail.
JH: A little bit of subtle bits in the animation I think is going to be some of it. The texture sizes are all the same between all the heads so that particularly doesn’t change much. Certain characters like you mention, Mark Strong has a very specific way that he speaks. So he kind of like me speaks out of mostly one side of his mouth so you really want to capture that, right? And that’s because he’s a Tier 0 so you really want to get that across.
CR: Yup so like I said, no Morrow’s Tour, that’s the past. We’ve moved on and done quite a lot and you can see in this video some of the differences for what it is and like we were talking earlier, there’s going to be a fair amount of improvement even from here in terms of what we’re doing with the shaders. So it’s generally the way it works in development is you think something looks good and then next year it looks better and the year after it looks better, you can just look at the Hornet or the Bengal carrier that we had when I launched the 2012 crowdfunding campaign and I thought that was awesome and I think a lot of other people did, but every year I see stuff moving even now, if you’ve just seen the video with Chris Smith’s new Hornet or the stuff that Nate was showing with the Bengal, it’s like a whole another level.
JH: Totally yeah.
CR: And that’s kind of where we’re pushing on the heads, but yeah what we want to do is we want you to feel like you’re in say Squadron 42 or even in the PU, having a conversation, interacting with NPCs and the quality of the character animation, the facial animation is what previously you would have experienced in a pre-rendered cinematic and so that is a goal to really try and put you in the story so to speak.
So that’s what we’re shooting for, I think we’re going to get there, but it’s definitely taking a lot of hard work. A lot of good people doing it like Josh here and his character team, we have a whole facial animation team over in the U.K. and a whole really talented cinematics team in Germany.
[21:07] Fathom asks: Will animations adapt to our character as they get older or recover from past injuries? For example, if my character is lucky enough to reach old age, will he or she take longer to stand up after sitting or perhaps start to walk with a limp having been shot in the leg earlier in the game?
CR: So, we’re definitely planning to do some animations that is specific to injuries and stuff like that. You know, if you get shot in the leg, shot in the arm we’re actually planning to have that sort of impact reflect on the character’s animation. We haven’t specifically thought about you aging as a character, it’s actually a very cool suggestion. We do and we have done in Squadron 42 is specifically capture different walking and running sets of locomotion, sets for the different actors because obviously someone like… let’s say Mark Hamill who’s a little older will move differently than, you know, a 20 year old.
JH: Than a younger Mark Hamill.
CR: Yeah, a younger Mark Hamill. So, you never know, we’re definitely going to have ‘older’ person locomotion, walking around NPC animation sets. We’ll have younger ones, so at that point it may actually not be that difficult to sort of move you through those but we haven’t got the inputs yet but I think that’s actually a pretty cool idea so…
JH: It’s a really cool idea.
CR: You never know in the future.
[22:31] Kyran asks: With a large number of incredibly detailed characters at once, both NPC and player, what’s the design plan to keep this from blowing up client resources within the persistent world game?
CR: We’ve thrown a bunch of numbers around like the tier zero’s got 40,000 polygons for the face. It’s got 440 blend shapes, the mesh itself I think with all the blendshapes is about 300 megabytes without LOD’s.
JH: Really, really expensive before.
CR: Yeah and so you could have maybe one of two heads on the screen and we had like 44 different areas of blended wrinkles and blended diffuse and the texture was really high, so we used to have 96 megabytes of blend textures which is quite a lot, and then we had the diffuse normal spec, transmittance, and the scattering mask which were all these extra layers that we baked the stuff in.
So we had about 150 megabytes of texture memory per head at the resolution we’re doing at for these characters, that’s all. That was a big focus of the graphics crew and how to bring that down. So we brought both the mesh data down and also the texture data down. So we basically reduced the 300 megabyte mesh stuff down to about 30 megabytes. So that was more about the base mesh and how we were going to delta it and some clever compression stuff they’re working on.
We also took the cost of all the textures down, you know from some I think for, we did a special thing for the wrinkle and the blend maps and stuff where we went from 72 megabytes just for those down to six. So essentially we’ve done a 10x saving which makes it much more viable to have a lot of these heads and that detail in the game which is good and then the last sort of segment that worked on was of course, when you have 440 blend shapes and I think there’s something like 175 joints or whatever, I don’t think I put this on my cliff notes, but that’s a lot of control.
So we have the blend shapes, we have the joints and I believe there’s also some other controllers for the wrinkle maps and the diffuse map, so in all ended up with very close to 1000 controllers to animate a face so that’s kind of the equivalent of animation a 1000 joints and if you have 10 characters, that’s 10,000. If anyone knows about game animation that’s a lot of joints to do.
JH: A lot
CR: So one of things that’s quite nice is we’ve been working with Three Lateral and they have a rig logic that has a lot fewer inputs that’s sort of more about the facial poses of position and that gets translated into where to move what blendshape to fire or where the joint is, or what normal map or diffuse maps, wrinkle maps firing, and that’s what we call the runtime rig technology and it’s also very critical because we use that, that’s how we’re going to be able to have all these different heads and faces and be able to blend them and stuff. It doesn’t matter, we can play the same dialogue on whatever head, doesn’t matter. It can be Mark Hamill, Gary Oldman, it could be a custom character one because it goes through this rig logic that takes the inputs and puts it out to the appropriate blend shape, joint and everything like that.
That also has the advantage of… it has about 183 inputs versus about 1000 controllers, so it’s a lot less data that’s going to go through in our animation data for the facial which is…
JH: Another big savings too.
CR: So those are all the things that we’re trying to keep the planned resources down and we you know, have a few other plans, but I think we’re going to be able to live a high quality that you would normally see in pre-rendered cinematics, but you would actually see it in the game, live, in real time.
[26:18] Felicia FasterThanLight asks: The two character design approaches I’m familiar with are the traditional choose from a number of presets approach or the newer, actually being able to edit vertex points within certain ranges approach like in Black Desert. Which approach is Star Citizen going to use? Or has the crew come up with an even more diabolical approach that will knock our socks off?
JH: I think it’s definitely a combination of both like we talked about earlier so starting with kind of a preset and you choose your face, your skin colour, your eye colour, your hair colour. Then being able to mix and match from all those other combinations of heads that we’ve seen. So, it really is a little bit looking like Black Desert stuff, you can definitely change the tilt of your cheekbone, you can move it around and you can do stuff like that but it’s not really actually bringing in other mesh or other character information.
CR: Yeah, I think if I remember when I was playing with it, you sort of selected the face, hair and then you modify it and you have some really nice stuff for like the hair and altering the hair and moving some stuff around.
JH: Yeah, they have some really cool stuff but it’s not actually from what I can tell, it’s not really drawing from fully another head, it’s just kind of pushing and twisting and altering.
CR: Yeah, I think one of the things that’s kind of cool that we’re going to try is we’re going to give you those tools but we’re also going to blend heads together so you can say, ‘oh, I like that guy and I like that guy or I like this girl and I like this girl and let’s merge her’. You know, maybe I’ll take the jaw from here and the forehead from here…
JH: Kind of that mix and match thing.
CR: We’ll blend them and blend there and that’s one of the reasons why this live real time rig technology is very important because we can do all this stuff and it doesn’t matter because it will all fall out of the sort of rig logic that gets calculated in real time. So, it inputs the animation, it doesn’t need to care where the eye position exactly is, it doesn’t care where the mouth is, it just needs to know that they’re mouth was moving and the eyes were moving around and then the logic for the rig sort of does it’s magic and goes through and changes it just on the appropriate rig. So, I think it’s going to be pretty interesting, we’ll see what happens.
CR: Well with that I think that is our 10 questions.
JH: I think that’s it.
CR: That’s all we had, and I’m sure you’ll be seen a lot in up and coming episodes of Around the ‘Verse.
JH: Probably yeah.
CR: Showing some more of the really cool character stuff that we’re doing here.
JH: Yeah I think we’ve got a lot really cool stuff and looking forward to showing it all off.
CR: Yeah no, it’s great! So I’m really psyched, to be building this beautiful world, it’s really cool.
JH: Yeah, it’s nice.
CR: Okay so thank you everybody. I’d like to thank subscribers for making this show and the many other shows we do possible because without your backing we wouldn’t have the resources and the extra staff that do all the video production stuff we do and obviously I’d like to thank all the backers out there for making Squadron 42 and Star Citizen possible so thank you very, very much.
It’s been fun, I hope you guys enjoyed the show and see you next time!