Discussions with industry experts from different industries
Industry Experts
So I had discussions with some industry people for design research 2. The problem with research part 2 is that you are restricted to the word count that you can present (200 words is not a lot per person).As I chatted to a few industry experienced people; I learnt a few interesting points and overall I gathered a lot of key info that can help me with both getting the right equipment and more importantly, getting into the right frame of mind to be a motion capture designer. As I wrote to all the people, my main point was to gather as much information as I could without triggering or going into NDA area's as I knew that would be a very big problem.
The 5 main questions I asked was mainly orientated around capture and process:
· When capturing, what techniques do you find useful to get a more efficient capture?
· What style of motion capture do you feel provides a better result if applicable?
· Do you feel you can get a much better quality capture from equipment from a much higher price tier than a lower?
· Do you use a specific external programs for the cleaning process such as motion-builder / Maya / Iclone 6 etc?
· How long does it efficiently take to clean a captured animation to the point where it is acceptable of use in games?
The following is the conversations with presentation of any following questions with an overall end feedback.
Demian Gordon - Motion Capture Engineer - Dreamworks Animation
DemianI find the best technique is proper experience. people often don't have the skills to do motion capture correctly and assume its simple or automatic. shooting things incorrectly makes things much more challenging. I prefer optical mocap to other types. I prefer systems that are robust and have been through a lot of production work. That tends to be the systems that are more expensive. I personally use Blade to clean mocap data and motionbuilder to motion edit. the length of time it takes to clean a move would depend on many factors. how well the system is set up, the number of cameras involved, level of occlusion, difficulty of motion, or number of people. on average a simple single person move with low occlusion shot with a system with adequate camera coverage and data quality is between 20 min to an hour depending on the length of the take. difficult motions can be from an hour to ...as long as it takes. Doing dogpiles with 11 actors on matrix took a week per file (as an example)
Byron
Thanks for the information. I can use this information to help with my research, especially the technique research and clean information specifically. Out of curiosity, can you direct me to anywhere that will help me learn useful techniques to help motion capture or is it more of a practise makes perfect? Many thanks.
Demian
There's some books but ya its mostly something you learn on the job
Joe Ponsford - Motion Capture Technician At University of Portsmouth
JoeWhen capturing data the calibration phase is pivotal to the success of the shoot, getting a good system calibration and also a good actor calibration before the client comes in are what’s key. As with a system calibration it dictates the accuracy of the cameras in 3D space in relation to each other indicated via an error threshold, this is done through a wand wave.
Typically you can use an active wand which will give you an extremely accurate system calibration however it doesn’t give you a true representation of how your system is going to perform as your shooting with passive markers. Also the actor calibration is key with Vicon Blade & Motion Analysis’s Cortex this dictates where the markers are in relation to each other via the co-variance, this allows the labelling system to work effectively.
Typically I would do this process before the shoot, during the shoot at lunch break and also after as someone could knock a camera or the suit can move as time wears on. As a studio makes it profit in the data cleanup not shooting therefore you want this to take as little time as possible. Also camera positions at varying heights are key as this gives you better coverage meaning less fills required.
The style of motion capture that offers the best is optical passive motion capture this is due to shooting at a higher fidelity in terms of fps and also accuracy. With video motion capture its cheaper however the results aren’t guaranteed and the user cannot shoot at a high fps. With inertial suits like Xsens and Perception neuron offer a cheaper alternative however there is an element of drift, it also involves a re-calibration process every 15-30 minutes which isn’t ideal if your shooting with a client as they require quick turnaround.
Differing between optical motion capture systems the camera’s all offer a similar result. With Vicon their software’s a lot more efficient so the data turnaround is quicker so more cost efficient. If you invest more in a motion capture system (etc more cameras) this can give you a better result as different angles so cleaner data. However too many cameras can be counterproductive as too many masks can mean more gaps in data.
Studios use varying software depending on the equipment their using typically its cortex, Blade or Motive. However you can use Motionbuilder if you write your own pipeline, Ikenima also offers cleaner results as its global optimisation solve and post results are impressive.
Tracking data can take a various amount of time depending on how long the take is and also the movement. If an actor is folding its arms in the volume or rolling on the floor this can take half a day up to a week. If it’s a walk cycle or simple motion where limited occlusion occurs then this can take 10 minutes to track the data.
Byron
Thanks Joe for the information! This is really interesting to know and brings up some really good areas for more research into. I do want to ask when doing the wand calibration with the Vicon, I'm guessing due to the calibration, time can be consumed if errors occur with the calibration test? I agree with the neuron system of the drift situation as after testing, I found that my data was incorrect with the arms / legs movements being off to what they are meant to be. If you don't mind me being intrusive, from the sounds of it it sounds like you are running a Vicon system?
Joe
Time can be consumed if there are more cameras used for the volume as this takes more time to compute, typically if the calibration is a high error threshold we would do a new calibration, or it can mean that a camera's f-stop (aperture) isn't set right so its not seeing the wand properly. This is indicated by the rating Blade gives each camera on calibration so we look to say if every camera says "Awesome" or "Excellent" any lower and we would do another one. Yeah we run a 20 T series camera Vicon system, combined between T and Ts (Half of them having daylight filters).
Games and movies are slightly different but I have done a considerable amount of both so its fine. Trying to get more efficient capture, that is to say, as close to pristine data as you can is about discipline - unfortunately in a working environment there are so many factors out of your control; however, when you can control them some things I keep an eye out for:
no one should be anywhere near the stage who isn't doing a job - unnecessary people on set should be in a specific area away from cameras so they can’t impact the shoot negatively.
Keep a constant temperature is you're using a scaff rig. it does affect your calibration if the rooms heat fluctuates wildly, as metal expands and contracts and we are talking about millimeter accuracy here. so also keep all doors shut once calibrated. we usually calibrate three times a day unless specifically called for i.e. a build on stage (sets etc) you would want a calibration with all the stuff in the volume as well as not if possible. Basically, make sure you get good calibrations this is paramount to having an easy time in post.
Ideally you want to be able to batch machine track the whole shoot without much human interaction. obviously, this can’t always be the case depending what you shoot. Hands down data quality and the best motion capture is optical no two ways about it. Vicon systems are top of the market right now with their brand-new vantage cameras and when Katana comes out it will be are much needed boost also (Katana is Vicons new software) if you were to make a movie or game professionally I’d wager that unless you're WETA or GIANT that you'll be using a Vicon system - the other are cheaper but frankly not as good, Optitrak for example, you'll get the same quality of data almost but it’s harder to obtain it and the post processing side is a letdown.
The style of motion capture that offers the best is optical passive motion capture this is due to shooting at a higher fidelity in terms of fps and also accuracy. With video motion capture its cheaper however the results aren’t guaranteed and the user cannot shoot at a high fps. With inertial suits like Xsens and Perception neuron offer a cheaper alternative however there is an element of drift, it also involves a re-calibration process every 15-30 minutes which isn’t ideal if your shooting with a client as they require quick turnaround.
Differing between optical motion capture systems the camera’s all offer a similar result. With Vicon their software’s a lot more efficient so the data turnaround is quicker so more cost efficient. If you invest more in a motion capture system (etc more cameras) this can give you a better result as different angles so cleaner data. However too many cameras can be counterproductive as too many masks can mean more gaps in data.
Studios use varying software depending on the equipment their using typically its cortex, Blade or Motive. However you can use Motionbuilder if you write your own pipeline, Ikenima also offers cleaner results as its global optimisation solve and post results are impressive.
Tracking data can take a various amount of time depending on how long the take is and also the movement. If an actor is folding its arms in the volume or rolling on the floor this can take half a day up to a week. If it’s a walk cycle or simple motion where limited occlusion occurs then this can take 10 minutes to track the data.
Byron
Thanks Joe for the information! This is really interesting to know and brings up some really good areas for more research into. I do want to ask when doing the wand calibration with the Vicon, I'm guessing due to the calibration, time can be consumed if errors occur with the calibration test? I agree with the neuron system of the drift situation as after testing, I found that my data was incorrect with the arms / legs movements being off to what they are meant to be. If you don't mind me being intrusive, from the sounds of it it sounds like you are running a Vicon system?
Joe
Time can be consumed if there are more cameras used for the volume as this takes more time to compute, typically if the calibration is a high error threshold we would do a new calibration, or it can mean that a camera's f-stop (aperture) isn't set right so its not seeing the wand properly. This is indicated by the rating Blade gives each camera on calibration so we look to say if every camera says "Awesome" or "Excellent" any lower and we would do another one. Yeah we run a 20 T series camera Vicon system, combined between T and Ts (Half of them having daylight filters).
Aaron Urquhart - Motion Capture Technician at The Imaginarium Studios
AaronGames and movies are slightly different but I have done a considerable amount of both so its fine. Trying to get more efficient capture, that is to say, as close to pristine data as you can is about discipline - unfortunately in a working environment there are so many factors out of your control; however, when you can control them some things I keep an eye out for:
no one should be anywhere near the stage who isn't doing a job - unnecessary people on set should be in a specific area away from cameras so they can’t impact the shoot negatively.
Keep a constant temperature is you're using a scaff rig. it does affect your calibration if the rooms heat fluctuates wildly, as metal expands and contracts and we are talking about millimeter accuracy here. so also keep all doors shut once calibrated. we usually calibrate three times a day unless specifically called for i.e. a build on stage (sets etc) you would want a calibration with all the stuff in the volume as well as not if possible. Basically, make sure you get good calibrations this is paramount to having an easy time in post.
Ideally you want to be able to batch machine track the whole shoot without much human interaction. obviously, this can’t always be the case depending what you shoot. Hands down data quality and the best motion capture is optical no two ways about it. Vicon systems are top of the market right now with their brand-new vantage cameras and when Katana comes out it will be are much needed boost also (Katana is Vicons new software) if you were to make a movie or game professionally I’d wager that unless you're WETA or GIANT that you'll be using a Vicon system - the other are cheaper but frankly not as good, Optitrak for example, you'll get the same quality of data almost but it’s harder to obtain it and the post processing side is a letdown.
I’d definitely say Vicon are worth the extra bucks if you can there are other examples of much cheaper mocap solutions such are xsense suits and other inertial based solutions. from what I've seen of the very best of these is that the data quality is a big step down- as there are no cameras there is no real-world translation. this stuff is good for live shows mainly in my opinion- I think there would be far too much clean up involved to make it game worthy animation, again in my opinion. I’d like to see the results if someone has done that though.
TLDR is- if you’re on your own making a mocap indie project get an xsense suit or something similar, if you have any capacity and space to do so you should set up a stage and optical system. Because I work with Vicon we use Vicon’s software for tracking - any optical system should come with post processing software also. however after tracking most people will be using Motionbuilder I think unless they have bespoke software ala WETA inertial systems would be in motion builder id imagine. Finally, THE question I am asked most by productions.
It entirely depends on how good the data is, how long the scene is, how complicated - rolling on the floor with props takes a lot longer than standing still doing a spoken piece. What I’d say is you spend about 75% of the time posting the fingers in a lot of cases and 25% on the rest of the body, so that's a really good call to make if you want to save time - no finger capture.
Byron
Thanks for the feedback! I understand that they are basic in questions but effectively it confirms a lot of my initial ideas. I do have a neuron suit and can confirm the work and data that is accumulated is sporadic and can breakdown very easily!
Aaron
We, as in the Imaginarium studios used an xsense suit for the live real time required for the RSC's production of the tempest which was really cool and largely successful - the key here was a calibration at every given opportunity - calibrations became costume changes for the actor. I think we've tested a neuron suit as well and decided the xsense was the one to use of the two if that's helpful to your future purchases.
When capturing, what techniques do you find useful to get a more efficient capture?
I find that one of the most important factors in getting the best out of any motion capture system is in the initial set-up. Get that right and your one step ahead of the game to begin with. The same goes for the marker setup and calibration of the actors. Ensuring that the actors remain within the specified volume is also a real help. Planning is everything when it comes to Motion Capture, from pre-prepared shot-lists that define the use of the space to rehearsal time to uncover any potential issues relating to the shoot.
What style of motion capture do you feel provides a better result if applicable?
From my understanding of the question – an optical system has proven to provide us with the best results – as stated in the next answer, that is based on our needs alone and in no way, represents that it is the best all round. Different requirements may define a need for an alternative system that could be better suited to the job in hand.
Do you feel you can get a much better quality capture from equipment from a much higher price tier than a lower?
We use a Vicon Optical system and have done for a very long time – we have looked at other systems in the past and find that for our purposes – we have the best system for the job. This is not to say that all other systems are inferior as it is a very contextual choice to opt for what we have in place. However, we have found certain elements of other systems do not work as efficiently or give such good results in certain circumstances.
Do you use a specific external program for the cleaning process such as motion-builder / Maya / Iclone 6 etc.?
Vicon have their own proprietary software for cleaning up the captured data. Any second pass on the data is handled by the animators, usually in MotionBuilder in the first instance and / or Maya for any finishing touches. This is based on our current pipelines, though – so isn’t a defined route.
How long does it efficiently take to clean a captured animation to the point where it is acceptable of use in games?
This would very much depend on the animation captured…? An idle may require very little or no clean-up, whereas a complicated death sequence might take a lot longer.
We are in an excellent position of having our own in-house Motion Capture Studio and if we capture anything that is taking longer than is deemed sensible to clean-up a take – we’ll just add the animation onto the next shoot to recapture
Byron
Hi Peter, you confirmed a lot of the things that I calculated and researched myself. Regarding the Vicon system, you're right on the system, I know that using lower end (Neuron suits) it's temperamental and the system does drift a lot which is annoying.
Out of curiosity, from your own personal practice, can you direct me to anywhere that will help me learn and improve my techniques to help the capture and cleaning or is it more of a practice makes perfect?
It entirely depends on how good the data is, how long the scene is, how complicated - rolling on the floor with props takes a lot longer than standing still doing a spoken piece. What I’d say is you spend about 75% of the time posting the fingers in a lot of cases and 25% on the rest of the body, so that's a really good call to make if you want to save time - no finger capture.
Byron
Thanks for the feedback! I understand that they are basic in questions but effectively it confirms a lot of my initial ideas. I do have a neuron suit and can confirm the work and data that is accumulated is sporadic and can breakdown very easily!
Aaron
We, as in the Imaginarium studios used an xsense suit for the live real time required for the RSC's production of the tempest which was really cool and largely successful - the key here was a calibration at every given opportunity - calibrations became costume changes for the actor. I think we've tested a neuron suit as well and decided the xsense was the one to use of the two if that's helpful to your future purchases.
Peter Clapperton - Animator / Motion Capture Manager at The Creative Assembly
PeterWhen capturing, what techniques do you find useful to get a more efficient capture?
I find that one of the most important factors in getting the best out of any motion capture system is in the initial set-up. Get that right and your one step ahead of the game to begin with. The same goes for the marker setup and calibration of the actors. Ensuring that the actors remain within the specified volume is also a real help. Planning is everything when it comes to Motion Capture, from pre-prepared shot-lists that define the use of the space to rehearsal time to uncover any potential issues relating to the shoot.
What style of motion capture do you feel provides a better result if applicable?
From my understanding of the question – an optical system has proven to provide us with the best results – as stated in the next answer, that is based on our needs alone and in no way, represents that it is the best all round. Different requirements may define a need for an alternative system that could be better suited to the job in hand.
Do you feel you can get a much better quality capture from equipment from a much higher price tier than a lower?
We use a Vicon Optical system and have done for a very long time – we have looked at other systems in the past and find that for our purposes – we have the best system for the job. This is not to say that all other systems are inferior as it is a very contextual choice to opt for what we have in place. However, we have found certain elements of other systems do not work as efficiently or give such good results in certain circumstances.
Do you use a specific external program for the cleaning process such as motion-builder / Maya / Iclone 6 etc.?
Vicon have their own proprietary software for cleaning up the captured data. Any second pass on the data is handled by the animators, usually in MotionBuilder in the first instance and / or Maya for any finishing touches. This is based on our current pipelines, though – so isn’t a defined route.
How long does it efficiently take to clean a captured animation to the point where it is acceptable of use in games?
This would very much depend on the animation captured…? An idle may require very little or no clean-up, whereas a complicated death sequence might take a lot longer.
We are in an excellent position of having our own in-house Motion Capture Studio and if we capture anything that is taking longer than is deemed sensible to clean-up a take – we’ll just add the animation onto the next shoot to recapture
Byron
Hi Peter, you confirmed a lot of the things that I calculated and researched myself. Regarding the Vicon system, you're right on the system, I know that using lower end (Neuron suits) it's temperamental and the system does drift a lot which is annoying.
Out of curiosity, from your own personal practice, can you direct me to anywhere that will help me learn and improve my techniques to help the capture and cleaning or is it more of a practice makes perfect?
Peter
It's definitely hands on experience that will give you a greater benefit - I remember when I was first handed over the MoCap system and I knew absolutely nothing about it - learning resources for this kind of thing were very few and far between and I have found that overcoming the obstacles and having no choice but to find a way of making something work, for me, is the absolute best way of gaining knowledge in this particular field... Making mistakes and making them only once...! But that could just be my favoured choice in learning style...!!
• When capturing, what techniques do you find useful to get a more efficient capture? There is no self-called technique. The basis of Motion Capture is accuracy, so you have to be very accurate and organised to produce the best results. Always keep in mind what you actually are capturing (the dynamics of a movement, not a collection of poses, meaning that what you are capturing is actually what's between the poses) and that all the work is done by legacy (if the rig is bad, no matter how you capture the results will be bad), the same with the tracking, the cleaning, the character mapping and the motion editing: a tiny error at the top of the pyramid becomes a huge mess at the end.This is what I call the "Garbage In-Garbage Out" rule. You have to be extremely aware of where you put the markers too. You don't put markers "somewhere there" or "around that area": putting a marker one inch from its ideal position could reproduce a complete different dynamic or worse, reproduce the dynamic of the wrong body part.
• What style of motion capture do you feel provides a better result if applicable? Without hesitation, optical Motion Capture is the best these days. Magnetic is old and obsolete due to physical constraints (no piece of metal around, lack of accuracy, one performer only...) and acceleration sensors based Motion Capture does have mathematical limitations which are acceptable for video games prototypes and/or playing around but can't compete optical Motion Capture in terms of accuracy and today's video games needs (datas amount, interactions between performers, number of simultaneous performers...)
It's definitely hands on experience that will give you a greater benefit - I remember when I was first handed over the MoCap system and I knew absolutely nothing about it - learning resources for this kind of thing were very few and far between and I have found that overcoming the obstacles and having no choice but to find a way of making something work, for me, is the absolute best way of gaining knowledge in this particular field... Making mistakes and making them only once...! But that could just be my favoured choice in learning style...!!
Jérémy Meunier - Motion Capture Supervisor at Game On
Jérémy• When capturing, what techniques do you find useful to get a more efficient capture? There is no self-called technique. The basis of Motion Capture is accuracy, so you have to be very accurate and organised to produce the best results. Always keep in mind what you actually are capturing (the dynamics of a movement, not a collection of poses, meaning that what you are capturing is actually what's between the poses) and that all the work is done by legacy (if the rig is bad, no matter how you capture the results will be bad), the same with the tracking, the cleaning, the character mapping and the motion editing: a tiny error at the top of the pyramid becomes a huge mess at the end.This is what I call the "Garbage In-Garbage Out" rule. You have to be extremely aware of where you put the markers too. You don't put markers "somewhere there" or "around that area": putting a marker one inch from its ideal position could reproduce a complete different dynamic or worse, reproduce the dynamic of the wrong body part.
• What style of motion capture do you feel provides a better result if applicable? Without hesitation, optical Motion Capture is the best these days. Magnetic is old and obsolete due to physical constraints (no piece of metal around, lack of accuracy, one performer only...) and acceleration sensors based Motion Capture does have mathematical limitations which are acceptable for video games prototypes and/or playing around but can't compete optical Motion Capture in terms of accuracy and today's video games needs (datas amount, interactions between performers, number of simultaneous performers...)
• Do you feel you can get a much better quality capture from equipment from a much higher price tier than a lower? Definitively not! There was a war between cameras manufacturers some years ago about who would produce the highest resolution cameras. Vicon won with their 16 millions pixels cameras (T160 series): the result is that every shiny dust piece would produce a ghost marker if not enough filtered (and the tweaks were very sensitive). This is the best example that the most expensive cameras don't produce the best results or with a lot of time consuming tweaks. On the other hand, that's a complete illusion to think that you could produce accurate results with Kinect or GoPro cameras: that's not only a matter of resolution, that's also a matter of CCD sensor, the ability to filter out dots in real time, framerate etc... Professional hardware is expensive and there is a reason for that
• Do you use a specific external programs for the cleaning process such as motion-builder / Maya / Iclone 6 etc? For data cleaning, we're using Blade (which is being replaced by Shogun). Motion Builder can clean data but it is mostly used for motion editing (and cleaning tools are very limited). Maya or 3DS Max are trying to imitate Motion Builder in their tools but without success, Motion Builder is definitely the best to edit Motion Capture.
• How long does it efficiently take to clean a captured animation to the point where it is acceptable of use in games? As always, it depends! As we work in "legacy mode", everything is exponential, not linear. It means that the post-production is slow in its beginning (rig formating, skeleton creation, tracking, etc) but after that, a lot of things could be batched if you're smart enough and get super fast until the motion editing process. It depends also on the contents of the animation itself of course: an animation with a lot of self-occlusion and/or interactions would take longer to process. One last advice, never trust the self-called Motion Capture "experts": by tracking down their resume and track records, you can easily filter them out ;-) Unfortunately, there is a lot of them in our industry...even those with a nice track record but they have no idea what they're doing and a lot of outsourcing to erase their lack of efficiency.
Question 1 - The more you put into preparation and testing, the better the overall outcome will be. Spend time, aiming and focusing each camera to get the best range and exposure form every camera. Making sure the system software is calibrated for the stage space your using, controlling the light and the visibility of the markers you're tracking all help to improve the data being captured. Every pipeline is different, but the best ones are the ones that pay attention to details.
Q2 - After working in films, movies and commercials, I realised that there's no such thing as the best. It really depends on what and where you are shooting. There is a quality and functionality difference between very cheap systems and very expensive systems yes, however, if you get the right system for what you need, then "best" is subjective. Especially if you know how to work with it. Since I've always worked in AAA games and big budget films when it comes to mocap, I've always used somewhat expensive systems with robust tools to get through entire productions. When you shoot mocap for 160 days, you can't rely on cheaper, more personalised size systems.
Q3 - yes, for higher price, you are improving the quality generally. That doesn't mean that every project needs that quality. But it is a factor worth measuring.
Q4 - If anyone tells you that you can learn the best methods for pipelines from a book, they're just trying to sell you something. The best knowledge comes from experience. Pipelines adapt constantly, so having a book or class claim to teach you all you need to know is a falsehood. Get on a stage and work and learn and work and refine and work and learn...then repeat. Forever.
• When capturing, what techniques do you find useful to get a more efficient capture? Well, when it comes to Mocap and shoots in general, the number one thing to consider is preparation - its much easier and way more efficient for everyone involved if you know precisely what you want to shoot, almost to the second. Of course there are shoots where they want to experiment and try different ideas but everything goes much smoother if clients are prepped and ready, makes our job easier and their days shorter and sweeter :) get your actors in nice and early and suited up and ROM'd before the clients/director even get there if you can, then once they've arrived you can get right down to business. If you need props or environmental obstacles built, make sure to get the dimensions and information well in advance so you're not building it on the day, build it the day before and then you just have to pop it in the volume.
• What style of motion capture do you feel provides a better result if applicable? There's a few methods kicking about for Mocap - the most popular and reliable tend to be Optical and Inertial, particularly Optical which is what we specialize in. It gives the most accurate capture you can get due to it usually being done in a controlled capture volume where the temperature and lighting isn't likely to vary much. Due to Optical being infrared based, it can be tricky to shoot anywhere other than inside a set up volume but most hardware companies now offer cameras that can be used outside where the lighting conditions are certain to change and the cameras can adapt their calibration to the conditions. Inertial or 'Markerless' motion capture that is entirely sensor or suit based is becoming more popular for projects with a tighter budget and indie filmmakers, systems like Xsens and Perception Neuron are doing very well. They don't require a massive amount of setup or lots of expensive gear to run.
• Do you feel you can get a much better quality capture from equipment from a much higher price tier than a lower? Obviously if you have access to a large optical system the capture will be much more accurate and the detail in the data will be much higher, but also costs a lot more and has a lengthier processing time so requires multiple people working in a pipeline in stages whereas the more affordable inertial markerless stuff will give you dirtier/less accurate data but quicker - it will need a lot more cleanup in post afterwards though.
• Do you use a specific external programs for the cleaning process such as motion-builder / Maya / Iclone 6 etc? We use a variety of software for different stages of the pipeline - during capture we use Cortex to capture all the data from the cameras in real time, and Motionbuilder hooked up to Cortex to power client rigs for realtime previs - we also use Unreal Engine to further the previs quality if the client wants, we can also hook this up to the HTC Vive and allow the directors to be a part of the scene as it happens. The captured data is then tracked using some in-house tools we've developed as well as Cortex or Blade for Vicon data, this allows us to clear up any marker flips or confusion during the shoot - sometimes if people are fighting for example, Markers can come loose or become misidentified by the system for one that's closer to it etc... Once the data is tracked you'll have a file of marker co-ordinates which we can then build a bespoke skeleton set for - either to best fit the data, or to best fit the clients character rig they'd like the data delivered on as they may have custom joint setups and multiple spine bones etc... The skeleton is built and solved once again in either an in-house tool we have called solver interface or in Cortex/Blade. In tandem to this solver skeleton, we'll also characterise the clients rig in Motionbuilder so that the bones match up with the solver setup, and the data will know which bone to re target to once we import it into Motionbuilder on the rig.
• How long does it efficiently take to clean a captured animation to the point where it is acceptable of use in games? This is totally dependent on the type of game/client and whether or not it'll be game play animation or cinematic/cut scene animation. Some clients want data to be exactly as it was captured and do the cleanup themselves so we'll deliver untouched raw solves baked on their skeletons for them to polish up themselves in-house. Other clients want it 100% finished with full facial animation and finger/contacts/prop animation for cut scenes, this is called Motion Editing and allows the client to basically drop the delivered scene straight into the game engine as it is. We would give time estimates for this process based on scene length and character count, so its hard to give an exact answer for this one - anything from half an hour to a week!
Byron
I was trying to look into motion capture pipelines, do you any good locations or books I can get that'll teach me the proper pipeline processes? Overall though, the information you provided is very helpful! Thanks for the info.
Stu
In terms of pipeline stuff, this book was really helpful when I was first starting to look into Mocap when I was doing my degree, still keep it with me today:
• Do you use a specific external programs for the cleaning process such as motion-builder / Maya / Iclone 6 etc? For data cleaning, we're using Blade (which is being replaced by Shogun). Motion Builder can clean data but it is mostly used for motion editing (and cleaning tools are very limited). Maya or 3DS Max are trying to imitate Motion Builder in their tools but without success, Motion Builder is definitely the best to edit Motion Capture.
• How long does it efficiently take to clean a captured animation to the point where it is acceptable of use in games? As always, it depends! As we work in "legacy mode", everything is exponential, not linear. It means that the post-production is slow in its beginning (rig formating, skeleton creation, tracking, etc) but after that, a lot of things could be batched if you're smart enough and get super fast until the motion editing process. It depends also on the contents of the animation itself of course: an animation with a lot of self-occlusion and/or interactions would take longer to process. One last advice, never trust the self-called Motion Capture "experts": by tracking down their resume and track records, you can easily filter them out ;-) Unfortunately, there is a lot of them in our industry...even those with a nice track record but they have no idea what they're doing and a lot of outsourcing to erase their lack of efficiency.
Marc Morriseau - Motion Capture Stage Manage at Infinity Studios
MarcQuestion 1 - The more you put into preparation and testing, the better the overall outcome will be. Spend time, aiming and focusing each camera to get the best range and exposure form every camera. Making sure the system software is calibrated for the stage space your using, controlling the light and the visibility of the markers you're tracking all help to improve the data being captured. Every pipeline is different, but the best ones are the ones that pay attention to details.
Q2 - After working in films, movies and commercials, I realised that there's no such thing as the best. It really depends on what and where you are shooting. There is a quality and functionality difference between very cheap systems and very expensive systems yes, however, if you get the right system for what you need, then "best" is subjective. Especially if you know how to work with it. Since I've always worked in AAA games and big budget films when it comes to mocap, I've always used somewhat expensive systems with robust tools to get through entire productions. When you shoot mocap for 160 days, you can't rely on cheaper, more personalised size systems.
Q3 - yes, for higher price, you are improving the quality generally. That doesn't mean that every project needs that quality. But it is a factor worth measuring.
Q4 - If anyone tells you that you can learn the best methods for pipelines from a book, they're just trying to sell you something. The best knowledge comes from experience. Pipelines adapt constantly, so having a book or class claim to teach you all you need to know is a falsehood. Get on a stage and work and learn and work and refine and work and learn...then repeat. Forever.
Stu Whitten - Motion Capture Specialist at Centriod
Stu• When capturing, what techniques do you find useful to get a more efficient capture? Well, when it comes to Mocap and shoots in general, the number one thing to consider is preparation - its much easier and way more efficient for everyone involved if you know precisely what you want to shoot, almost to the second. Of course there are shoots where they want to experiment and try different ideas but everything goes much smoother if clients are prepped and ready, makes our job easier and their days shorter and sweeter :) get your actors in nice and early and suited up and ROM'd before the clients/director even get there if you can, then once they've arrived you can get right down to business. If you need props or environmental obstacles built, make sure to get the dimensions and information well in advance so you're not building it on the day, build it the day before and then you just have to pop it in the volume.
• What style of motion capture do you feel provides a better result if applicable? There's a few methods kicking about for Mocap - the most popular and reliable tend to be Optical and Inertial, particularly Optical which is what we specialize in. It gives the most accurate capture you can get due to it usually being done in a controlled capture volume where the temperature and lighting isn't likely to vary much. Due to Optical being infrared based, it can be tricky to shoot anywhere other than inside a set up volume but most hardware companies now offer cameras that can be used outside where the lighting conditions are certain to change and the cameras can adapt their calibration to the conditions. Inertial or 'Markerless' motion capture that is entirely sensor or suit based is becoming more popular for projects with a tighter budget and indie filmmakers, systems like Xsens and Perception Neuron are doing very well. They don't require a massive amount of setup or lots of expensive gear to run.
• Do you feel you can get a much better quality capture from equipment from a much higher price tier than a lower? Obviously if you have access to a large optical system the capture will be much more accurate and the detail in the data will be much higher, but also costs a lot more and has a lengthier processing time so requires multiple people working in a pipeline in stages whereas the more affordable inertial markerless stuff will give you dirtier/less accurate data but quicker - it will need a lot more cleanup in post afterwards though.
• Do you use a specific external programs for the cleaning process such as motion-builder / Maya / Iclone 6 etc? We use a variety of software for different stages of the pipeline - during capture we use Cortex to capture all the data from the cameras in real time, and Motionbuilder hooked up to Cortex to power client rigs for realtime previs - we also use Unreal Engine to further the previs quality if the client wants, we can also hook this up to the HTC Vive and allow the directors to be a part of the scene as it happens. The captured data is then tracked using some in-house tools we've developed as well as Cortex or Blade for Vicon data, this allows us to clear up any marker flips or confusion during the shoot - sometimes if people are fighting for example, Markers can come loose or become misidentified by the system for one that's closer to it etc... Once the data is tracked you'll have a file of marker co-ordinates which we can then build a bespoke skeleton set for - either to best fit the data, or to best fit the clients character rig they'd like the data delivered on as they may have custom joint setups and multiple spine bones etc... The skeleton is built and solved once again in either an in-house tool we have called solver interface or in Cortex/Blade. In tandem to this solver skeleton, we'll also characterise the clients rig in Motionbuilder so that the bones match up with the solver setup, and the data will know which bone to re target to once we import it into Motionbuilder on the rig.
• How long does it efficiently take to clean a captured animation to the point where it is acceptable of use in games? This is totally dependent on the type of game/client and whether or not it'll be game play animation or cinematic/cut scene animation. Some clients want data to be exactly as it was captured and do the cleanup themselves so we'll deliver untouched raw solves baked on their skeletons for them to polish up themselves in-house. Other clients want it 100% finished with full facial animation and finger/contacts/prop animation for cut scenes, this is called Motion Editing and allows the client to basically drop the delivered scene straight into the game engine as it is. We would give time estimates for this process based on scene length and character count, so its hard to give an exact answer for this one - anything from half an hour to a week!
Byron
I was trying to look into motion capture pipelines, do you any good locations or books I can get that'll teach me the proper pipeline processes? Overall though, the information you provided is very helpful! Thanks for the info.
Stu
In terms of pipeline stuff, this book was really helpful when I was first starting to look into Mocap when I was doing my degree, still keep it with me today:
https://www.amazon.co.uk/MoCap-Artists-Workflow-Techniques-Capture/dp/0240810007
The software's evolved slightly but the process and principles are still the same :)
Overall Consensus
Equipment / Preference Of Capture Style
It's is seen that a few of the experts stipulate that the perception neuron equipment is not the best for the price, it does seem that the XSens which is around the same price is actually much more in preference if a inertia suit is required. After reading the discussions again, the general consensus can be seen to the preference of equipment which is Vicon (Optical motion capture)
The capture style preference from industry as a whole doesn't really surprise me. Looking at the input, it is mostly based around a couple of specific reasons from what was presented:
Already Industry Used (Norm) - It's common use within the industry already (especially) in AAA companies and many big indie companies to use Optical based mocap. It's easy to set up, can hold multiple people and can easily handle most of the processing information from optical to digital data for cleaning.
Why get the cheapest and not the best? - A lot say you get the quality for the price you put in. This is especially seen within the capturing side of the animation process. A lot say, you are paying more but the quality of the products and the results of the capture that will guarantee a better end result for the capture artist.
All the software provided works well with each other! - The software that Vicon provided work well with other software and overall, they specifically designed it so it can be effective and works to what they need it to do. As well, it's designed to show if problems are seen in the sight of the camera with specific tools like the wand.
The capture style preference from industry as a whole doesn't really surprise me. Looking at the input, it is mostly based around a couple of specific reasons from what was presented:
Already Industry Used (Norm) - It's common use within the industry already (especially) in AAA companies and many big indie companies to use Optical based mocap. It's easy to set up, can hold multiple people and can easily handle most of the processing information from optical to digital data for cleaning.
Why get the cheapest and not the best? - A lot say you get the quality for the price you put in. This is especially seen within the capturing side of the animation process. A lot say, you are paying more but the quality of the products and the results of the capture that will guarantee a better end result for the capture artist.
All the software provided works well with each other! - The software that Vicon provided work well with other software and overall, they specifically designed it so it can be effective and works to what they need it to do. As well, it's designed to show if problems are seen in the sight of the camera with specific tools like the wand.
Practice
I did ask regarding practice if it was more practice with books or get messy? The overall presentation, especially the supervisors and managers state that although books are useful, it's practice. Learn it, break it once and don't break it again mentality which is understandable as the quote "Time is money" is a big factor, the faster your data is cleaned, the better your workflow will be, but this is all dependant on the demand of the recording, results of the capture and how competent the capture artist is at cleaning.
Comments
Post a Comment