Motion Capture from Body-Mounted Cameras

Motion Capture from Body-Mounted Cameras

Takaaki Shiratori¹          Hyun Soo Park²          Leonid Sigal¹
Yaser Sheikh²          Jessica K. Hodgins^2,1
¹Disney Research, Pittsburgh     ²Carnegie Mellon University

Motion capture technology generally requires that recordings be performed in a laboratory or closed stage setting with controlled lighting. This restriction precludes the capture of motions that require an outdoor setting or the traversal of large areas. In this paper, we present the theory and practice of using body-mounted cameras to reconstruct the motion of a subject. Outward-looking cameras are attached to the limbs of the subject, and the joint angles and root pose are estimated through non-linear optimization. The optimization objective function incorporates terms for image matching error and temporal continuity of motion. Structure-from-motion is used to estimate the skeleton structure and to provide initialization for the non-linear optimization procedure. Global motion is estimated and drift is controlled by matching the captured set of videos to reference imagery. We show results in settings where capture would be difficult or impossible with traditional motion capture systems, including walking outside and swinging on monkey bars. The quality of the motion reconstruction is evaluated by comparing our results against motion capture data produced by a commercially available optical system.

Takaaki Shiratori, Hyun Soo Park, Leonid Sigal, Yaser Sheikh, Jessica K. Hodgins
"Motion Capture from Body-Mounted Cameras"
ACM Transactions on Graphics, Vol. 30, No. 4 (Proc. ACM SIGGRAPH 2011), July 2011
[Paper (pdf, 17MB)] [BiBTeX]
[Movie (QuickTime, 85MB)]

Media Coverage

CMU, "Press Release: Body-Mounted Cameras Turn Motion Capture Inside Out"
Wired, "Disney Research Turns Mo-Cap Inside-Out With Body-Mounted Cameras"
Gizmodo, "Disney's New Way to Do Motion Capture Lets You Record It Anywhere" (Spanish version)
Slashdot, "Breaking Motion Capture Out of the Studio" (Japanese version)
Gizmag, "Motion capture system makes actors the camera instead of putting them in front of it"
GoPro, "GoPro HD HERO Cameras used for Disney and Carnegie Mellon Motion Capture"
Washington Post, "Disney and Carnegie Mellon Create Mocap On The Go"
MSNBC, "A big move for motion capture"
abc12, "Body-Mounted Cameras Turn Motion Capture Inside Out"
Gizmag, "Motion capture system makes actors the camera instead of putting them in front of it"
Reuters, "Body-Mounted Cameras Turn Motion Capture Inside Out"
Discovery News, "Actors Wear 20 Cameras for New Motion Capture"
International Business Times, "With Wearable Cameras, Motion Capture Turns Inside Out"
Pittsburgh Business Times, "CMU and Disney Research Pittsburgh take motion capture to new levels"
Science Blog, "Researchers turn motion capture inside out"
TechCrunch, "Disney And Carnegie Mellon Create Motion Capture On The Go" (Japanese version)
PR Newswire, "Body-Mounted Cameras Turn Motion Capture Inside Out"
Business Standard, "Breakthough in motion-capture tech"
Stitch Kingdom, "Disney, Carnegie Mellon Free MoCap Technology from Studio Confines"
Science Daily, "Body-Mounted Cameras Turn Motion Capture Inside out"
Tom's Guide, "Motion Capture from Body-Mounted Cameras"
Inc. Technology, "Disney Research Gives Motion Capture a Makeover"
Pop City, "Is it a swarm of insects or a video game? Disney Research, Pittsburgh unleashes new terrors"
Tech Gadget, "Motion-capture existence goes inside-out"
Photonics.com, "Turning Motion Capture Technology Inside Out"
slashCAM, "Free-body motion capture using cameras" (German version)
Essential Public Radio, "Body Mounted Cameras Coming to The People"
PhysOrg.com, "Body-Mounted Cameras Turn Motion Capture Inside Out"
keystone edge, "Is it a swarm of insects or a video game? Disney Research, Pittsburgh unleashes powerful new tech"
PC Pro, "Researchers boast motion capture breakthrough"
Coolest Gadgets, "Motion capture technology improved by Carnegie Mellon and Disney"
golem.de, "US-Forscher entwickeln neues Motion-Capturing-Verfahren" (in German)
Digital Production, "Auf der Siggraph vorgestellt: Motion-Capturing mittels Körper-Kamera" (in German)
Gazeta.pl Technologie, "Disney pracuje nad łatwiejszym motion capture" (in Polish)
Компьютерра Online, "Технологию захвата движения «вывернули наизнанку»" (in Russian)

Copyright notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.