News

09.08.16

Software creates lifelike animated 3D figures from animal videos

Whether it’s King Kong for Hollywood’s dream factory or monkeys in a computer game: creating digital three-dimensional animals and animating them in a lifelike way requires great effort. Often this is done by experts who work with special computer programs. Even they generally need multiple days. Now, researchers at the Max Planck Institute for Informatics have developed software that allows even laypeople to create these 3D figures in a few minutes—as long as there is a video that shows the animal in motion. The user need only mark the head, body, and limbs; all the rest is done by the software from Saarbrucken.

A cheetah runs at top speed through a grass savannah. The video sequence that shows this lasts only a few seconds. It’s enough to calculate a three-dimensional, moving digital model of the animal. Bernhard Reinert and Professor Hans-Peter Seidel, of the Max Planck Institute for Informatics, developed the software that makes this possible. Along with minimal help from the user, the program requires only a video showing the animal in motion. For example, to bring the 3D cheetah to life, the user must draw a stick-figure-like “skeleton” to mark the limbs on a frame from the video.

For this, the researchers have developed a user interface that looks like a simple paint program. With its help, it takes only a few mouse clicks to draw an orange line along the cheetah’s tail; the spine, head, and front and hind legs are marked with additional colored lines. The software transfers these markings over to all individual frames of the video sequence, so that the lines always mark the same body parts, even when they change positions in the other images. The user checks that the program has correctly captured all parts by looking at five more frames and correcting them by hand if necessary. “All existing measures to automate the detection of the limbs have failed here. Therefore, we have developed a new computational method that relies on special random processes, so-called Markov chains,” explains Bernhard Reinert, who completed his dissertation at the Max Planck Institute for Informatics a few weeks ago.

If the user confirms that the software has correctly recognized the cheetah’s limbs from the five key video frames, it then automatically produces a three-dimensional, digital mesh model of the animal. In so doing it separates the two-dimensional representation of the cheetah from the foreground and background and attempts to replace the drawn-in limbs with length- and diameter-variable cylinders until it finds a good fit for each individual still image. “The process is somewhat like the challenge of forming an animal out of a long balloon, but it is much more exact about what happens with the individual segments,” explains Reinert. In the last step the coat of the cheetah is copied from the same image that was used to designate its limbs, and this is then used as the so-called texture to cover the 3D mesh model. Now the three-dimensional cheetah model is complete.

“In this way, in a few minutes you can get a good 3D model that you can edit or refine further, when higher quality is needed,” explains Tobias Ritschel, Senior Lecturer at University College London. The quality of the unedited 3D model is nonetheless already good enough that it can be immediately used on a 3D printer to print a sculpture of the animal. The researchers have tested their software on different animal video clips that they found on the video platform YouTube, and thus set up an entire zoo of 3D models. The short videos are ideal, because in them it is often only the animal that is moving—the camera is not being moved back and forth wildly, which can present difficulties for the software. The 3D model can be displayed in different poses, with different fur or skin, and it can also be cloned, so that for example on the basis of one cheetah, an entire pack can be made.

Until now, something like this was only possible with a lot of manual work. This was shown by the researchers through another study. While one user group built a simple digital animal after a few minutes with the new software, others, despite having expert knowledge and special software, were not satisfied with their results after a half hour.

Further information:

Project page with video:

http://resources.mpi-inf.mpg.de/DeformableObjectExtraction/

 

Questions can be directed to:

Bernhard Reinert

Max-Planck-Institut für Informatik
Saarland Informatics Campus
Phone: +49 681 9325 4043
E-mail: breinert@mpi-inf.mpg.de
 

Dr. Tobias Ritschel

University College London

Department of Computer Science

Phone: +44 (0)20 3108 7397

E-mail: t.ritschel@ucl.ac.uk

Editor:
Gordon Bolduan
Kompetenzzentrum Informatik Saarland
Universität des Saarlandes
Phone: +49 681 302-70741
E-mail: bolduan@mmci.uni-saarland.de 

newsarchive >>