Choreographing Amateur Performers using Video Examples

彬彬

1 Introduction

We often imagine what it would be like to perform like a professional

dancer or an action star. However, because our physical skills

are insufficient, the resulting performance is poor. To make it appear

to be much better, we have developed a method for quickly and

easily choreographing an amateur performer in a video utilizing a

video example of a professional performance. Our method allows

the amateur performance to be edited by transferring the timing and

poses of the professional performance.

There are several methods to edit a performance video. One combines

video tracking and dynamic time warping (DTW) [Rao et al.

2003]. Video tracking produces motion trajectories of the corresponding

body parts in a pair of videos. DTW is then applied to

temporally align the amateur with the professional performance. It

supports only temporal alignment. Here, we extend this to spatial

pose deformation. Jain et al. reshaped the human body in a

video [2010]. Their method successfully makes a performer appear

thinner or fatter, but we want a more drastic edit, such as editing

an awkward moonwalk to create a smoother one. Xu et al. allowed

the user to create a performance video by specifying a sequence of

skeleton poses [2011].This could be used for our purpose, but the

method is expensive; it relies on a multi-view video database that

must be captured in a special studio. Additionally, the input sequence

of the skeleton poses must be three-dimensional (3D). By

contrast, our method involves simple two-dimensional (2D) video

editing, which does not allow one to change the view point but is

much easier to use and is inexpensive.

In our method, the user starts the choreography by preparing a pair

of performance videos (Fig. 1).Both videos involve similar performances,

such as a Karate kick, but one shows an amateur performer

(Fig. 1-a), and the other shows a professional performer (Fig. 1-b).

To simplify the problem, we assume that each video is captured

using a monocular camera. Our approach has three stages. First,

we apply semi-automatic video tracking to the corresponding body

parts in the pair of videos and extract motion trajectories. Automatic

video tracking often fails to produce a correct trajectory, especially

for fast body motions. We provide a user interface and an

optimization algorithm to correct such errors by enabling the user to

specify a few control points. Second, our method applies DTW to

the motion trajectories of corresponding body parts between videos

e-mail: mizui@onailab.com

ye-mail: m.o@acm.org

ze-mail: onai@cs.uec.ac.jp

to align their timing. Finally, our method deforms the amateur

poses to match the professional poses frame by frame so that the

corresponding motion trajectories match between the videos. The

first and second stages are semi-automatic, and the final stage is

automatic. We successfully applied our method to amateur performances

involving a Japanese sumo wrestling squat, a karate kick,

and a moonwalk, making them all look ‘cooler.’

2 Semi-automatic Video Tracking

The video tracking of corresponding body parts is the most important

process in our method. Accurate motion trajectories enable the

system to choreograph the amateur nicely. Inaccuracies or noise in

the videos may result in unnatural motions or vibration artifacts in

the animation. However, it is almost impossible to produce perfect

motion trajectories automatically. Even a state-of-the-art method

such as that of [Sand and Teller 2006] failed to track a fast body

motion. Therefore, we propose a novel user interface that fixes such

failures to capture a trajectory efficiently.

Figure 2: With a tracker, our system computes the motion trajectory,

as shown on the left. However, it fails to track the circle using

raw optical flow. The image on the right shows the motion trajectory

as set by the user. Note that our optimization algorithm has

made it smoother than the one on the left.

We explain our video tracking using the simple example of a red circle

moving along a curve(Fig. 2). The user specifies a tracker at the

center of the circle in the first frame. Then, the system computes

the motion trajectory (Fig. 2-left) based on the optical flow computed

using the method of Zach et al[2007]. However, because the

circle has no texture but a flat appearance, the tracker starts to drift

and drops out of the circle before the last frame. To correct such

a failure, the user can drag the last point to the correct position,

and the motion trajectory is updated by our optimization algorithm

(Fig. 2-right). Our algorithm is designed so that the resulting motion

trajectory 1) must go through the user-specified control points,

2) must follow the underlying optical flow as much as possible, and

彬彬

晃晃

你们都躲开，我来顶

晃晃

读铁系缘分，顶铁系友情

菜刀吻电线

好铁多多发，感激分享

晃晃

无聊时可以刷屏幕 灌水 也可以试试 帖子的标题究竟可以写多长

tc

提醒猪猪，千万不能让你看见

tc

已阵亡的 蝶 随 风 舞 说过  偶尔按一下 CTRL A 会发现 世界还有另一面

奇

真不错,全存下来了.

菜刀吻电线

呵呵，很好，方便罗。