Automatic Singing System of Japanese Songs in Chinese 6th Nov. 2008 - - PowerPoint PPT Presentation

▶
automatic singing system of japanese songs in chinese
SMART_READER_LITE
LIVE PREVIEW

Automatic Singing System of Japanese Songs in Chinese 6th Nov. 2008 - - PowerPoint PPT Presentation

Oct 26, 2022 305 likes •513 views

Automatic Singing System of Japanese Songs in Chinese 6th Nov. 2008 CJNLP-2008 EHARA Terumasa Tokyo University of Science, Suwa CJNLP-2008 1 Motivation To enhance the cultural exchange between Japan and China To make Chinese

slide-1
SLIDE 1

CJNLP-2008 1

Automatic Singing System of Japanese Songs in Chinese

6th Nov. 2008 CJNLP-2008 EHARA Terumasa Tokyo University of Science, Suwa

slide-2
SLIDE 2

CJNLP-2008 2

Motivation

  • To enhance the cultural exchange

between Japan and China

  • To make Chinese automatic singing

system

  • To improve the quality of singing software

to it of the human singing

slide-3
SLIDE 3

CJNLP-2008 3

Procedure of automatic singing

Japanese song Lyrics Musical score Machine-translation / post-editing Speaking in Chinese Converting to singing voice Singing in Chinese Tone / duration

slide-4
SLIDE 4

CJNLP-2008 4

Converting to singing voice

Speaking voice STRAIGHT* (analyzing part)

* Kawahara, H. et al., “Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible role of a repetitive structure in sounds”, Speech Communication, Vol. 27, pp. 187–207, 1999.

F0 contour Spectrum envelope Musical score STRAIGHT (synthesizing part) Singing voice Duration and F0 control

slide-5
SLIDE 5

CJNLP-2008 5

Duration control

  • piecewise linear function

inp - out 000-000 360-200 540-400 630-650 830-750

slide-6
SLIDE 6

CJNLP-2008 6

F0 control

  • Model 0

F0(t) = Y0

  • Model 1

F0(t) = Y0 + (a1＊t + a0)

  • Model 2

F0(t) = Y0 + (a2＊t 2 + a1＊t + a0)

slide-7
SLIDE 7

CJNLP-2008 7

Controlled F0 (model 0)

Blue line ： original F0 Red line ： controlled F0

slide-8
SLIDE 8

CJNLP-2008 8

Preliminary study

Original 妈 麻 马 骂

Model 0 Model 1 Model 2

slide-9
SLIDE 9

CJNLP-2008 9

Experiment

slide-10
SLIDE 10

CJNLP-2008 10

Original Japanese song

(Spring has come)

slide-11
SLIDE 11

CJNLP-2008 11

Machine-translation / post-editing

春が来た 春が来た どこに来た 山に来た 里に来た 野にも来た 春天来临了 春天来临了 来了哪里 来了到山 来了村落 也来了野地 春天已来了 春天已来了 来到了哪里 来到了山里 来到了村落 也来到野地 Machine-translation Post-editing

slide-12
SLIDE 12

CJNLP-2008 12

Convert to a singing voice

Speaking Singing (F0 model 0) 来 到 了 哪 里 来 到 了 哪 里～～

slide-13
SLIDE 13

CJNLP-2008 13

Duration control

来 到 了 哪 里～～～～ 来 到 了 哪 里

slide-14
SLIDE 14

CJNLP-2008 14

Speaking voice

slide-15
SLIDE 15

CJNLP-2008 15

Singing voice

slide-16
SLIDE 16

CJNLP-2008 16

Conclusion

  • To develop an automatic singing system of

Japanese songs in Chinese.

  • To translate Japanese lyrics into Chinese

using machine-translation and post-editing.

  • To convert a speaking voice to a singing

voice in Chinese.

slide-17
SLIDE 17

CJNLP-2008 17

Future works

  • To control Chinese syllabic tones appropriately.
  • To use more sophisticated F0 and duration

control technique*

  • To extend our method to more complicated

Japanese songs other than children’s songs

* Takeshi Saitou et. al : Vocal conversion from speaking voice to singing voice using STRAIGHT, Interspeech 2007, Aug. 2007.

slide-18
SLIDE 18

CJNLP-2008 18

Acknowledgement

  • Providing STRAIGHT in research use
  • Prof. KAWAHARA Hideki (Wakayama univ.)
  • Collaborators

Assistant prof. OTA Kenko Former graduate student KUBO Yasushi Under graduate student KOBAYASHI Minoru, YAMAKOSHI Yuuta Research student Qigeqi

slide-19
SLIDE 19

CJNLP-2008 19

Thank you for your attention