MDCNN

Multimodal Voice Conversion Using a Deep Convolutional Neural Network in an Adverse Environment

ZHOU JIAN1, HU YUTING1, LIAN HAILUN1, WANG HUABIN1,TAO LIANG1, Hon Keung Kwan2

1MOE Key Laboratory of Intelligent Computing and Signal Processing, School of Computer Science and Technology, Anhui University, Hefei, China

2Department of Electrical and Computer Engineering, University of Windsor, Windsor, ON N9B 3P4, Canada

This demo page provides some samples of speech conversion results using three different speech conversion algorithms in different noise environments. The proposed MDCNN, the traditional NMF and the DNN algorithms are used respectively to conduct male to female speech conversion (Demo 1) as well as female to female speech conversion (Demo 2). Six types of noise from the noise97 database are used to corrupt the clean speech to obtain the noisy speech in different noise environments.

Recommended browser: Google Chrome

Cite as: Jian Zhou, Yuting Hu, Hailun Lian, Huabin Wang, Liang Tao, and Hon Keung Kwan, "Multimodal voice conversion under adverse environment using a deep convolutional neural network," IEEE Access, vol. 7, pp. 170878-170887, 26 November 2019, DOI: 10.1109/ACCESS.2019.2955982


Demo1: Male to female voice converison.


source (without noise)

就是下雨也去。

JIU SHI XIA YU YE QU.

target

就是下雨也去。

JIU SHI XIA YU YE QU.

NMF

就是下雨也去。

JIU SHI XIA YU YE QU.

DNN

就是下雨也去。

JIU SHI XIA YU YE QU.

MDCNN

就是下雨也去。

JIU SHI XIA YU YE QU.

source (5db with White noise)

就是下雨也去。

JIU SHI XIA YU YE QU.

target

就是下雨也去。

JIU SHI XIA YU YE QU.

NMF

就是下雨也去。

JIU SHI XIA YU YE QU.

DNN

就是下雨也去。

JIU SHI XIA YU YE QU.

MDCNN

就是下雨也去。

JIU SHI XIA YU YE QU.

source (5db with Pink noise)

就是下雨也去。

JIU SHI XIA YU YE QU.

target

就是下雨也去。

JIU SHI XIA YU YE QU.

NMF

就是下雨也去。

JIU SHI XIA YU YE QU.

DNN

就是下雨也去。

JIU SHI XIA YU YE QU.

MDCNN

就是下雨也去。

JIU SHI XIA YU YE QU.

source (5db with Volvo noise)

就是下雨也去。

JIU SHI XIA YU YE QU.

target

就是下雨也去。

JIU SHI XIA YU YE QU.

NMF

就是下雨也去。

JIU SHI XIA YU YE QU.

DNN

就是下雨也去。

JIU SHI XIA YU YE QU.

MDCNN

就是下雨也去。

JIU SHI XIA YU YE QU.


Demo2: Female to female voice converison.


source (with F16 noise)

他诧异地问。

TA CHA YI DE WEN.

target

他诧异地问。

TA CHA YI DE WEN.

NMF

他诧异地问。

TA CHA YI DE WEN.

DNN

他诧异地问。

TA CHA YI DE WEN.

MDCNN

他诧异地问。

TA CHA YI DE WEN.

source (5db with Factory noise)

他诧异地问。

TA CHA YI DE WEN.

target

他诧异地问。

TA CHA YI DE WEN.

NMF

他诧异地问。

TA CHA YI DE WEN.

DNN

他诧异地问。

TA CHA YI DE WEN.

MDCNN

他诧异地问。

TA CHA YI DE WEN.

source (5db with HFChannel noise)

他诧异地问。

TA CHA YI DE WEN.

target

他诧异地问。

TA CHA YI DE WEN.

NMF

他诧异地问。

TA CHA YI DE WEN.

DNN

他诧异地问。

TA CHA YI DE WEN.

MDCNN

他诧异地问。

TA CHA YI DE WEN.