Currently I'm working on a similar project. Building a GAN (generative adversarial network) for voice is a lot of work and testing, you need a big dataset for labeled voices. Wavenet currently support English[0]. Also you need a neural NLP model for this language.

[0]https://cloud.google.com/text-to-speech/docs/voices