tlhIngan Hol jatlhmeH, ghoghwIj lo' De'wI'wIj
https://soundcloud.com/user-649584902/tlhingan-hol-jatlhmeh-ghoghwij-lo-dewi... tlhIngan Hol vIjatlhtaHvIS, wab tamey puS vIchenmoH. ghIq, wab tamey ghoghwIj poj nIqHom. pojvammo', DaH tlhIngan Hol jatlhlaH nIqHomvam, 'ej ghoghwIj rurqu'. pa' nIqHomvam tu'lu': github.com/CSTR-Edinburgh/Ossian I’m experimenting with TTS software that works via training a neural network on speech samples with transcripts. The sample linked above was generated using a model built off of about forty minutes of my voice speaking Klingon. I think I might re-record some of the corpus, since the weird pacing and clipping is due to some inconsistencies in the recordings used to build the model, but the results so far are fairly promising. I originally had less than ten minutes of data fed into the model, and the resulting speech sounded like a 1980’s speech synthesizer. It still sounds markedly synthetic, but I’m much happier with the results now, and I believe they can be improved further. The original corpus I fed into it was just me reading all of the Klingon sentences from “Conversation Klingon”, because I wanted to see if it would produce a reasonable model with such a small amount of input. I was hoping that it might be possible to feed the NN some Klingon audiobooks and get “Worf’s” voice to come out the other end. While the results with such a small corpus were indeed intelligible Klingon, the quality of the speech was not good, and somewhat resembled the quality of the classic “Speak & Spell” toy, hence the choice of the example sentence in the recording (I’m not intentionally not providing a transcription, as I’m hoping the quality of the recording is sufficient for you to figure out what it says). When that limited corpus proved insufficient, I supplemented it with some proverbs and song lyrics, and then supplemented it further with random Klingon text generated from KAG’s online generator tool. I’m hoping to be able to take the results of this training and package it up in a way that offers convenient Klingon TTS services for whatever purposes one might want Klingon TTS functionality. If I’m successful in doing so, I’m wondering if some skilled Klingon speakers might be willing to lend their voices to such an effort. Experience with e.g. doing voiceover work would be.a plus, as I suspect that my POC would sound much better if I had been more disciplined about being consistent with the cadence and volume of my speech. And yes, I know De’vID already has a Klingon TTS software; I’m just curious about exploring other approaches.
On Sun, May 27, 2018 at 11:29 PM, Daniel Dadap <daniel@dadap.net> wrote:
https://soundcloud.com/user-649584902/tlhingan-hol- jatlhmeh-ghoghwij-lo-dewiwij
tlhIngan Hol vIjatlhtaHvIS, wab tamey puS vIchenmoH. ghIq, wab tamey ghoghwIj poj nIqHom. pojvammo', DaH tlhIngan Hol jatlhlaH nIqHomvam, 'ej ghoghwIj rurqu'.
pa' nIqHomvam tu'lu': github.com/CSTR-Edinburgh/Ossian
Daj. Are there instructions for running it under Windows, or must I try to get Linux running on a computer I no longer use?
(I’m not intentionally not providing a transcription, as I’m hoping the quality of the recording is sufficient for you to figure out what it says).
The first syllable of the third word is unclear, as is the fourth from last word. The {j} sounds a lot like {ch}, but even accounting for that I can't make sense of it grammatically. -- ghunchu'wI'
On May 27, 2018, at 23:59, Alan Anderson <qunchuy@alcaco.net> wrote:
On Sun, May 27, 2018 at 11:29 PM, Daniel Dadap <daniel@dadap.net> wrote: https://soundcloud.com/user-649584902/tlhingan-hol-jatlhmeh-ghoghwij-lo-dewi...
tlhIngan Hol vIjatlhtaHvIS, wab tamey puS vIchenmoH. ghIq, wab tamey ghoghwIj poj nIqHom. pojvammo', DaH tlhIngan Hol jatlhlaH nIqHomvam, 'ej ghoghwIj rurqu'.
pa' nIqHomvam tu'lu': github.com/CSTR-Edinburgh/Ossian
Daj. Are there instructions for running it under Windows, or must I try to get Linux running on a computer I no longer use?
If instructions for Windows exist, I haven’t seen them. The bulk of the tools and even most of the dependencies are Python and in theory could run on Windows, but there are some places where I could imagine that things weren’t done in an especially portable way. And there could be some dependency that doesn’t port to Windows as well. In particular, the setup scripts assume a Unix-like system, but it might still be possible to replicate what they’re doing outside of a Unix-like environment. I have gotten the tools working on GNU/Linux and macOS without much trouble, so if you have access to a Mac that could be another option. If you’re intrigued enough to consider getting GNU/Linux going, I would consider trying (if you’re on Windows 10) the Windows Subsystem for Linux to see if that is sufficient. Naively, it seems that it ought to be. You should probably know that it requires quite a bit of computing power to do the training, and while the training times will be tolerable on a CPU for a tiny corpus (like the example Romanian corpus in the instructions), for anything serious you will probably want to be doing it on a GPU, and I’m nearly certain that you won’t be able to get Theano’s GPU support running on Windows, or even WSL. (Disclaimer: I work for NVIDIA, and all opinions are my own.) If you just want to play with a small corpus, feel free to try getting the tools running on your own system, but if you want to train anything substantial and don’t have a GNU/Linux system with a suitable GPU, it might be easier to send me recordings for training. You could still get the tools going on your own system to handle the synthesis, which requires substantially less compute power.
(I’m not intentionally not providing a transcription, as I’m hoping the quality of the recording is sufficient for you to figure out what it says).
The first syllable of the third word is unclear, as is the fourth from last word.
The fourth from last word is {chaq}. The third word is key to the meaning of the whole utterance, and I agree that it doesn’t come out clearly. It’s a somewhat obscure word and doesn’t occur in the corpus (though neither do several of the other words you didn’t seem to have a problem with).
The {j} sounds a lot like {ch},
Yeah, it seems to have a problem with voicing. {gh} sounds too close to {H} in some places. I’ve gotten the validation loss down from ~190 with my initial corpus to ~175 with the current corpus, and the difference in quality is stark, so I am hoping that additional training with higher quality recordings will improve things further.
but even accounting for that I can't make sense of it grammatically.
Yeah, admittedly the reason I can understand it is because I wrote the sentence and I know what it’s supposed to say, so I can hear past the deficiencies in the audio. Of course, I don’t actually speak Klingon particularly well yet, so it’s possible the sentence just doesn’t make sense at all. I’ll send you the solution privately, to judge whether the sentence is sensible regardless of the speech production, and if you don’t think it makes sense I’ll go ahead and post the transcript to this thread.
-- ghunchu'wI' _______________________________________________ tlhIngan-Hol mailing list tlhIngan-Hol@lists.kli.org http://lists.kli.org/listinfo.cgi/tlhingan-hol-kli.org
On 28 May 2018 at 13:27, Daniel Dadap <daniel@dadap.net> wrote:
The fourth from last word is {chaq}. The third word is key to the meaning of the whole utterance, and I agree that it doesn’t come out clearly. It’s a somewhat obscure word and doesn’t occur in the corpus (though neither do several of the other words you didn’t seem to have a problem with).
What I hear is: *{tera'ngan poH Hoch tej ngoy jIH net chal SuH vaj tlhIngan Hol vIjatlhtahvIS chaq HolwIj 'oH Holvam'e'.} Since you say the third word is key, I'd guess what I hear as {Hoch tej} is actually {Holtej}. Maybe it begins {tera' Dabbogh Holtej noy}? -- De'vID
On May 28, 2018, at 10:16, De'vID <de.vid.jonpin@gmail.com> wrote:
On 28 May 2018 at 13:27, Daniel Dadap <daniel@dadap.net> wrote: The fourth from last word is {chaq}. The third word is key to the meaning of the whole utterance, and I agree that it doesn’t come out clearly. It’s a somewhat obscure word and doesn’t occur in the corpus (though neither do several of the other words you didn’t seem to have a problem with).
What I hear is: *{tera'ngan poH Hoch tej ngoy jIH net chal SuH vaj tlhIngan Hol vIjatlhtahvIS chaq HolwIj 'oH Holvam'e'.}
Since you say the third word is key, I'd guess what I hear as {Hoch tej} is actually {Holtej}. Maybe it begins {tera' Dabbogh Holtej noy}?
Yes, it does start with {tera' Dabbogh}, and you are close with the rest. In order to eliminate the input (i.e. my voice) as the source of the difficulty rather than the output (i.e. the model of my voice) I have recorded myself speaking the phrase in question: https://soundcloud.com/user-649584902/tlhingan-hol-vijatlhmeh-ghoghwij-vilo
And yes, I know De’vID already has a Klingon TTS software; I’m just curious about exploring other approaches.
Training a neural network to generate the speech was already requested: https://github.com/De7vID/klingon-assistant-tts-android/issues/6
Cool, I hadn’t seen that. If I am successful in producing a speech model of reasonable quality with a reasonably portable frontend for synthesis, I will be certain to contribute what I can.
-- De'vID _______________________________________________ tlhIngan-Hol mailing list tlhIngan-Hol@lists.kli.org http://lists.kli.org/listinfo.cgi/tlhingan-hol-kli.org
On May 28, 2018, at 7:27 AM, Daniel Dadap <daniel@dadap.net> wrote:
The fourth from last word is {chaq}.
Understanding that, it now makes sense grammatically. The third word is still very muddled (if one can apply the term to consonants), and even knowing what it is I’m still not hearing the end of the first syllable properly. Unvoiced and non-plosive stops in the generated sentence are hard to distinguish. The glottal stop is going to be a difficult thing to train the network with in general. How does one properly teach a sound which half the time is only there because the preceding vowel ends abruptly? :-) I would suggest devising a training set that gives special attention to it in every combination of leading and trailing sounds. — ghunchu'wI'
On May 28, 2018, at 10:44, Alan Anderson <qunchuy@alcaco.net> wrote:
On May 28, 2018, at 7:27 AM, Daniel Dadap <daniel@dadap.net> wrote:
The fourth from last word is {chaq}.
Understanding that, it now makes sense grammatically. The third word is still very muddled (if one can apply the term to consonants), and even knowing what it is I’m still not hearing the end of the first syllable properly.
Unvoiced and non-plosive stops in the generated sentence are hard to distinguish. The glottal stop is going to be a difficult thing to train the network with in general. How does one properly teach a sound which half the time is only there because the preceding vowel ends abruptly? :-) I would suggest devising a training set that gives special attention to it in every combination of leading and trailing sounds.
Yeah, I was hoping it would be sufficient for it to learn organically if given sufficient input (and it may yet be), but if not I will certainly enhance the training data with “designer” phrases.
— ghunchu'wI' _______________________________________________ tlhIngan-Hol mailing list tlhIngan-Hol@lists.kli.org http://lists.kli.org/listinfo.cgi/tlhingan-hol-kli.org
On 28 May 2018 at 05:29, Daniel Dadap <daniel@dadap.net> wrote:
And yes, I know De’vID already has a Klingon TTS software; I’m just curious about exploring other approaches.
Training a neural network to generate the speech was already requested: https://github.com/De7vID/klingon-assistant-tts-android/issues/6 -- De'vID
participants (3)
-
Alan Anderson -
Daniel Dadap -
De'vID