/ gamedev

Retro Voice Acting System in Unreal Engine 4

In this post I'll give you a look at a small proof-of-concept Blueprint system I built to create a 'retro' voice acting system. What I mean by that is using a short audio-sample repeatedly and giving it the semblence of human speech. A good example is how Undertale does it.

Overview

The system basically works like this:

  • An NPC is given the DialogueComponent Blueprint
  • Fill in their various Dialogue lines
  • When a player talks to them, process these lines
  • Play audio as necessary

To me, the interesting part is using Float Curves to define various pitch and volume changes in a sentence, to make the audio less robotic.

Note: Beyond this point, links will open up images detailing the parts of the Blueprints.

Player Blueprint

Now, the way I built most of the system is a bit silly: I put nearly all the Dialogue functionality in my PlayerCharacter Blueprint, this is not advisable, but it'll be fine for my prototype.

I perform a LineTrace whenever I click, when I hit something I check if the target has a DialogueComponent attached to it, make sure I keep track of what the current line is, and spawn a floating text containing the line. When that's all done, I set up a looping timed function that fires off my audio and stuff..

The Core Functionality

Here's where it gets pretty fancy, in the looping timed function I split up the current sentence into an array of words, meaning I will from this point know how many words are in the sentence. Then, based on the last character in the last word of the sentence, I select one of the three Float Curves I made. If the sentence ends with a question mark, it picks the PitchCurveQuestion. For an exclamation point, the PitchCurveShouting is selected, and for all other cases, we use the PitchCurveStatement. These PitchCurves are simple Float Curves that somewhat follow the progression in pitch for questions and statements, the Shouting curve is mostly just slightly higher-pitched and louder.

In order to change the pitch and volume of our tiny audio-sample throughout the sentence, I map the range of the wordcount from 0.0 to 1.0, matching the length of the PitchCurves. And to end this timed function, I clear the timer that's calling it whenever there are no more words in the current sentence.
Because I use the PitchCurves as a modifier/multiplier, I can still change the overall pitch and add small randomization within the AudioCue I play.

Fun fact: You can, instead of counting the words, also use the length in characters of the sentence and play a sound for every character, that way it sounds more like the reference material.

The DialogueComponent

I subclassed this Blueprint from ActorComponent, meaning I can place it on any actor I want, even individual ones already placed in the level. Because I made the DialogueLines array variable publicly editable, I can add or remove lines from any actor I add this component to.

And that's pretty much all there is to it.
Now, keep in mind, this is by no means meant to be a tutorial, just some information on how I built it. This is not a production-ready system either, so I'd advise against using this as-is.

Thanks for taking the time to read all this!