Sentence Generator: An example of dataset design

The Problem:

We needed to produce a machine learning dataset consisting of thousands of unique utterances each with the same intent.  Some examples of the types of intents: asking a voice assistant for the weather forecast, requesting news updates, playing music, making a phone call, setting a timer.

Challenges: 

• We needed to produce high-quality data, meaning it sounded natural and contained a lot of variety

• The dataset needed to be delivered with an extremely fast turnaround. 

• Hiring and onboarding freelancers is an operational cost, and a time-consuming endeavor. 

• Quality assurance on top of the data produced by freelancers is a challenge on top of data production

The Solution: 

I built sample sets of data so that I could analyze and understand the problem.  In doing so, I recognized that breaking sentences down into syntactic and semantic components would yield several templates that sentences of the same intent could follow. I would find words or short phrases that could be used interchangeably within the components of these templates, whereby each template could yield potentially hundreds of unique sentences.

Ensuring the solution produced high quality data would thereafter become a matter of introducing enough variety, which would require linguistic analysis and data management.

To illustrate:

Intent: Tell a voice assistant to play a song or record.

Sample Templates for this intent, composed of Components (in brackets):

• Template 1: [play] [name of record]

• Template 2: [play] [name of record] [on] [my] [speaker]

• Template 3: [play] [name of record] [on] [my] [living room] [speaker]

Wordlists: for each Component of a template, a list of sample words that can be used interchangeably:

• [play] = [play, stream, put on, air, run, start, fire up, turn on, start playing]

• [on] = [on, over, using]

• [my] = [my, our, the, this]

• [speaker] = [speaker, sound system, stereo, soundbar, audio system]

• [living room] = [living room, kitchen, dining room, bookshelf, bedroom]

Below I demonstrate what it looks like to go from templates and wordlists to a dataset of unique utterances: