How to Script a Voice-Sensitive Robot Animation in p5.js

JavaScript is the language of the internet. Working in front-end development without it is impossible. For beginners, kids especially, it can be challenging to get used to the language’s syntax.

To complete novices, even installing and running a local server in the browser can be a stumbling block. What if there was a way to make something cool with JavaScript, without installing anything? Enter p5.js, a coding library designed with creativity in mind.

Here’s how to make a sound reactive animated robot head using some simple coding principles.

What Is p5.js?

The p5.js library was created by Los Angeles Based artist Lauren MacCarthy. It is designed to provide a similar platform to Processing, for creative expression and art. Clear tools go along with well explained tutorials and reference documents to make it perfect for beginners.

Whether you are making animations, music, simple games, or even connecting to external hardware, p5.js can help.

The p5.js Editor

To begin, open your browser window and navigate to the p5.js web editor. Before starting, there are two optional steps.

1. You must be signed in to p5.js to save your sketches, so making an account is advisable. Signing up is free, and you can use your Google or GitHub account to sign in if you wish.

2. Click on the settings cog wheel in the top right corner. Here you can alter the theme and text size for all of you dark theme lovers out there.

The p5.js editor combines a code editor, console, and output window in the same space. So far there are two functions already set up for you.

Basics of p5.js

Every p5.js sketch begins with two elements. The setup() function, and the draw() function. To those of you who have programmed an Arduino before, this will be familiar!

function setup() {
  createCanvas(500, 500);
}

The setup() function runs at the start of your program. In this case, setup creates a 400-pixel square canvas. Change this to (500, 500) for a slightly bigger area to work.

Setup runs only once and is used to create the parts needed for your program, along with initial values for your program.

function draw() {
  background(220);
}

The draw() method is called every frame. This works much like the loop function on Arduino, or the update function in Unity 3D. Anything that needs to change in your sketch happens here. For now, the background gets redrawn every frame. At the top you’ll see two icons, play and stop. Click play.

This is the program so far—a 500×500 canvas with a grey background,

Creating a Shape

Working with shapes in p5.js is a little different to the preset shape objects used the mo.js web animation tutorial. To create a simple ellipse, add this to the code in the draw() method, just below where you set the background color.

function draw() {
  background(220);
  ellipse(250,250,50)
}

The ellipse() method takes several arguments. The first two are its X and Y position on the canvas. Since the canvas is 500 pixels wide, setting these two values to 250 puts the ellipse in the center. The third argument is the width of the circle—in this case, 50 pixels.

So you have a background and a circle, but it doesn’t look all that good. Time to fix that.

Adding Some Style

Begin by changing the background color. The number in the brackets represents a grayscale value. So, 0 is black, and 255 is white, with different shades of grey in between. To make the background black, change this value to 0.

function draw() {
  background(0);
  ellipse(250,250,50);
}

Now when you click play the background will be black. To give the circle some color, we will want to affect its RGB (red green and blue) values individually. Create some variables at the top of your script (right at the start, before the setup function) to store these values.

var r, g, b;

In the setup function, set the value of r to 255, and give the others a value of 0. Its combined RGB color will be bright red!

r=255;
g=0;
b=0;

To apply the color to the circle, add a fill() call to the draw method, just before drawing the circle.

  fill(r,g,b);
  ellipse(250,250,50);

Now click play, and you should see a red circle on a black background.

Making the circle is a good start, but adding interactivity is where it gets really interesting!

Click to Change Color

Adding code to run on a mouse click is quite easy in p5.js. Create a new function below the draw() method.

function mousePressed() {
  r = random(256);
  g = random(256);
  b = random(256);
}

The mousePressed() listens for any mouse presses and carries out the code inside the brackets. The random() function returns a random value between 0 and a given number. In this case, you are setting the r, g, and b values to between 0 and 255 on each mouse press.

Rerun the code, and click the mouse a few times. The circle should change color when you do.

Now your animation is reactive to mouse clicks, but what about using the user’s voice?

Setting Up Voice Control

Using the system mic is made easy with the p5.js sound library. At the top of your script, create a new variable called mic.

var r, g, b;
var mic;

Add these lines to your setup() function to assign mic to the audio input.

mic = new p5.AudioIn()
mic.start();

When you click play now, you will get a dialog box asking permission to access the onboard mic.

Click Allow. Depending on which browser you use, it may remember your choice, or you may have to click a box confirming. Now the mic is set up, and it’s time to put it to use.

Scaling by Volume

To use the volume of the microphone as a value in your program, you’ll need to store it as a variable. Change your draw() method to look like this:

function draw() {
  var micLevel = mic.getLevel();
  background(0);
  fill(r,g,b);
  ellipse(250,250,50 + micLevel * 2000);  
}

At the start of the function, a new variable called micLevel is created, and assigned to the current microphone amplitude.

The ellipse had a fixed width of 50 pixels. Now 50 pixels is the minimum width, added to the micLevel value which updates every frame. The number the micLevel multiplies with will vary depending on the sensitivity of your microphone.

Experiment with higher values—a value of 5000 will create a more dramatic change in scale!

Note: if this does not work for you, your microphone may not be set up correctly. The browser uses the system default microphone, and you may need to modify your sound settings and refresh.

Constructing the Robot

Now you have all the tools needed to create the robot. First, move the ellipse you have created, and add another one to make the eyes.

  ellipse(150,150,50 + micLevel * 2000);  
  ellipse(350,150,100 +micLevel * 2000);

The “teeth” are a series of rectangles extending from the bottom of the screen. Note that the rect() method requires an extra parameter for its fixed width.

    rect(0, 500,100, -100 -micLevel * 5000);
    rect(400, 500,100, -100 -micLevel * 5000);
    rect(100, 500,100, -100 -micLevel * 3000);
    rect(300, 500,100, -100 -micLevel * 3000);
    rect(200, 500,100, -100 -micLevel * 1000);

This time, you only want the height of the teeth to change, and to give the “smiling” effect they must have different sensitivities. The minimum height is -100 (as it comes up from the bottom of the canvas), so the micLevel must also be a negative number.

Click play, and you should see an almost finished robot face!

Once again, you may need to tweak your multiplier numbers for the best results.

Adding Finishing Touches

Add pupils to the eyes with even more ellipse() calls. Use another fill() to make them white. Add this to your draw() method, below the “teeth” you just created.

fill(255);
ellipse(150,150,20 + micLevel * 1000);
ellipse(345,150,30 + micLevel * 1000);

Finally, to give the whole piece a little definition, change the stroke weight in your setup() function

strokeWeight(5);

That’s it!

If anything isn’t working, check your code thoroughly, and you can see the full code from this tutorial in the p5 editor if needed.

Moving Forward With p5.js

This beginner tutorial covers some basic concepts with p5.js, showing just how easy it is to be creative. As always, this project barely scrapes the surface of all that p5.js is capable of.

Spending time creating art with p5.js is time well spent, as you are familiarizing yourself with thinking like a programmer and JavaScript syntax. These are all important skills to take forward if you decide to dive in wholeheartedly and really learn JavaScript!

Read the full article: How to Script a Voice-Sensitive Robot Animation in p5.js