Working with Media


Read through Chapter 6 of Getting Started with Processing and work all the code examples. This chapter includes an introduction to PImage and PFont. You can get a basic idea of how to use both from the book, but you'll learn a lot more by delving into the pages for both in the Processing Reference.

For example, a quick read of PImage shows the functions I used in my video above:
But also shows some other cool stuff as well:
  • imageMode() - This should be pretty familiar. Very similar to rectMode() which you've used before.
  • background() - Wait, this isn't new. Or is it... You can do background(img) and then have a background image!
  • blend() - Lots of possibilities here. Play with it and see what you can do.
Thing to Try - Create a program that applies some filters or blends with images dynamically. That is to say the user controls it in some way. Mouse motion or dragging would work well here.

Generate Sound

We can make tones in processing using the Minim library. The easiest way to experiment with it is to go to File->Examples->Libraries->Minim. For this Lesson you should open up SineWaveSignal.

Run the program. As you move your mouse around in the window you should hear changes in the sound. You'll notice that both the volume and the pitch change.

Lets break down what we see:
The opening block imports the piece of the Minim library that we need, just like we had to import the Arduino library. 

Minim minim;  //Creates an instance of Minim called minim
AudioOutput out;  //Creates an instance of the AudioOut procedure called "out"
SineWave sine;  //Creates an instance of the SineWave procedure called "sine"

These names could be anything. For example you could just as easily have:  SineWave bob;

Our setup procedure has some new stuff. The comments do a pretty good job of explaining, I'll just expand a little. Portamento is an actual term used in music and is very important for electronic synthesizers. It deals with how quickly the note will change. Set it to 20 and then 2000, or try sine.noPortamento(). You should hear clear differences.

If you're going to want to play more than one note at a time you'll need to create more instances of SineWave at the top and then reproduce some of the setup lines, for example:

The code above will set us up to play two notes at the same time. Think chords here. If you want three or four notes you'll need three or four instances of SineWave. The draw procedure of this sketch just shows us the wave form of the sound we're hearing. We'll go into how it works in another lesson.

In the mouseMoved() procedure you see where the frequency changes happen. The table below shows some basic commands.
 sine.setFreq(##) Will create a frequency(pitch) at the number specified
 sine.setPan(##) Sets speaker ballance. The number should be between -1 and 1. -1 Means left speaker only +1 means right speaker only
 sine.setAmp(##) Adjusts volume. The number should be between 0 and 1. 0 means off and 1 means full volume

Other fun stuff to try. This sketch produces a sine wave, but can als produce a square wave or sawtooth wave. Try each and see how the sound differs. To do this you need to change the SineWave instance to SquareWave or SawWave. You need to do this both at the top and in the setup procedure.

Working with Audio Files


You can have processing grab an audio file and play it. Stick with mp3 or wav files. Lets play with the LoadFile example sketch (File->Examples->Libraries->Minim->LoadFile).

Time to Break it Down:

The first bit imports the Minim Library, Starts Minim, and create an instance of AudioPlayer called player. Again, "player" is just a name, it could be anything.

The comments do a good job of explaining. One thing to know. Processing will look for the audio file in the sketch's "data" folder. The important piece here is; This is the command that plays the file. If you take it out of the setup() and do something like:

Then the song will wait till you press a key. If you're working on a touchscreen interface you could change this to void mousePressed() instead.

Player Commands (remember, "player" is just an instance of AudioPlayer you could use any word you want instead of "player"):  Plays the audio file loaded into player
 player.pause()  Pause the audio file being played by player
 player.loop(##)  Play the audio file in player ## times
 player.loop()  Replaces to loop forever
 player.rewind()  Start the audio file in player over again
 player.skip(##)  Skip ahead ## milliseconds (use negative ## to skip Back)

To learn more check out the Minim Code Library for a more thorough explanation of Playable procedures.


This works just like LoadFile, but is used for short audio samples you will play over and over again. Grab it from File-Examples-Libraries-Minim and take a look. It should look very familiar. In fact, I'm not going to explain it all. You should be able to understand it all without my help. The only new bit is name.trigger() which basically just plays the file.

The main difference between LoadSample and LoadFile is when the audio file is copied into the computer's RAM. If you have short audio files you should use LoadSample. It will take slightly longer for the program to start when you run it, but it will run better.

You have all the same control options in AudioSample as you did with AudioPlayer. Again, your audio files need to be in the data folder so Processing can find them. Things to try:
    • Make a keyboard Drum Machine - Load in some drum sounds (Free Drum Sounds) and set them to be triggered with key presses from your keyboard.
    • Make a touchscreen Drum Machine - Modify your keyboard drum machine to draw ellipses and then play the appropriate loop when the ellipse is tapped.