PComp final documentation


I was almost ready to give up on the huge scrub bar and use my back up prefab touch potentiometer that I got from Sparkfun. I did some more testing and ran the thing by Michael Colombo and Eric Hagan and got some good advice.  The connection between the mylar layer and the ground wire wasn’t consistent enough so I had to clamp it down to get a good reading.  I bought a door hinge and revered it to get the clamp.

(video documentation coming soon)

Then I had to change the resistance because it was taking all of the current on its own.  I tried a few different resistors and finally found one that gave me a good range about 150 values.  All of a sudden it was working perfectly.

After that I build the circuit into the box with three knob potentiometer’s and another slider pot to control the scene.


I drilled holes in the box to screw down the Arduino, as well as holes for the connection with the big pot and USB cable to connect the Arduino to the computer.

Then last weekend I did a film shoot with a few friends to make the movie which would be featured in the installation.

Just as the installation takes a basic convention of film making, linear and chronological time, and undoes it by using a manual sensor to progress the video, the film, called Relationships Are Really Hard to Avoid, also plays with the relationship between actor and character.  Four actors simultaneously play each role, wearing different costumes.

I also recorded a man playing the Er Hu, a chinese instrument that sounds like a violin, at the Union Square subway and recorded myself playing drums and the actors reading voice over narration for the soundtrack of the movie.

Once I put everything together I tweaked the code and spent a lot of time working on the video compression.  I needed to get the video small to make it run smoother.  It was difficult to achieve a good balance of quality and size and I’m still working on it, but I got something that moved well enough to show in class.  I also put z-clamps on the back of each object and mounted them to a board to show how they could be mounted to a plain wall.

The code didn’t give me too much trouble.  I was simply reading five sensors to control audio and video.  I had some trouble working out the looping function of the Minim library for the audio but I figured it out with help from Alessandra Villaamil and some examples online.  I also had to tweak the interaction of the large scrub bar so it wouldn’t jump all over the place, which was actually pretty simple.  Since the range of the sensor was about 40-200 as it came in the Serial port.  So when you aren’t touching the sensor it reads 0, meaning I could tell it to only store the sensor value if it was over 10, which meant that the 0’s never got read and therefore the video didn’t jump around when you lifted your finger up.

Here’s the Processing code:

import processing.video.*;
import processing.serial.*;
import processing.opengl.*;
import ddf.minim.*;

Movie scene1;

Serial port;

float framePos = 0;
float movieDuration = 0;
float scene = 0;
int scenes = 40;
float sceneChange;

Minim minim;
AudioPlayer erhu;
AudioPlayer voice;
AudioPlayer drums;

float erhuVol = 0;
float voiceVol = 0;
float drumsVol = 0;

float rewind = .96;

void setup() {
  size(800, 480, OPENGL);
  frame.setLocation(0, 1050);

  scene1 = new Movie(this, "final5-800-50.mov");

  String portName = Serial.list()[0];
  port = new Serial(this, portName, 9600);

  minim = new Minim(this);
  erhu = minim.loadFile("erhu1.mp3", 512);
  voice = minim.loadFile("voice2.mp3", 512);
    drums = minim.loadFile("drums2.mp3", 512);

void draw() {

// set volume of audio tracks, pause if they reach the lowest volume
// rewind when the position of the audio reaches the 'rewind' value
  if (erhuVol > -44) {
  else if (erhuVol < -44) {     erhu.pause();   }   if (erhu.position() > rewind*erhu.length()){

  if (voiceVol > -44) {
  else if (voiceVol < -44) {     voice.pause();   }   if (voice.position() > rewind*voice.length()){

  if (drumsVol > -44) {
  else if (drumsVol < -44) {     drums.pause();   }   if (drums.position() > rewind*drums.length()){

// loop the video
  if (scene1.available()) {
  image(scene1, 0, 0);

// set video according to sliders
  float sceneMath = scene * (scene1.duration()/scenes);
  if (sceneChange != scene) {
    framePos = 0;
    println("SCENE CHANGE");
    println(sceneMath + framePos);
    sceneChange = scene;

  //println(sceneMath + framePos);
  scene1.jump(sceneMath + framePos);

//remove 'exit' from full screen
public void init() {

//read sensors
void serialEvent (Serial myPort) {
  String myString = myPort.readStringUntil('n');
  if (myString != null) {

  myString = trim(myString);
  int sensors[] = int(split(myString, ','));
  for (int i = 0; i < sensors.length; i++) {     //print("Sensor" + i + ": " + sensors[i] + "t");   }   //println();   if (sensors.length > 4) { // wait for all sensors
    if (sensors[0] > 10) {
      framePos = map(sensors[0], 40, 150, 0, scene1.duration()/scenes);
    scene = int(map(sensors[1], 0, 1023, 0, scenes));
    voiceVol = map(sensors[2], 0, 1023, -45, 45);
    erhuVol = map(sensors[3], 0, 1023, -45, 45);
    drumsVol = map(sensors[4], 0, 1023, -45, 45);


void stop()
  // always close Minim audio classes when you are done with them
  // always stop Minim before exiting.

The biggest code issue that remains is that the playback is sluggish because it only jumps frames every draw loop, while doing a bunch of other things.  It would probably work better if I had two different things controlling the audio and video.  I should probably try the Call-and-Response Serial method when I do further work on this.

The Arduino code was very simple, just reading 5 sensors:

void setup() {
  // open serial communications at 9600 bps

void loop() {
    for (int thisSensor = 0; thisSensor < 5; thisSensor++) {
       int sensorValue = analogRead(thisSensor);
       Serial.print(sensorValue, DEC);

       // if this is the last sensor value, end with a println().
       // otherwise, print a comma:
       if (thisSensor == 4) {
       } else {


Author: owen ribbit


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s