Final Project

My inspiration for the final project was initially to force myself to make a musical instrument using left over parts and sensors that I had lying around from my other musical projects where I had a shopping list for. So it was going to be a "Stone Soup" type of instrument.

I also wanted to use the Teensy 3.2 micro controller board because I really have fallen in love with it's small size, copious inputs, and capacitive touch pins. I just so happen to have had one lying around from a different project so I went to work removing the headers so I could get to more of the pins on the back side of the board.... An hour later I put in an order to Amazon for a new Teensy because I totally mauled some of the copper inserts on the board and rendered a few of the TouchSense pins unusable...Derp
All was not lost though I had enough working pins to put together I small touch keyboard!

Ordering a new Teensy also gave me the chance to splurge for some LED buttons because I had to spend 35$ for next-day shipping. While I waited for Amazon I got to work in Adobe Illustrator planning out the template for the instrument to laser cut. I took an inventory of buttons, sliders, dials, lasers, light sensors, and copper tape and went to town.

I was thankful for the work I put into learning how to laser cut and laser print earlier in the semester because I was pleasantly surprised for everything to some out nicely on the first try. Yay practice!

I mounted the buttons and some pots to see if they would fit and then I got to the grind of it... wiring and solder. Luckily for me I find it sort of therapeutic. It feels rewarding to take pieces and make wholes. I remembered to take lots of pictures because I knew a lot of things would be connecting to this tiny board. 

Big plans for a small thing

TouchSense Soldering First

Early Mockup

Pots and buttons next

Getting messy

Buttons are lit, copper tape is live!

I couldn't keep myself from trying it out as it was at this point in the build. 

It was all going together so well... until the lasers.

I had issues with the lasers shorting themselves out and damaging the diodes which made them useless in triggering the light sensors I was using to trigger the notes....Another order to Amazon.

I got the lasers a wired up but the grab bag of photoresistors were proving difficult to work with. Let me correct myself...The photoresistors were working exactly how they were supposed to, but not how I wanted them to work. After consulting Prof. Dec I was able to use a multimeter to choose resistors that are better suited for the mystery photosensors. The photosensors are normally analog sensors but I wired them as a digital on/off sensor using a voltage divider but I was using resistors that were too high and no light could be bright enough to trigger the sensors the way I had wired (3)them. A minor setback, but a real learning experience.

I had planned on coving the lasers and light sensors up after learning the hard way with my laser harp. I would use hot glue to aim the lasers and would not have to worry about aesthetics because they would be out of sight! Hopefully it would all work out according to plan.

Poking out through the top with a little hot glue

Secured with hot glue!

Nestled in their laser-cave

The last part was getting the drum pads to work. They can't flop about but they also need some breathing room to move when hit/tapped because the movement of the sensor generates a small electrical signal that signals a note-on message. After some trial and error I got them working to a point where you could get away with a light tap instead of a whack.

I also had a goal of writing the code myself, or as much by myself as I could as a test of what I have learned. I found the using the MIDI_Controller.h library was easy to work with and allowed me to change MIDI channels as necessary(because I would want to assign a separate instrument for the harp, buttons, drums, etc.).

At this point in the build the meat and potatoes were all ready(meaning all of the electrical things were working and I needed a bottom(something I hadn't planned out before. I printed a bottom and had a think about how I might use spacers or dowels to bridge the top and bottom.

I headed to the Thingspace to put everything together(maybe) and chopped up some dowel rods and glued them into place. I took some inspiration from Jackson Pollock and drove the "Palette" idea home with some paint splatters.

Here is the Final Product:

Rube Goldberg Machine

I'll start out by saying OKGO is a inspiration in many ways to me. The way they think and brainstorm in creating their music videos is fodder for my own creativity as well as my students. I particularly love their video Needing/Getting. I would show This to Shall Pass to my classes when I taught EFL in Korea once a year for a summer camp lesson. We always had a fun time planing and explaining them. 



I decided to keep it simple because I know that there would be some troubleshooting needed when the group was going to be putting it all together during class the following week. I erred towards reliability and predictability in my servo code. I had envisioned some sort of ball or cylinder rolling along to hit a button that would trigger the code on the servo...which would trigger something else!

During class we started out with a plan on paper and then pieced it together. I had a lot of fun trying things out and seeing where we needed to tweak and reenforce. The first time we did a run through it worked perfectly to our surprise! When we tried it a 2nd time the balloon didn't pop, though... oh no!

Scratch Day

 On December 1st I attended the Scratch Day at TC. I wasn't sure what to expect but I knew that people where going to be there because the registration was filling up quickly and some of the workshops were already at capacity by the time I signed up which was less than a day after registration opened.

 I opted to sign for a double session music-themed workshop that featured the Circuit Playground Express. This is a micro controller that has a lot of built-in capabilities like velocity, gravity, light and temperature sensors in addition to the 10 RGB LEDs on the front of the controller. It's form factor is similar to the LilyPad Arduino I had used before, but after looking at the pinout diagram I saw that this was a very versatile controller.

The theme of the session was to make a "trumpet" that responded to pitch and volume by tilting the controller on it's X-axis and Y-axis. While it is possible to program the controller with the Arduino IDE software we used the Makecode website that had a browser based block dragging coding format similar to what the microbit uses.

I followed the tutorial on Adafruit's website for this, and it was a lot like following some Lego instructions. The instrument itself leaves something to be desired as far as being a responsive musical tool, but I think that the controller could have the potential for some very creative musical projects. 

A Fantasy Musical Instrument

It was fun to let my imagination run wild for this project. Of all of the projects so far I think this one got rolling the quickest.

My initial idea was some sort of VR musical instrument, I was inspired by lots of things but, somewhat embarrassingly I drew quite a bit of inspiration from the Holophonor from the Futurama Animated Series. Upon further investigation of this I found that the writers of Futurama were inspired to create this instrument from an instrument that Issac Asimov includes in the Foundation series of novels called the Visi-Sonor.

My device will have the player interact with a grid laid out in front and around. How the player moves his arms/fingers through the space in front of them will manipulate the sound that the device produces.

This requires some sophisticated sensors and computing power, but it could be a simple monophonic instrument like any wind instrument or a multi-phonic instrument limited only to how the user customizes the interface. For example, the piano is limited to one sound being played on up to 10 different notes. The interface could allow a single finger to play multiple notes depending on the depth the user is occupying in the virtual grid.


I imagine this might appear first on a heads-up type of display or even a google glass device.

I want to have the note and tambre selection isolated in the arms/hands/fingers. So the note velocity could be dictated by a breath controller. These already exist as a potentiometer. I would add some sort of haptic feedback device to help the player establish a better affinity with the virtual device. 

I was excited to get this little touch screen shield for the Arduino to work with I want sure how I was going to use it because I knew that the complexity of the code would be limited because the memory of the uno was just 32k. I was hopeful at first, it was Adafruit product so there would be some decent documentation.... My hope was my downfall. 

I'll was expecting to have to upload a library... But not three libraries, I missed one of them and it caused me about two hours of strife... Learning moment I guess. Eventually my hard headed approach worked and I got an example sketch to run...I needed that as I was getting frustrated. I still want sure how I was going to do my project but I knew that it worked, even if I barely knew why. 

I thought it might be a good start to use one of the sensors I got in my starter kit but never got around to using. I found a temperature/humidity sensor that, taking into account the changing weather might be interesting to get to know. I found through the model number on the back that this was a fairly common sensor that had well supported libraries. I choose to not see it up with the touch lcd screen yet as I thought it was best to tackle one new thing at a time. I set it up with a 16x2 lcd screen I had used in a prior project and copied some coffee into the sketch and fiddled with the program to have it display both Fahrenheit and Celsius. After I was more confident manipulating the sensor code I focused on the touch lcd.

I ended up taking a few lines of code from example sketch and using the sensor code from the small lcd sketch to, eventually, make it work. There was a few troubleshooting issues that I had to address: whatever is displayed stays displayed until you cover the whole screen in another color, I used black. This has the effect of resetting the display. 

When the sensor is powered on it takes about five seconds for the sensor to give reliable data so I put in a waiting screen in the code. It doesn't actually relate to the sensor's loading progress but it does allow the sensor to boot up and give an accurate first reading. 

I was happy with how it was running so I experimented with some shape drawing, I can better describe this in the demonstration video. I know that there is a more efficient way of drawing Therese shapes but I wrote the coffee how I understood it and I had very few errors and bigs to troubleshoot! I learned some things!

MicroBit Alarm

To create an alarm device using the MicroBit's sensor(s) I partnered with Marianna (after successfully stabbing a balloon via MicroBit) with to sort out how to make an alarm device with our two combined MicroBits.

We had a head start using some of the blocks from the balloon pop code that outlined how our MicroBits would talk to each other. After trying out the different light, sliders, sound and water sensors we decided on using the light sensor to make an alarm that could be triggered by opening a drawer or box.

Mariana's MicroBit would be the "sensor" and mine would be the alarm/recorder. Using the speaker add on for my microbit I wrote a code that would play a sound when the light sensor on the other MicroBit triggered a radio message. I also added a block of code that kept track(roughly) of how long the box/drawer had been open.

There were/are a few bugs like the LED images not reseting and sometimes a line of code will not run completely. We tried inserting some "pause" lines but the bugs weren't gone completely. 

You can see Marianna's code here.

MicroBit Light Sensor Instrument

I dove into this project with the idea of using the speaker add-on that allows the Microbit to play sounds with a little more elegance than using alligator clips and a stereo jack.

I have used micro-controllers as musical instruments before, but never with the Microbit so I was a little unsure on how I wanted to use it.

I decided on viewing the Microbit through the lens of a student, and concentrated less on what I was making but how I was using the tools to create my project.

I started by cruising the music project lists on the website before clicking on this guitar-type thing that had a few different projects within the instrument. I liked the way the light sensor tone production project looked and started following the tutorial.

It was not very hard to replicate the project. The drag and drop code blocks were similar to scratch so it was easier to try and find the right blocks to use(compared to the first time using Scratch).

When I ran the code with the light sensor, I noticed a few things:
     1. I need to calibrate the code to the space I am using it in.
     2. It seemed very sensitive.
     3. The sound does not stop.

I then was inspired to find solutions to these observations by changing the code to better suit my vision.

Step 1: Calibration

I could adjust the scale that was graphed by the  Microbit to get a better feel on how much light was 255 and how much light was 20 and all the differences in between.

Step 2: Limit available notes

I wanted to be in control of what note was being played so I decided to make the amount of notes availible to be played less. Specifically just 6 notes, a pentatonic scale C-D-F-G-A-C. I then made some code blocks that made the microbit play a certain note if the light level was between two numbers.

Step 3: User controlled sound

This was the most simple of the solutions. All I had to do was replace the "Forever" block with the "When A button is pressed". Boom! now I have a user, controlled light sensitive instrument!

Here is the finished code:

Video of the Microbit:

Sewing LEDs

After finding sewing a circuit in class was not majorly difficult(just making sure not to cross threads because there was not the kind of insulation I was used to with normal wires), I decided to test out the Lilypad Arduino to see if I could incorporate it my sewing circuit project. First I started with some simple fade sketches and then tinkered with the variables within the code to see what is looked like with multiple LEDs. Feeling comfortable I then turned my attention to the Lilypad LEDs I got from the Thingspace. The LEDs recive a data signal telling which RGB combination to send power to(and how much power). Also the lights are meant to be strung together and the code has a number to change for how many lights are included in the circuit. This is quite nifty, but it requires a positive, ground, data in and data out connections.

I had an idea of using many LEDs to create a moving effect, like a sign for a business or something like that but after stringing up some circuits I new how much thread I would have to use and I did not know how fast progress would go and what kind of troubleshooting I would encounter.

I then moved on to selecting the piece of clothing I would be sewing up. I ended up deciding on a messenger bag I had won at the end-of-year raffle at the school last year. It is a fine bag, but I have another one I like more so it has never been used...perfect!

I decided to attach the LEDs to the outside of the bag, because I could put the battery pack on the inside pocket and still use the rest of the bag without disrupting things too much. It took just one attempt at putting the needle through the nylon fabric before I realized I had made things tough for myself.... The fabric was so dense and it was almost impossible to shove the needle through. It was not 100% impossible and after stabbing myself once in the thumb I found a method using tape to increase my grip on the needle. It was still quite difficult, though.

I fumbled through sewing the LEDs and then using regular thread I secured the Lilypad to the inside of the bag and found that I was running quite low on thread! I really underestimated the amount that I would need. I had just enough to finish the circuit and reenforce some connections.

I tested it out a few times and it seemed to work with ~80% reliability. This was mostly because I should have made a few more stitches to make the thread less floppy. I may experiment with some tape to keep the threads in place.

Here is the finished product!

Spooky Card

One of the goals of taking this class was to put a little "theory" behind my "practice". In music ed there is a lot of published material under the umbrella of Theory & Practice. Well, in a "same page, different book" I was a little "practiced" in using circuits and LEDs but could do with a bit of theory to help figure out why some things worked and why other things did not.

I had an "A-Ha" moment when talking about parallel and series circuits and how I could experiment with the two different wiring methods to make this card work for my vision. But before I could finish, I needed to start!

Easy to sketch with a blank sheet of paper

I started laying out Some of the copper tape with a loose idea of stringing up a few LEDs along the way. It ended up looking sort of like this...

This is the "after" product with the required modifications in place to minimize the inconsistencies I had encountered in testing. As you can see, I have wired the LEDs in series. Also, I got frustrated putting tape and to maintain contact and ended up soldering the +- terminals to the copper tape.

                                      

This was the general shape that manifested itself as a result of my sketching. Maybe it is a little bit of a backwards design seeing where the LEDs ended up and making my card around it, but it seemed to work. 

Next I was inspired to "Increase my Challenge" by using some sort of switch. I had seen a youtube video using a microphone as a sound sensor switch... Maybe this is how the Artwork "Dandelions" by Jie Qi did it. Anywho, I rummaged around in my box-o-things and found what I thought was a microphone module for use with an Arduino. So I did what any electronic moonshiner would have done a duct taped an Arduino and breadboard to the back of the card. 

"Moonshine Electronics"

After adjusting the sensitivity of the microphone I started to get a device that sort of did what I wanted it to do some of the time! Blowing on the microphone creates quite a lot of input so I went with that in mind in order to weed out any ambient sounds that might trigger the switch.

A happy accident that happened to result is the "flickering" the lights make when the user blows on it. This might not be desirable for a winter holiday card but it does suit a Halloween themed card.

Here is the finished product:

Makey Makey

I had some trouble getting rubber to the road for this project, I put myself under some pressure to try and reinvent the wheel eight different times. To my detriment I think I forgot to just have fun with it for a while.

...3 days later...

This project and spoke to me not because I'm a music teacher, but because I have recently dove into designing and building my own electronic instruments using Arduino. I never played with a Makey Makey before so after worrying too much about making something fantastic I grabbed my thingamajig box and started hooking wires up to things and fooling around with the apps.

Recipe for... Fun!

This, of course, turned out to be a better catalyst than any amount of thinking I had done and soon after I started I was taking apart my french press to try and make a trigger out of that. My lovely girlfriend came home after a long day to find a mess in the kitchen, but this time it was a mess of wires!

Grounding

After experimenting with different methods of grounding and completing circuits I came up with a "Side-dish-o-phone"!

My experimentation yielded these "observations"...

-French Press was the coolest to use but most temperamental.
-Pickles were the most consistent to trigger sounds.
-I needed to wrap a chopstick in foil for it to work.
-Kimchi was as easy to play as the pickles(salty & wet?)
-Ice cream worked equally well frozen and soft.
-Tea bag needed to have a completely soaked string for it to work well.
-Hummus was fun!

Here is a video of the note demonstration:

I also had a little bit of fun the the french press, some butter knifes and the program "Soundplant". Soundplant is essentially a sample cue application. You can load up audio files to play and assign them to play after the designated keyboard key is pressed. I couldn't resist using an audio clip from one of my other projects to nerd-out a little bit!

The short essay with Jay Silver resonated with my drive to develop an "Invention Literacy" not only within myself, but with my students as well. I have found that creating my own midi instruments has made me feel more "alive" than I have for a long time.

For those curious I keep a process portfolio with my different projects. Check it out here!

Scratch program for Makey Makey: See Inside

Fodder for growth

Papert had such a bold memory of his fascination with gears. He saw how this experience drove him to explore other areas and relate new challenges to in terms that he could understand. Perhaps more importantly he recognized this "falling in love" as something intangible that spurred future learning and experimentation. Papert recognized that the fact that this event had anything to do with gears was of minor importance, instead he knew that the value was in developing a deep emotional attachment with...anything at all!

I think my earliest memories might go back to when I was 3-4 years old, nothing before that. As I was searching for my version of "gears" I looked at where I am now and what events might have been pivotal to getting me here.

There is a piece of music that has been a part of my life for quite some time. Specifically the opening melody to Mussorgsy's "Pictures at an Exhibition". I have really liked the entire work, listening to the orchestral arrangement by Ravel and watching 10 different interpretations of the piece played by artists performing at the Van Cliburn Piano Competition. I've played movements on the piano myself, and even performed the 1st movement playing MIDI trumpet with a synthesizer ensemble. This piece has been a revolving fixture in my musical life for more than two decades. But I was not in a musical setting when I first heard it....

https://scratch.mit.edu/projects/248724398


I first heard the piece when I was playing a computer game, I didn't know what the piece was... I just knew I liked it.

I struggled a little bit with programing this game. I tried looking for other scratch programs that did what I wanted mine to do but I couldn't find any that I could use.
One that came close was this...


I was looking for a way to have the correct melody trigger the final screen... but I couldn't find a way. It turned into a bit of a black hole for time. I would start working after dinner and then the next thing I knew it was edging in on midnight.... oh dear.

I'm sure it will become less tedious and more efficient that more I work with the platform, but it all still seems really thick to move through.

Turtle Power!

Although I encounter devices running computer code every day in the music classroom, I usually do not stop to think about how a device does what it does. I have thought from time to time that it would be nice if I had an app that could do ______. But alas, my knowledge writing code and programing is limited to merely knowing about it or knowing that such a thing exists. I found it sort of funny that I was one of only a few people that had to download the "Java development kit" in order to get the Turtle Art application to run... I assumed it was because other people have had to download it previously for other programing projects, hah!

I found the application easy to use after only a few minutes of tutorial and then I was enjoying the sandbox nature of the program.

Drawing squares has never been so rewarding!

After playing around some more I felt like I had enough of a tenuous grasp on things to try and draw something inspired by a piece of art. 

I think I was equal parts inspired by my doodling to find a piece of art that reminded me as I was inspired to create and modify my program to emulate the artwork. 

The two methods sort of built upon each other and what seemed like a few minutes later(in reality a little more than an hour) I created an interpretation of this field of flowers!

ta-da!

I started writing the program to create a flower shape and then set the xy coordinates to random. I decided a smaller flower would create some balance and variety in the drawing so I wrote that in the code as well as a random color selector. I experimented with the "if-then" tabs and got the flowers to be drawn as 12-sided polygons if they were assigned a y-value greater than 0. I included two sizes to polygons in each of the two strands of shape producing code(one for medium and one for small. After some tweaking with the color palette a little bit I ended up with the result above. It's not a nuanced as the painting, but I think keeps in the spirit of the original work!

My next piece was inspired not by a piece of visual art but by a piece of music by Steve Reich. The piece is the 1st movement from his larger work titled Electric Counterpoint. 

The piece is minimalist and really drives home slight changes of periods time. The timbres, ryhthms 

and melodies are very angular which inspired me to choose the diamond as the theme that would repeat and change over time. 

I started writing the code to do a single diamond, then a row, then a second row and then the whole screen. I experimented with different kinds of repeating... offset, bigger, smaller, and then decided on the slowly evolving filling in of the interlocking diamonds. Like the piece before I played around with the color selector but this time I wanted more order and predictability so I used an equation to that(somehow) ended up filling in the diamonds with a uniform color each time the turtle added on. While the finished product is relevant to the inspiration, I think the full experience is running the code while the piece plays and watching the turtle draw to the music!

The Turtle's Process:

The .png files that include the code.

Making an interactive Thingamajig

Step one: Overthink Everything

Days prior to actually putting anything together I was wracking my brain to think of some way to combine these things in such a way that would be, in a word, extraordinary. I may have done myself a disservice by not digging into Rob Faludi's webpage with the very useful phrase, "Don't over-think it". Well, it was a little late in the game to un-over-think things but I did just that, I stopped by the dollar store and grocery store to pick up some "synthetic" feathers(see description/reasoning in the picture) and some lentils... Somehow I would cook up something for this project.

The "feathers" are substituted with party blowers because the real feathers I found on the ground were from pigeons, and I'm just not sure I'm at a point in my life where I feel like I should be collecting pigeon feathers off the street in Astoria.

I had this idea to combine the cup and lentils with some kind of movement to make a sound. My inclination was to make a shaker kind of percussion instrument but I would experiment with the other movements to see if anything surprised me. 

While many of the "inputs" to the lentil-cup did little except make me glad there wasn't a mirror or anybody watching me, dancing elicited the most interactive result. So, dancing lentil-cup it is. 

There was more to be done, though. I had spent all this time trying to find a not-gross feather on the ground in NY and settled for some feather-esque things and you'd better believe that I was going to find a way to use them. 

While I was walking around (I do a lot of walking, it has both a brainstorming and mind-cleansing effect) I had the idea to try and combine the feathers and breath. I had imagined the feathers flying up and around under the power of me blowing on them; that was the rough idea, not I had to figure out a way to do it. Naturally, the first step was to go to the kitchen and get the bread knife.

Bread knife...birds eat bread...birds have feathers...this is a feather replacement...RATIONAL CONNECTION MADE!

It was sometime during this process that I had the idea to mount my creations into a box from Amazon. I thought it would help in making seem a little more interactive? Rather than just dancing around with a cup full of lentils...

Draft #1:

Breath goes in... something something something... fake feathers move

It was late and I thought I would benefit from coming at it again in the morning, so I went to bed and dreamt about dancing around in a land of lentils. 

Prototype breath direction channel:

This think kind of reminds me of something else, but I just can't put my finger on it... I know, I'll jam out on my melodica to clear my head. 

Wait a minute, that might just be...

Perfect!

It's like it was made for this.

I ended up just getting rid of the plastic mouthpiece part entirely and rolling the feathers up and sliding them into the end of the mouthpiece from the melodica. It actually reminds me a little of the flailing tube people at car dealerships, so maybe I would call this output a "dance as well!

At the time of writing, I think I may be at a finishing point of sorts. I did drill an extra hole in the box for no reason whatsoever so maybe I will dream up a possibility for it tonight. #workinprogress