Flagons Den

I have gone through the baptism of fire with the live demo in front of my peers, and I think it went pretty well. I had quite a bit of interest about the looptop and there was a lot of people testing it out.

Mr Fox Rocking the Looptop

I kept it short and sweet, but thinking back I could have done a small performance to show the potential.  Unfortunately the video of my demo corrupted so here’s some homemade videos showing the looptop in action.

There was one other person - Jav - who was using ReacTIVision and he managed to get a whole library of samples playing simultaneously which is an amendment I need to add to the Looptop. That would give a considerably better aural arsenal and improve the soundscapes. So keep a lookout for the beta version.

What is Looptop?

Looptop is a tabletop tangible interface, which captures movement of reacTIVision fiducial markers via a webcam and MaxMSP utilises this to create MIDI sound, control sample manipulation and generate visual representation.  The captured movements are looped at 4 second intervals to create a soundscape and evolving graphical image.

The use of the looptop prototype encompasses several models – it is both performative as it allows the live creation of a soundscape and visual drawing and exploratory as the relationships between the symbols and resulting soundscapes are only discerned by playing with the markers.  The aim is to create a interactive device which allows non musicians to create a soundscape through movement capture and provide visual feedback of their movements.

Development Update

I have finally completed the project - bar a few lines of documentation.  I have managed to get all the bugs fixed with the sample playback and the fiducial markers are all mounted on foam board with the appropriate icons on the top.  The opaque glass (tracing paper) technique didn’t work as efficiently as I hoped and so I had to scrap it - which means the inner workings are visible but as this is only a prototype I’m sure thats ok.

Looptop prototype

I have made a small demo video - unfortunately the sounds from the looptop are a bit quiet (I forgot to turn up the speakers enough) - but it explains the functionality.

I have made a quick track in one take using the looptop to show the performative model - though you would need to get the full AV treatment for it to feel more of a performance. I need to hook up the patch to a projector for that. Instead - heres a screenshot of a drawing I made earlier and if you look at it while playing the track…. well you have to use your imagination admittedly.

screenshot of MaxMSP patch drawing the soundscape

Looptop track 1 Click to hear the track.

Completely change tack and rethink your whole approach.  And guess what? It works! I have sacked off the colour tracking after many hours of effort but little positive results to try instead the reactivision software. I found it pretty hard to find my way at first, as they are determined not to give you any examples - and the supposed patches supplied for max tried to open in photoshop!

reactivision fiducial markers

I was able to remove the concerns about tracking colours and ambient light due to the easy camera calibration setup in reactivision - the markers were extremely clear to see but I couldn’t figure out how to take that info into MaxMSP - I had the TuioClient Object installed and it was giving me reams of numbers but I didn’t know how to unpack it to make use of the data.

After some serious forum digging I found an extremely helpful post (here) which had a working patch using the TUIO interface into MaxMSP - Thankyou!. I was able to utilise the patch and incorporate some of the functionality into my existing paintsounds app substituting the jit.findbounds object to capture the x and y input for the data captured from the fiducial markers.

I have assigned a sound and drawing patch for four separate fiducial id’s, capturing x and y info for pitch and velocity and the rotation for the duration of the sounds. The addition of a fudicial object triggers the sound and starts the drawing loop and the removal stops the the loop. I used two more for other functions. One clears the lcd screen and the other triggers a vocal sample which can be sped up or slowed down via rotation.

Once I figured out how to get the fiducial markers data into MaxMSP and Ableton the results were pretty impressive and very straightforward to implement. I will now add a few more samples to play with, make the markers more engaging and tangible and make the glass top of my light box opaque - and hopefully that should be the project completed (bar the paperwork).

Well, I have hit a wall with my project. My problem was parsing the colours to a jit.findbounds object so instead I created four of them with a min and max bounds for each colour - red, green, blue and yellow. The objects picked up the various colours and sent an x and y variable to a drawing subpatch which would draw the paths of a tracked object and play a sound accordingly - which works lovely but only with a movie or animation.

screenshot of patch working with an animation

When I then used a webcam - the tracking subpatch couldn’t find anything. I tried ramping up the colour and brightness with a jit.brcosa object but this became messy and only really found red colours.  Even after adding a suckah and then recalibrating each jit.findbounds to match the colour tracking still didn’t work.

I thought it must be down to the variance in ambient light - so I attempted to build a lightbox to ensure a consistent level of light under all conditions.  The design was to have a light and webcam inside a box with a glass top - the objects to be tracked would be lit and seen from beneath and so would be easy to calibrate.

wooden box

box interior painted white

light and webcam installed

lightbox completed

It looked great, unfortunately the outcome was awful. The light reflected into the webcam and bleached out most of the image, causing the tracking patch to miss the objects. I tried to diffuse the light with cloth but the light problem persisted. After a few tense hours, I then tried a new tack. I used an anglepoise lamp and webcam set up above the lightbox with the jit.brcosa to raise the brightness and contrast which made the colours closer to pure RGB.

Webcam and lamp above diffused lightbox.

However - I still cannot get the tracking to work properly - only a stuttering bit of red and blue. Which is infuriating. I will speak to Dan Livingstone once more tomorrow and hopefully he can help me - or put me out of my misery!

I am having a mare with this colour tracking motion capture - the cv.jit library has such little tech support for the tracking objects that it’s less than useless. Mining through forums and previous projects has raised more questions than answers and currently I am creating a theramin by waving a blue book at the i-Sight camera - which is fun but predominantly pointless. So far the huge sticking point is the ability to parse the RGB to a jit.finbounds object with a suckah. I can see the values for each and I’m using the $ syntax to try and set the min and max values dynamically from these float number objects but - no dice. I cannot find anyway around it - which makes me know it will be a really obvious thing to overcome - but after hours of staring at a patch it becomes more and more indecipherable.

I tried EyesWeb but after downloading and putting the appropriate files in the designated spaces it wouldn’t work - maybe it’s another anti Intel Mac thing. So now I am completely stymied.

Anyone who wants to post an easy ‘how to’ fix in the comments feel free - Help!

Doyouthinkysaurus

I downloaded cyclops for maxMSP to try to improve the colour tracking but I can’t get Max to utilise the object from the library which is pretty frustrating.  I have made my own rough version using four different flash animations as test movies to read the colour info from using the jit.getbounds object - which works but has some bugs with redrawing the patterns.  Will work on it tomorrow - but I’m pretty sure I will have to unpick the patch and start from scratch.

I have worked up a rough and ready and also non working version of the paintsounds applet which is fed from colour tracking instead of drawing - It sort of works but slowly grinds to a halt when its running of the i-Sight camera.  My plan will be to have (maybe) a top down camera on a white background and the interface being a cube with different coloured sides.  Each one triggering the different instruments and the captured movement drawing the sound. - The resulting drawing loops will be projected on a screen.

I had another think about the MC app - another factor at live shows is getting audience members to suggest topics to freestyle about, and shoving the mic in their face in front of a crowd often makes them uncomfortable or come up with generic responses.  If I were to make a database of responses from members of the public answering the question - What random topic would you like to hear a rapper freestyle about? - I could make a fruit machine style randomiser which could be used at live gigs through a projector to initiate freestyles.  A sort of rappers vox pop.

VJMC

I did a trial run of my VJMC idea - and the results were unimpressive to say the least.  Too repetitive, and lacking any real ability to pose a question/answer response as I would have hoped for. So that’s scrapped.

So that leaves the PaintSound 2.0 app idea.  Creating a colour tracking tool which will track different blocks of colour and make that into the x and y coordinates of the lcd panel input to drive the applet and create the sound and image.  I have looked at the jitter tutorials for video - which incorporate some colour tracking.  I just need to adapt that for camera input and set up the parameters for each colour to be read as a different instrument. I will work on this before the next tutorial.

This is the funniest thing so far this year.