Pitch

Been trying to work on it for a few days. I have no idea how to present it or what to include!! This is going to be very stressful! 

'Dance Pad' lets your feet do the typing

Steven Drucker, a researcher in Microsoft Corp.'s Step User Interface Project Group, uses a commercially available "dance-pad" device to demonstrate how the pad can be used to control an experimental application that lets users view, categorise, delete, and perform other functions related to their digital photo collections, all with their feet.Photo: AP

Ever feel like you're not making good enough use of your feet when you're catching up on your e-mail or sorting through all those digital pictures you took on that last vacation?

Computer scientists in Microsoft's research division have developed a colour-coded "dance pad" with buttons you can tap with your feet - or jump on - to scroll through electronic files.

It may never make it to store shelves, but then again, Microsoft spends billions every year researching far-out technologies without worrying about whether every gizmo will sell.

This week the software giant held its annual internal trade show where hundreds of researchers were showing off their work. The "Step User Interface" technology was one of the concepts available for a sneak peek.

"This is just one off-the-shelf piece of hardware we can use," A.J. Brush, the lead researcher on the project, said after demonstrating the technology. "Now we're looking at broadening, thinking about accelerometers or other things you could strap onto your feet so you really could be just sitting at your desk and kicking your email away under the desk."

AP

Proposal Final

 

Student Name:	Christopher Seymour (08024421)
Module Title:	Major Project in Interactive New Media
Title / Subject matter:	Footsteps (Working Title)
Submission Date:	06/11/2009
Description: Exploring my interest in childhood development i.e. learning skills, the psychology of learning, teaching etc. and my growing passion for interactive media, I wish to develop the theme of play explored in year two as a means to create a learning environment for the lower part of school age (i.e. 5 to 8) years. Employing a programme such as Processing, "an open source programming language and environment for people who want to create images, animations, and interactions" (Processing, 2010), for the software and pressure sensitive buttons integrated into a floor mat, the environment will allow the target audience to play a sound/image matching game. This will coincide with a neurological defect known as Synesthesia where people's senses are triggered to match at certain points triggered in their brain i.e. they would smell 'x' when they heard 'x'. Thus creating a simulation where users will hear part of a story and in order to progress through the narrative, react by reacting to images on the interface to make a sound within the environment The Interface is going to be an important factor to the game where each move will rely on footsteps or action. Reaction will play an important part, much like Nintendo Wii and the recent venture from Playstation, 'Move' the user will need to stay focused to be able to progress and not become bored of the application.  The aim would be to so that children of a young age are able to assign sounds with images as well as build on their memory skills , through the understanding that they would need to remember to correctly identify the sounds with a sound they've heard before. A sound will play and the child would then match that with an image on the pad but increasingly becoming more intense, only so much that a child will be able to mentally challenge. Therefore as the application is designated for that particular age group, my application will be designed for a playroom for young children. The design itself should be childish so that they feel comfortable within their surroundings with the ability to relate to their everyday environment at that age. Research, published in an article on synaesthesia by Ramachandran and Hubbard (R&H, 2001), shows that children routinely exhibit cross-modal associates very early in their lives. When presented with auditory stimuli followed by visual ones, infants were found to make preferential mappings of values from one sensory modality to those of the other. With that in mind, I feel as though I should solely concentrate on infants as a learning material as opposed to widening the application so that adults are able to use it as well.
Objectives: -To effectively learn the 'processing' application to the best of my knowledge so that it may be used to undertake this project. -Create a children's application that will be perceived as a game but also act as a piece of leaning material. - Correctly identify and match the audio/visuals to fit into a childlike linear narrative. - Design a custom made floor mat for the interaction between user and application.

Techniquest

- Posted using BlogPress from my iPhone

Proposal (Draft 1)

Student Name:	Christopher Seymour
Module Title:	Major Project in Interactive New Media
Title / Subject matter:	FOOTSTEPS (WORKING TITLE)
Submission Date:	06/11/2009
Description: For the final major project, I will reflect on my learning during the second year and explore the theme of play. Whilst the application, Director was useful, I'm going to take on the challenge of learning a global, user generated application called Processing.  Interface is going to be an important factor to my game where each move will rely on footsteps or action. Designated for children, my application will be designed for a playroom for young children. The aim would be to so that children of a young age are able to build on their memory skills as well as correctly assigning sounds with images. A sound will play and the child would then match that with an image on the pad but increasingly becoming more intense, only so much that a child will be able to mentally challenge. This would coincide and requires a lot of research into a disorder known as 'Synesthesia' where people's senses are triggered to match at certain points triggered in their brain i.e they would smell 'x' when they hear 'x'. Research suggests that children routinely exhibit cross-modal associates very early in their lives. When presented with auditory stimuli followed by visual ones, infants were found to make preferential mappings of values from one sensory modality to those of the other. With that in mind, I feel as though I should soley concentrate on infants as a learning material as opposed to widening the application so that adults are able to use it as well.
Objectives: -    TO EFFECTIVELY LEARN THE 'PROCESSING' APPLICATION TO THE BEST OF MY KNOWLEDGE SO THAT IT MAY BE USED TO UNDERTAKE THIS PROJECT. -    CREATE A CHILDREN'S APPLICATION THAT WILL BE PERCIEVED AS A GAME BUT ALSO ACT AS A PIECE OF LEANING MATERIAL. -TO MAKE SURE I KEEP MY DOCUMENTATION OF RESEARCH AND EXAMPLES THAT I HAVE FOUND UPDATED REGULARLY, MORE SO THAN DONE OVER THE LAST TWO YEARS. THIS WILL TAKE SHAPE AS A WEB BLOG
Assessors Comments:

Dance Pad Came

The dancepad arrived this morning, a week and 2 days after ordering it. I hate having to wait for things for so long!

I think I've finally come up with an idea now which, will be in talks today with G @ 4!
I wrote my proposal which I'll add to my next blog.

something I came across in a book

It's quite clever actually. Where it has nothing to do with children, it does have something which relates audio visuals. The paper, I assume is made from plastic bubble wrapping which when pressed would represent the sound a type writer key would make, hence seeing sounds !

Ideas

.

    IF AIMED AT CHILDREN.

• MATCHING SHAPES WITH COLOURS?
  
  • – EACH BUTTON PRESSED WITH BE CORRESPONDED WITH A SOUND
    
• EACH COLOURED BUTTON WILL HAVE A PIANO KEY SOUND ASSIGNED TO IT
  
  • – THIS COULD BE DONE WITH NURSERY RHYMES E.G. IN DIFFERENT ORDERS, THE PRESSED BUTTONS WILL MAKE UP A NURSERY RHYME?
    
• A GAME THAT LOOKS LIKE PAIRS? USING YOU'RE FEET, MATCH THE COLOURS ON THE BOARD, TO THAT ON THE SCREEN.
  

 

IF AIMED AT ADULTS.

• MAYBE AN INTERACTIVE TWISTER? W/ VIDEO CAPTURE, SO THAT YOU CAN SEE WHERE YOU ARE ON THE SCREEN. THE SCREEN WILL TELL YOU IF SOMEONE HAS TAKEN THEIR BODY PART AWAY FROM WHERE IT'S MEANT TO BE?
  

Nightmare

I think I'm suffering from writer's block. Except it's got nothing to do with writing but more so with that of trying to think of a sufficient idea that'll work and that I feel is passable.

The idea now was to design my own 'dance pad' so to speak. For this, I'd need conductive fabric, switches and crocodile clips. I wish I had enough faith in myself to just do it.

THERE ARE JUST SO MANY DIFFERENT OPTIONS TO CHOOSE FROM & THIS CONFUSES ME!

I mean, look at this website http://www.kobakant.at/DIY/?p=376 

My current (of the electrical kind) terminology is very limited (on/off) so as far as I can make out, everything looks the same and does exactly the same thing. They just range in textures - zelt, copper, zinc. What on earth is zelt?

In other news, as from the picture above, from what I can see, the pad has 18 buttons, which (providing I come up with a substantial idea) would suit my needs. 

Plus, I go out a book today from Cardiff Library. Good Luck to me eh?!

IN OTHER NEWS.

HOW COOL IS THIS?

"A Toy Piano embedded on a T-shirt. It has 8 keys from Do to Do (1 octave). You can play simple music by wearing the shirt and pushing the fabric button on the shirt. All the components from the toy piano (batteries, speaker, circuit board) are placed on the shirt and connected with poppers. All these hard components are detachable so that you can wash it if you wish.
This particular Instructable is made for the Electronic Textile workshop that will be held in Zurich/Switzerland on Saturday 7th December 2009 as part of the DIY Festival Zurich . If you are interested in this workshop, please contact the festival.

Sourced from here.

Yesterday

I went to see Garrett yesterday, whether or not it laid my mind to rest remains to be seen. After discussion, it appears as though I'm going to attempt to make a a twister mat... but electronically so that it will correspond with whatever game I intend to make. I think, the way I described it, would mean that I'd be aiming this piece of software at children so with it I could perhaps make levels where they'd need to correspond the colour with a piece of sound or shapes with that on the screen.

I need to see how I could aim it at adults also because as G said, not many adults are going to want to take their shoes off and play on a mat.

So, apparently I need conductive fabrics. There's one on mutr.co.uk for £32 but I've sent an email to Garrett asking if http://www.mutr.co.uk/product_info.php?cPath=6_572&products_id=1009619 will suffice.  We'll see.

Jane Mackay

Jane is an artist, born and bred in London. Her art is not typical or contemporary and would only relate to a handful of people. She paints, what she listens to. 

"I capture these pictures and release them into my paintings and prints to produce unique and personal works charged with creative energy."

She goes on to say on her website, Sounding Art that she is extremely fascinated by the relationship between the visual and auditory parts of the brain she captures and commits to paper the images she visualises when listening to music. She has found these 'synaesthetic' images a constant source of creative inspiration and, together with her love and knowledge of the classical repertoire, fundamental to her life as an artist. She is both a choral singer and an oboist. Jane is best known for her Britten Series paintings, over two hundred of which are distributed in collections worldwide.

I think this above image, is one that particularly represents the music she listens to. She says on her website that her favourite genre to paint to is Classical, and I feel the images evokes that. That said, it also represents how music is often visually seen by those who don't have the disorder and that being Waveform. It does look like something I'd expect to see as a visualisation on WMP. 

Virtual Piano from Japan

"It was bound to happen sooner or later. Ever since those keyboards of the typing variety got all virtual on us, we knew the instrument version would go infrared and turn our rhythmic table tapping into beautiful music. The new device, from Digital Information Development, operates -- and looks -- quite like that Bluetooth version from iTech, 'cept there's a speaker in this unit to pump the tunes you're generating. Though there's just a 25-key keyboard in this version, DID is promising an 88-key "grand piano" unit, and even some sort of weighted notes (we'll believe it when we see it). This first iteration includes piano, organ, pipe organ and harpsichord sounds, and should cost around 15,000 yen ($130 USD) when it hits stores in Japan this November."

NOT THAT I UNDERSTAND WHAT IS BEING SAID, IT APPEARS TO BE A REALLY COOL PIECE OF KIT.
I ♥ the japanese!

Thumps Up

So, I've started to have a play around with Processing. It's tutorials seem to be easy following and straightforward. It's reminding me alot of html, which I liked very much so this could be the better program to get my teeth in to.

In this, the code sets out that where ever the mouse pointer is, then a white eclipse will be 'drawn' on the screen, hence the mass array of circles in the window to the left. But if the mouse button is held down, then the circle will turn black.

Horizon

Horizon never fail to make interesting programmes and thankfully enough, back in March 2006, they filmed a documentary on the disorder (bored of spelling it wrong over and over again). 


I stumbled across this section of their website, and even participated in a scientific experiment. It's safe to say I'm not a Synaesthetes. It was a strange experiment. It involved matching colours to letters, numbers and days of the week, sneakily, all 23 of them were repeated twice to total it up to 46 matching sequences. I also had to find a shape made of 2's from a picture filled with 5's. To cross examine this, this test also came up on http://mindbluff.com/syntest.htm where it said to detract the red from the green, being a red-green colour blind person, this was a no-no! 

Next asked what to assign a name (choice of two - bouba and kiki) to the shapes. 

It's apparent that 73% of people  who took the test on the BBC/Horizon Experiment, all gave the same answer, giving the shape on the left the name of bouba and kiki to the right. Stating the obvious in a way, I think our determination relies on the phonetic sounds. Kiki, has two very harsh and short sounds velar plosive's, so our brain automatically assigns it to the most harsh looking shape where Bouba is a very soft sounding word, so we assign it to the softest looking shape. 

Our Brains work is really strange and mysterious ways, but some more than other obviously for those who pass the test with results being that they are Synaesthetes.

Check out the BBC one 

Do You See What I See?

Do You Hear What I Hear?

The two links lead to to two different tests. Not to state the obvious or anything but one's visual and the other is audible. 

Sound → color synesthesia

Cytowic calls sound → color synesthesia "something like fireworks": voice, music, and assorted environmental sounds such as clattering dishes or dog barks trigger color and simple shapes that arise, move around, and then fade when the sound stimulus ends. For some, the stimulus type is limited (e.g., music only, or even just a specific musical key); for others, a wide variety of sounds triggers synesthesia.

Sound often changes the perceived hue, brightness, scintillation, and directional movement. Some individuals see music on a "screen" in front of their face. Deni Simon, for whom music produces waving lines "like oscilloscope configurations—lines moving in color, often metallic with height, width and, most importantly, depth. My favorite music has lines that extend horizontally beyond the 'screen' area."

Individuals rarely agree on what color a given sound is (composers Liszt and Rimsky-Korsakov famously disagreed on the colors of music keys); however, synesthetes show the same trends as non-synesthetes do. For example, both groups say that loud tones are brighter than soft tones and that lower tones are darker than higher tones.

Seeing Sounds - Example Numero #1

Research into how 'synesthesia' has led me to the critically accliamed album, 'Seeing Sounds' released by N*E*R*D in 2003.


After the album was released, Pharrell, the front lead for the band was questioned on what the album offered and why did they choose the direction they did for the music;

"Williams explained that the title was based on synesthesia, a neurological phenomenon the band learned of while watching the Discovery Channel. They were inspired to create music that could be conceived as a live show. He was surprised to learn not everyone experienced while listening to music. He described synesthesia as the outcome when "one sense gets more information than what's intended", stimulated by sensory deprivation. Haley further elaborated, saying that it is a mixture of senses that causes a person to experience colorful hearing and auditory smell. When the band created the album, it was the music they envisioned and decided the appropriate title for the album, Seeing Sounds. Williams pointed out that the lyrics were also inspired by synesthesia. He said that his fans want to "rock out" and be taken on an "emotional roller coaster", and the only way to do that is to give the fans some records they could blast out on their speakers. "

          http://www.boston.com/ae/music/articles/2008/05/11/for_nerd_its_time_to_ramp_up_production/

          http://emusician.com/remixmag/artists_interviews/musicians/remix_old_money_new/index.html

I think the title in itself being 'Seeing Sounds' is something to kind of emulate in my work. With that I could still use my piano/keyboard idea but give it a different concept i.e. assign each key with a spectrum of colours. 

Initial Idea.

On Thursday, I set out this idea to Garrett who made the interesting point in saying that research and planning for this would potentially be extremely boring. This, I have to agree with because alot of learning software's' designs are plain and simple, with the most obvious instructions. 

Idea #1

I wanted to create something which would perhaps fool the user into thinking that the on screen person was copying their moves on the keyboard. I didn't know how to explain it very well in class so perhaps Garrett thought I was chatting absolute nonsense. Then he suggested (to refer back to earlier blog) that I look into synesthesia and perhaps include things like  each key pressed would be associated with a certain colour or image. 

I'll get back to you with this.

Comparison

As previously mentioned I may have to learn a new programme. The choices are. Pure Date (Pd), Max 5, or Processing. They're all programming languages of some sort, each having their own code.



Pure Date is a visual programming language which is used to create and manipulate video, OpenGL graphics, images, etc. in realtime with seemingly endless possibilities for interactivity with audio, external sensors, etc.


It also gives networks across the globe, the chance to seemlessly creates things together in real time.
+ free. Wide online based community



MAX 5 has been around a little bit longer than PD, but it just looks easier to navigate with. Like Pure Date is allows users to create things and allow them to run together in real time. However, it's appears as though its more so used by musicians.

+ COST - $59 FOR 9 MONTHS USAGE (FOR STUDENTS)
(ALL THIS OF THIS IS VERY CONFUSING







Processing is an open source programming language and environment for people who want to create images, animations, and interactions. Initially developed to serve as a software sketchbook and to teach fundamentals of computer programming within a visual context, Processing also has evolved into a tool for generating finished professional work. Today, tens of thousands of students, artists, designers, researchers, and hobbyists who use Processing for learning, prototyping, and production.



+ free and open source. Wide online community with tutorials.

negative point : like director, it hasnt got very good video capabilities, which means if I wanted to create a project which required a full screen video image, then this would be a struggle.



<br>

Something is telling me that Processing would be the better option. There are alot of examples of the programme on the website, , to play with and libraries where you can sources audio. There are alot of tutorials available with the programme.