Gestural Graphic Interface

Partnering with two other people, my final project for ICM and PComp is the same: A graphic and musical interface that uses gloves  to enable inexperienced people to create, enhance and play music. After brainstorming for about a week and letting our simple ideas multiply and evolve into something that stays faithful to our project goals, and researching the scope and methods of other similar projects, we decided to take on a multi-faceted, holistic music creating interface that lets the user draw basic music notes in the form of doodles on a screen, and enhance it with the glove. The program will take inputs from the coordinates of the drawings to play and loop it, and from the sensors in the glove to add drastic effects, both musical and visual, to show the fun and the power in the art form itself.

Our inspiration through this research has been the Mi.Mu glove made by Imogen Heap, a glove that records, loops, plays and adds effects to sounds and music, while the user performs.

Inspirations..

Hand gestures — to make the music making process as intuitive as possible

To make this glove possible, we plan to build one using flex sensors, accelerometers, and probably even some tactile sensors, however that will greatly rely on the degrees of control we need.

Circuit diagram of one of the simplest glove designs

Our aim with the glove involves giving super powers to user when it comes to affecting change to the music. the first iteration might look similar to this one.

For the interface on the screen, our designs are still coming together, but the current target is around halfway between a traditional step sequencer and Kandinsky from the Chrome Music Lab Experiments. We researched a new library of functions for our project, Tone.js, and learned about its abilities and pitfalls.

Check it out:

https://musiclab.chromeexperiments.com/Kandinsky

The Progress:

We are currently researching the timeline, looping and how we are add and delete new user-added sounds to an existing loop. It is challenging but our progress is motivational.

Using Tone.js, we have so far created a sketch that uses this library to play beats at different frequencies and note lengths, and a sketch that creates markers to let the user draw, and maps sounds to each coordinate (and adds it to an array).

[Please refresh in case the sketches below do not work]

Or click the link:

TONE.JS

 

Or click the link:

MARKERS AND TONES

We are still understanding the workings of Tone.js and the next step is to serially communicate with the glove and add more changeable parameters to our sounds in code. We’re still in the process of acquiring materials for construction of the glove. This project has turned into an enormous undertaking, but it’s moving steadily forward, and I can’t wait to see how it develops.

 

Trash

Team Members

Amena Hayat

Yipeng Chen

Sam Chasan

For animation class, we were tasked with making a 30 second stop-motion animation with a character. Using Dragon Stop Motion, Adobe After Effects and Adobe Premier, we shot and compiled a short animation called “Trash”.

In it, we brought to life a piece of trash who comes to life when someone unsuccessfully throws it in to the trash basket, and takes upon itself the mission of going back home. After many attempts of jumping back in, getting super frustrated in the process of going “home”, dragging objects to use as high platforms, it gets lucky when someone else walks by and makes the task easy for it.

Here’s the final edit:

 

TRASH ANIMATION from Amena Hayat on Vimeo.

Faces on a slider

For my PComp midterm, I used a few frames of some hand made sketches that used a proximity sensor to change between them.

For the sake of learning and applying the HTML slider, I uploaded the sketches on to p5.js and used simple if conditions to traverse through the images using the slider and get the animation effect.

Here is my code:

function setup() {
createCanvas(420, 450);
one = loadImage(“faces/1.png”);
two = loadImage(“faces/2.png”);
three = loadImage(“faces/3.png”);
four = loadImage(“faces/4.png”);
five = loadImage(“faces/5.png”);
six = loadImage(“faces/6.png”);
seven = loadImage(“faces/7.png”);
eight = loadImage(“faces/8.png”);
nine = loadImage(“faces/9.png”);
ten = loadImage(“faces/10.png”);
slider = createSlider(0,10,0,0);
slider.position(10,height-40);
slider.size(width-20,20);
x = slider.value();
}

function draw() {
background(255);
console.log(slider.value());
if(slider.value() <= 1){
image(one,0,0,width,height);
}
else if(slider.value() > 1 && slider.value() <= 2){
image(two,0,0,width,height);
}
else if(slider.value() > 2 && slider.value() <= 3){
image(three,0,0,width,height);
}
else if(slider.value() > 3 && slider.value() <= 4){
image(four,0,0,width,height);
}
else if(slider.value() > 4 && slider.value() <= 5){
image(five,0,0,width,height);
}
else if(slider.value() > 5 && slider.value() <= 6){
image(six,0,0,width,height);
}
else if(slider.value() > 6 && slider.value() <= 7){
image(seven,0,0,width,height);
}
else if(slider.value() > 7 && slider.value() <= 8){
image(eight,0,0,width,height);
}
else if(slider.value() > 8 && slider.value() <= 9){
image(nine,0,0,width,height);
}
else {
image(ten,0,0,width,height);
}
}

Maternity and Race

This week for ICM I wanted to illustrate statistics for the problems women face, and I wanted to do it in a way that helps viewers understand the issues women of colour or lower education attainment face, and also make it easier to understand and visualise — using emojis of course.

I imported the data in a .csv file from https://mchb.hrsa.gov, HRSA for Maternal and Child Health, and read up on of their surveys done in 2015 for maternity leaves taken in the US, divided by race, educational attainment and reasons for not getting or taking the leave.

Maternity leave from employment after childbirth provides critical time for maternal-infant bonding and adjustment to life with a new baby. Longer length of maternity leave is associated with increased breastfeeding duration, as well as improved maternal mental health and child development. The WOMEN’S HEALTH USA 2011 stated that “the Family and Medical Leave Act (FMLA) guarantees both women and men up to 12 weeks of unpaid leave around the birth or adoption of a child as long as they work for larger employers (50+ employees) and meet certain tenure and working hour requirements.” However, many women cannot afford to take unpaid leave and usually use a combination of short-term disability, sick leave, vacation, and personal days in order to have some portion of their maternity leave paid. The U.S. is one of only 5 countries in the world that does not mandate paid maternity leave.

According to their study:

In 2006-2015, 66.0 percent of women reported being employed during their last pregnancy, of whom 69.7 percent reported taking maternity leave. Thus, nearly one-third of employed women did not report taking any maternity leave (30.3 percent). Women with at least a college degree were more likely to have taken leave than those who had attended college but not graduated (80.0 versus 71.6 percent, respectively) while less than half of women without a high school degree reported having taken leave. Hispanic and non-Hispanic Black women were less likely to report having taken maternity leave than non-Hispanic White women (62.5 and 64.3 percent, respectively, versus 72.2 percent). When taken, the average length of maternity leave was 10.0 weeks (data not shown). Among employed women who did not take maternity leave for their last pregnancy, 5.1 percent did not take it because it was not offered or allowed by their employer. Of non-Hispanic White women, 3.2 percent reported this reason, compared to 8.2 percent of Hispanic women and 10.2 percent of non-Hispanic Black women.

I graphed these using skin colour as a major visual tool to tell the difference between the treatment of women at work.

Below are the visualisations and my code:

 

The key: (will be added to the sketch later)

Women who took maternity leave based on race

Employer’s disagreement on maternity leave based on race

Women of (presented) colour who enrolled in college

Educational Attainment levels:

Less than High School

High School

Some College

Bachelors Degree and higher

CODE:

let dataRace;
let dataEducation;
function preload() {
dataRace = loadTable(“race.csv”,
“csv”,
“header”);
dataEducation = loadTable(“education.csv”,
“csv”,
“header”);
}
function setup() {
createCanvas(430, 400);
console.log(dataRace.getRowCount());
console.log(dataRace.getColumnCount());
white = loadImage(“maternity/white.png”);
black = loadImage(“maternity/black.png”);
hispanic = loadImage(“maternity/hispanic.png”);
mixed = loadImage(“maternity/mixed.png”);
e1 = loadImage(“maternity/e1.png”);
e2 = loadImage(“maternity/e2.png”);
e3 = loadImage(“maternity/e3.png”);
e4 = loadImage(“maternity/e4.png”);
whiteemp = loadImage(“maternity/whiteemp.png”);
blackemp = loadImage(“maternity/blackemp.png”);
hispanicemp = loadImage(“maternity/hispanicemp.png”);
multipleemp = loadImage(“maternity/mixeddemp.png”);
whitegraduate = loadImage(“maternity/whitegraduate.png”);
blackgraduate = loadImage(“maternity/blackgraduate.png”);
multiplegraduate = loadImage(“maternity/mixedgraduate.png”);
hispanicgraduate = loadImage(“maternity/hispanicgraduate.png”);
//slider1 = createSlider(0,100,0,1);
//slider1.position(10,height-70);
//slider1.size(width-20,20);
//x = slider1.value();
console.log(dataRace.getNum(3, “Took Maternity Leave”));
//console.log(dataEducation.getNum(3, “Percent of Women”));

}

function draw() {
background(255);

let whiteNum =dataRace.getNum(1, “Took Maternity Leave”);
let blackNum =dataRace.getNum(2, “Took Maternity Leave”);
let multipleNum =dataRace.getNum(3, “Took Maternity Leave”);
let hispanicNum =dataRace.getNum(4, “Took Maternity Leave”);
let whiteEMP =dataRace.getNum(1, “Did Not Take Maternity Leave – Not Offered or Allowed by Employer”);
let blackEMP =dataRace.getNum(2, “Did Not Take Maternity Leave – Not Offered or Allowed by Employer”);
let multipleEMP=dataRace.getNum(3, “Did Not Take Maternity Leave – Not Offered or Allowed by Employer”);
let hispanicEMP =dataRace.getNum(4, “Did Not Take Maternity Leave – Not Offered or Allowed by Employer”);
let whiteCol =dataRace.getNum(1, “Enrolled in College”);
let blackCol =dataRace.getNum(2, “Enrolled in College”);
let multipleCol =dataRace.getNum(3, “Enrolled in College”);
let hispanicCol =dataRace.getNum(4, “Enrolled in College”);
let lessHigh =dataRace.getNum(1, “Percent of Women”);
let highSchool =dataRace.getNum(2, “Percent of Women”);
let someCol =dataRace.getNum(3, “Percent of Women”);
let colHigher =dataRace.getNum(4, “Percent of Women”);

let y = 25;
let x = 5;
//if(slider1.value() < hispanicNum){
strokeWeight(30);
stroke(145, 86, 34);
line(x+30,y+80,x+30,y+80+hispanicNum);
image(hispanic,x,y,50,85);
//}
//if(slider1.value() < blackNum){
strokeWeight(30);
stroke(51, 30, 12);
line(x+80,y+80,x+80,y+80+blackNum);
image(black,x+50,y,50,85);
//}
//if(slider1.value() < whiteNum){
strokeWeight(30);
stroke(249, 222, 199);
line(x+130,y+80,x+130,y+80+whiteNum);
image(white,x+100,y,50,85);
//}
//if (slider1.value() < multipleNum){
strokeWeight(30);
stroke(183, 143, 110);
line(x+180,y+80,x+180,y+80+multipleNum);
image(mixed,x+150,y,50,85);
//}
//noStroke();
//textSize(30);
//text(slider1.value(),90,200);
//textSize(20);
//text(“% of women”,46,220);
//let edu1 =dataEducation.getNum(
strokeWeight(30);
stroke(100);
line(x+250,y+60,x+250,y+80+lessHigh);
image(e1,x+230,y,45,58);
strokeWeight(30);
stroke(75);
line(x+300,y+60,x+300,y+80+highSchool);
image(e2,x+280,y,45,58);
strokeWeight(30);
stroke(50);
line(x+350,y+60,x+350,y+80+someCol);
image(e3,x+330,y,45,58);
strokeWeight(30);
stroke(0);
line(x+400,y+60,x+400,y+80+colHigher);
image(e4,x+380,y,47,60);

strokeWeight(30);
stroke(145, 86, 34);
line(x+30,y+250,x+30,y+250+hispanicEMP*8);
image(hispanicemp,x,y+200,50,72);
strokeWeight(30);
stroke(33, 20, 9);
line(x+80,y+250,x+80,y+250+blackEMP*8);
image(blackemp,x+50,y+200,50,72);
strokeWeight(30);
stroke(249, 222, 199);
line(x+130,y+250,x+130,y+250+whiteEMP*8);
image(whiteemp,x+100,y+200,50,72);
strokeWeight(30);
stroke(183, 143, 110);
line(x+180,y+250,x+180,y+250+multipleEMP*8);
image(multipleemp,x+150,y+200,50,72);

strokeWeight(30);
stroke(145, 86, 34);
line(x+250,y+260,x+250,y+250+hispanicCol);
image(hispanicgraduate,x+230,y+200,45,58);
strokeWeight(30);
stroke(33, 20, 9);
line(x+300,y+260,x+300,y+260+blackCol);
image(blackgraduate,x+280,y+200,45,58);
strokeWeight(30);
stroke(249, 222, 199);
line(x+350,y+260,x+350,y+260+whiteCol);
image(whitegraduate,x+330,y+200,45,58);
strokeWeight(30);
stroke(183, 143, 110);
line(x+400,y+260,x+400,y+260+multipleCol);
image(multiplegraduate,x+380,y+200,45,58);

}

https://alpha.editor.p5js.org/amena91/sketches/BJpwh9TAb