Final project: fetal movement visualization

Here is a link to the serial sketch.

I finally got my sketch working in serial with my piezoresistive pressure matrix. My success was in no small part a result of the help of my friend Jeremy Baron. Jeremy and I worked together at Blue Moon Fish at the Grand Army Plaza Greenmarket selling fish on Saturdays for a couple years. He’s a talented coder, a good friend, and I couldn’t gotten serial communication to work with this sensor in any meaningful way without him.

This is an actual recording of a live baby kick visualized through my p5 program

The sensor sends a 40-point array of data in serial to the sketch. That part is easy. The hard part is making sense of that string in a way that activates the colorful ripples in the sketch in a meaningful way. The sensor has 5 columns and 8 rows. That was articulated by the mathematical concept of:

let sensorX = int(i / 5); let sensorY = int(i % 5);

The other tricky part was getting creating a variable that keeps track of time, so the program knows how often to pay attention to whether or not anything has happened with the sensor. It all sounds simple, but actually implementing it was complicated — especially under the duress of impending finals and the Winter Show.

The sketch still has some problems. When it came time to play test it on my wife’s belly, the ripples would spontaneously disappear after 5-45 seconds, and then sometimes reappear equally spontaneously. I talked to Dan Shiffman about it, and he suggests that this may just be a limitation of the p5 web editor, and that I may have more success off-lining and running it in Processing or some other offline editor.

The other problem is that the audio still gets overwhelmed when too many oscillators are made. I tried porting my audio functionality over to the Class I created for the visualization. My theory is that if the oscillators lived as objects in tandem with my ripples, and that if I kept that array to a maximum of ten, then it would actually delete oscillators once they exceeded ten iterations and thus negate the glitches. This would probably solve the problem, but I didn’t quite succeed. This version has other problems. Please feel free to take a look and tell me where I went wrong.

On a personal note, I enjoyed my introduction to computational media, and my introduction to JavaScript a great deal. I’m not entirely sure to what end I will use this knowledge, but I enjoy doing it, so I intend to continue in this vein. Next semester I’m taking Dan Shiffman’s Nature of Code, so stay tuned for my adventures in JavaScript.

Week 11! Sound!

Click on the grey box to make music.

Click here to view the sketch.

I’m writing this post retroactively, because finals got the better of me and I’m just now getting to it post facto. So, I hope my account of my process and struggles is still accurate. For my physical computing final, I was making a sensor that visualizes baby kicks (my wife, Marina, is pregnant). I wanted the visualization to also include some nice audio, so it was fortuitous that we were doing sound in .p5 at the exact time that I needed to learn it.

The grey box sketch you see above is just the event-based music that would eventually become the audio track to my baby kick sensor. Some of the struggles I had with it: initially made the very basic and dumb mistake of calling oscillators in the draw loop. That was easy enough to fix once I realized it.

The other problem I ran into was a browser / computer performance issue. Even though each sound seems like it’s stopping, it’s just decreasing in volume to zero. The oscillator actually remains. If you make too many of them, the result is weird, glitchy and awful. Give it a try!

Week 9! Pixel Manipulation.

Spacebar for fullscreen.
Press 1 or 2 to toggle redshift or blueshift. Press 0 to reset.

Some years ago my wife was involved with a massive space survey during an internship at the American Museum of Natural History. They were working in tandem with the Sloan Digital Sky Survey, a comprehensive effort to map the universe using ultra-sensitive light sensors. They used a vast array of giant metal plates with strategically placed holes in them to place over the imaging lenses to filter out the noise of lesser celestial bodies. We actually got our hands on one of these plates, and it sits above our couch in our living room, so I think about it often.

The P5 sketch (link here) manipulates pixel data from one of the images created from this epic sky survey. I say created, because they are essentially renderings extrapolated from this wavelength data. I mean, an image is just an interpretation of wavelengths of light, so it’s not different in that regard. The difference is that this image was intellectually built by crunching wavelength data. They were trying to create a 3-dimensional model of the universe using red-shift and blue-shift — a concept accredited to Mr. Hubble (of the famous telescope, and the biggest metric to English measurement conversion snafu in history that oddly enough led to the invention of the anti-aliasing filter). You can find the process at this link.

The idea is similar to the Doppler effect, where an object traveling into its own sound energy in your direction sounds higher in pitch, while an object moving away sounds lower. The same applies with light. Celestial bodies moving away from us appear red, and subsequently those moving towards us appear blue. This is the foundation of our belief in the expanding universe.

What I’ve done with this sketch is re-image the red and blue shift data back out of their composite rendering, thus isolating the core foundation that this artificial image is built from. What I like about this sketch is that it’s actually useful for interpreting celestial images, and we can very easily see which celestial bodies are coming and which are going.

At its core, this sketch is changing the fill color to random pixel values from the source image and drawing them as tiny circles. I also added a few lines that make sure it doesn’t repeat the same pixel twice, so it can actually accomplish drawing the image rather than the theoretical idea of completing something with random numbers that might take until the end of time from a certain point of view.

P.S. I wanted to give a shout out to Edgar Allan Poe for being the first person to write about the concept of the expanding universe. In “Eureka: a Prose Poem,” he basically conjectures that if the universe wasn’t expanding, we’d be burnt to a crisp by now with all of that violent solar energy being blasted at us from every direction of the universe.

Week 6! Objects, Arrays and More Planets!

Spacebar for fullscreen. Click to create new stars!

If you haven’t yet, please take a moment to play with the above sketch. Here is a link in the p5 editor.

This week we learned how to use objects and arrays, which is really liberating for a lot of reasons. It feels like these are the missing pieces to break out of the confines of .p5. Last week I made a space scene (click here for link). In that sketch, I created a backdrop of stars using the noise() call, which allowed me to make random points in the draw loop. I created essentially the same thing here, except this time I did it using objects and arrays. By defining the array in setup with random coordinates and varying widths, I was able to call the array in draw to make the star scene you see here. Same concept, but a lot neater in code, and a lot less “hacky.”

However, one advantage of using noise is that it does live in the Draw loop. Because of this, it was easier to scale when in full screen. In this version, when I tried to do the same thing, this is what happens:

As far as I can tell, because the stars are based on random coordinates within width and height, and because that is called in the setup, the stars won’t scale when you grow the canvas. The way I fixed this was by changing the random arguments to (0, 3840) and (0, 2160) respectively (4K resolution). So, if you’re not in full screen, there’s just a bunch of canvas that you’re not seeing. The previous version didn’t have to use this workaround because everything was scaled. If I had more time, I would work that into this sketch.

In this version I used the mousePressed function to call the planet objects from an array. When you click on the canvas, a new planet appears at a random size and random color (constrained to parameters I defined). I also created a sort of coin flip based on the mousePressed that determines whether or not the planet has a ring on it. At some point I would like to be able to click and drag planets to custom sizes. Maybe for the midterm….

Another thing I couldn’t quite figure out was how to delete an object with mousePressed in spite of the fact that mousePressed is how you create an object. I have it set up right now to splice from the array to remove an object with a mouse click, but it simultaneously creates a new one in the same space. I hope to find a solution that soon as well.

Oh! I also made a version of the planet sketch using a potentiometer to zoom in and out. check it out (filmed by Nick Grant):

Week 5! Functions and Sparkling Stars!

Click and drag to zoom, double-click to change colors.

Press Spacebar for fullscreen.

Press "C" to reset colors to default.

This week’s assignment was all about wrapping up our code into functions. Making your own functions is great for two reasons. 1: It’s a nice way to tidy up your code, making it much easier to debug and isolate variables and such. 2: By giving your function parameters you can easily repeat it with variation. Here is a link to the sketch.

Oh, and before I continue, I should add that we were urged to execute this on one of our old sketches. So I chose my very first sketch to revisit. Here is a link to that for comparison.

In the above example I used eight functions, two of which I created from scratch. The ones I created are called “drawStars” and “drawPlanets.” Guess what those do.

For “drawStars,” I gave my function just one boolean true/false paramater. Basically it says if “sparkleState” is true (cute, right?), do this:

The beauty of functions if that all it says in the Draw workspace is: “drawStars(true);”, and this one line accounts for 30 lines of code. Similarly, my “drawStars” function accounts for about 60 lines of code. “drawStars,” however has significantly more parameters that I defined. I gave it 12 parameters, so you can give it up to 12 arguments.

Those parameters are:

• X coordinates

• Y coordinates

• planet width

• planet height

• planet ring width ( a percentage value based on planet size)

• planet ring height (also a similar percentage value)

• planet ring color

• red value of planet

• green value of planet

• blue value of planet

• plant ring angle

• zoom value (a percentage value in relation to overall scaling)

By utilizing these parameters, I can repeat the planets with a great deal of control over the variation. For example, the top-left planet has a ring more in-line with the horizon, as well as a darker color, while the planet in the bottom-right has no ring. Additionally I was able to use the scale value variability I created with the parameters to be able to scale the planets at different rates when you zoom. This creates the illusion that the two smaller planets are further away. If they scale at the same rate as the one “in front,” the illusion is not as effective.

Here’s what that looks like:

The “if” statements preceding each “drawPlanet” function call are what make the planets disappear if you zoom in far enough. This creates the illusion that you’re sort of zooming past them, or traveling through them.

I also added some mouse and key functionality as you probably noticed. You can change the color back to the default red by pressing “C,” and you can also zoom in with the mouse and randomize color with a double-click.

My biggest success, though, was finally making a sketch that can be viewed in full screen mode. I accomplished this by using the “fullscreen()” function. But, that on its own is not enough. I also had to make a boolean expression that would toggle back and forth between my 600x400 pixel canvas size and a full-window canvas size, and it works great! BUT… if you press the escape button while in full screen mode it messes up my system for toggling the canvas size back and forth. Still trying to work that one out.

I also wanted to talk about how I made the star background. There were a lot of other little things I did to finesse this particular sketch, but I for sure want to mention that. With some guidance from our talented professor, Allison Parrish, I was able to wrap my brain around the “noise” call, at least a little bit anyways. I wanted to randomize the stars in the background, but as anyone who has played with p5js knows, putting a “random” call inside of the Draw loop gets really annoying really fast. Like epileptic seizure annoying. Enter, the “noise” call.

I don’t totally understand its origin, but “noise” generates a random value between 0 and 1. What sets it apart from “random” is that the value it generates will be the same for every cycle of the draw loop, so in this way you can create static objects in random-looking places. The behavior is weird in practice, though, because it seems to center the center of the canvas. If you map noise to values between 0 and the canvas bounds, you get this:

Where are all the stars at the edge? As you can see, when you map noise this way it disproportionately favors the center. So, the way I fixed this was by mapping the values 25% outside of the canvas bounds. Then you get this:

Not bad right? A little more even and fair to the edges? I call it a success. You’re welcome to disagree.

As for the sparkling stars (the animated ones), I used the “random” call to my advantage. If you generate random stars in random places on its own, it does end up being really annoying. The workaround I came up with was to make the background in the Draw loop have a subdued alpha channel, meaning I lowered its opacity. This creates the appearance of the stars fading out, when in reality the background is fading in by layering a translucent background repeatedly on top of the stars with every cycle of the Draw function. This is also a good tip for making anything animated using “random” to be less annoying. Oh! This is also what gives the psychedelic tracer look to the planets as you zoom in and out.

I hope you enjoyed this one. Until next week….

Hugs,
Zach

Week 4! Chasing the 10 Print Dragon.

I was trying to make something based off the famous 10 Print concept (example link here), which is extremely elegant in its simplicity, but I wanted to introduce some color. The way I figured I could introduce color to 10 Print was by making them as varied rectangles instead of lines. However, I ran into some struggles with using translate and rotate in a loop. Like a lot of struggles. Here is a link to my sketch.

This is also not the first time I’ve struggled with translate and rotate in .p5js. It’s extremely counter-intuitive to me. Quick recap: in order to rotate an object in P5, you have change the entire way the program interprets coordinates. It gets really confusing really fast. Anyway, here’s my attempt (and here is a link):

As you can see, it’s not as organized or graceful as my first example, or as 10Print. I hope to be able to improve this design in the near future. Always looking forward to more progress.

One of the requirements of our assignment this week was to use a for loop. While I did not use the standard for loop format of “for (i=0; i < 20… etc.),” this program does contain two loops. I made a version of my program using a proper for loop for posterity’s sake, which you can find at this link, but it wasn’t necessary.

UPDATE: I had some success (kind of) after class a lengthy class discussion:

In other news, I photographed a stink bug’s face this week that I really like, so I’m going to share it here, just because I can.

Week 3! Interactivity!

First and foremost, I finally figured out how to embed .p5js sketches into my Squarespace site. So, special thanks to Timothy B. at Squarespace customer service, and also to Prof. Luca Damasco at Carnegie Mellon University School of Art for sharing some of their insight online as well. Here is a link on how to do it for anyone else who needs it.

Hopefully you’ve clicked the button by now and reveled in its functionality. Nick Grant and I collaborated on this one. We wanted to make a button that would generate a number of different scenarios, but randomly. So we made the button generate a random value and used the “int” conversion to make those random values into integers. By doing this we were able to map different scenarios to an integer value, meaning if random came back as three, execute scenario three.

Here is a link to the sketch in the p5 web editor.

Another thing I did that I thought was cool: I took the random RGB values that were generating the colored backgrounds and multiplied them by two for the text colors. By doing this, the text will always (even thought the colors are random) be brighter than the background, giving the text more contrast. The only except is if the backgrounds values are already at 255, which is why I put a small stroke on the text as well.

It was interesting collaborating on a program, because coding is sort of inherently solitary. We spent some time coding together, trying to figure out this random scenario functionality. We almost had it when we ran into a block. And then Skylar (whose last name I don’t know at the moment) pointed out that we weren’t using a boolean expression in a boolean situation (we were using = instead of ==…. duh… obvs….). Then everything fell into place. After that, all we had to do was make our own cat scenarios and plug them into the infrastructure we had built together. And now the world has this cat button. You’re welcome.

Oh, the cat imagery. We pulled all these images from free stock websites. We thought about using pictures of our own cats, but that would’ve been a lot of extra work silhouetting them out in Photoshop. I got mine from hiclipart.com. I’m not sure where Nick got his, but they were also fair use images, as well as the meow .mp3 file.

Hugs,
Zach

P.S. Here is a picture of my cat.

Week 2! Variables! Excitement!

This week the assignment was to create a sketch that includes:

• One element controlled by the mouse.

• One element that changes over time, independently of the mouse.

• One element that is different every time you run the sketch.

In my sketch you can control the background color by moving the mouse around, while at the same time controlling the color of the circle. In order to satisfy the other two points, the squares in the center rotate independently of the mouse, and they also become a different color every time you run the sketch.

Click here to view the sketch in the p5.js web editor.

One of the other things we were to aim for was to avoid using hard numbers. My previous sketch satisfied this, because a fixed inadequate resolution made me uncomfortable as a photographer. So, instead of using pixel coordinates to define shapes and locations I’ve been using width and height to indicate this. — meaning, instead of telling the program to create a square at 300 pixels in and 200 pixels down, I would tell it to create a square at width * 0.5 and height * 0.5 (halfway in and halfway down).

I used kind of a shortcut to keep the changing colors of the background and the circle separate. Rather than make a unique set of variables to make a sort of “Random V2” (which is the only way I currently know how to make two separate objects random colors without them being the same color at the same time), I modified the variables with some little math tweaks. Since the circle was set to random variables for red, green and blue values, I set the background randomness to basically “red random value divided by 0.5” and “blue random value divided by 2.” Also I should point out that the colors aren’t actually random, they are dictated by the X and Y position of the mouse cursor. I just said random for ease of conversation.

The code looks like this:

r = map(mouseX, 0, windowWidth, 230,0);

g = map(mouseY, 0, windowHeight, 30, 255);

b = map(mouseX, 0, windowWidth, 255, 70);

background(r/0.5, g, b/2);

fill(r,g,b);

noStroke();

ellipse(mouseX, mouseY, width * 0.05, height * 0.075);

I was also trying to figure out how to get the circle cursor to leave the frame. It works naturally on the right and bottom, but not the other sides. I was trying to use some boolean code to reduce the cursor’s opacity when it approached the edge of the frame, but I couldn’t get it to work. Looks like we’re going to diving deeper into the if, else boolean action pretty soon here, so I look forward to that. In other news, I finally feel like I’m starting to screw up where the curly brackets go much less often. So that’s a win. On to the next coding adventure!