This week, me and Alan began the project by scanning 3D models of ourselves. And, we took a video on both of us of sneezing. Later, we used slow motion to capture each frame and convert each motion into a panoramas.

Right now, we successfully import the panoramas with my 3D scanned object into the environment.

Next, we tried to connect the environment with Kinect and Kinectron. The code seems to grab the data from Kinect successfully; however, there’s nothing happen….


Assignment 6_drawing instruction

So, I thought of doing a live computer generated drawing instructions using spaCy’s built-in word vectors and annoy vectors semantic similarity for this week’s assignment. The drawing will be produced base on the current environment. I’m thinking to develop this project into my final. To improve, I want to make the script to be more complicated and be more interesting so that the drawing will be more arbitrary and the instruction to be more specific.


Week 3_ideas of the storylines

For this week, me and Alan had conversations about the ideas of how to intersect multiple stories in different spaces. We were inspired by the book, Einstein’s Dream, about the spaces that exist in different time. How can we bridge multiple time zones together and tell a story? This is the discussion that we have been discussing over the week. We decide to start brainstorming for final project and break down our working process starting from this week.

And, we came up with some ideas:

  1. Two Spaces Intersecting in Different Time
    We were thinking to shoot a 360 video at a public place, such as a coffee shot. The video captures the space at a fixed angle for couple hours. And, bring in a 3D model while working in three.js and set a focal point on a chair and table. The idea is just create a spot light for this location and bring audience attention of what to see and also show the time at the coffee shop. We were thinking even layering another video footage of a person interacting with the video that we shot at the coffee shop. So, a person sitting on a chair (laying with a 3D model chair) and the person from another video is placed on another chair that at the same table working independently.
  2. Two Spaces Laying on Top of Each Other in Different Time
    To extend the idea from the previous one; instead, we change the composition of the videos. For example, a second video is placed outside the first video. The camera is placing at the center of the two videos so when the inner video have lower transparency, the audience can see the outer video (second one).
  3. Alan Sneezing
    So, we thought if we 3D scan Alan’s head and duplicates to min 4 heads and place them into a circle all facing toward the center. Each head’s facial expression captures Alan’s sneezing moment starting from the beginning to after sneezing. And, when the camera turns to the last head (when the sneeze happens), something pop up and throw to the camera.


  1. Bring memory back to life
    With the increasing number of elderly which suffer from Alzheimer’s, technologies that could improve their quality of life and probably alleviate their symptom would be desirable. If we can implant the mindset of the patient’s close family or friend into a 3D printed human body of that person, and let him/her keep communicate with the patient, this can bring some of the memory back and probably immerse the patient in the joyful moments in the past.
  2. Swipe Left or Swipe Right
    Dating apps are becoming more and more popular with the increasingly pace of life. The so called “scientific proven” matching algorithms behind those apps are always under question. We can somehow develop a personality emulator that creates our virtual embodiment and evolve in a computer program to simulate all the intricacies happened in inter-personal relationships and record details involved in dating couples and rate them based on certain metrics. This would be a more “scientific proven” matching method.

  3. Spiritual Embodiment
    Is our spirit defined by our physical embodiment or does the latter shape/appearance become the former? If humans would have the body of a dog, would our common mindset be totally different from the one we currently possess? If a dog owns the spirit and consciousness of a human being, would their dog-like body evolve to cater the superiority of a human mind, probably capable of intellectual endeavors such as dancing, coding, deception, or empathy?

Project idea of using Syphon and Isadora or other softwares:
Medium: Syphon + Isadora + processing + perception neurons live motion capture
Project name: VJ Manipulation and/or Variation

The idea originates from Karlovy Lázně, the biggest music club in central Europe, which contains five different floors, with each floor playing a different style of music and visual. However, as one music club, one missing piece is the union of all the artistic components. The idea is to originate visuals of the five different floors based on one common raw video stream, created by either a common base music or a main DJ’s motion. Through Syphon the base visual could be shared and streamed to different Isadora programs to manipulate the base visual based on each floor’s different theme/style. In this case, the different themes or styles of each floor is maintained while the unifying theme as one single biggest music club in central Europe is also introduced, since all the manipulated VJs through Isadora are implemented through Syphon sharing from a common based visual.


MOMA visit

This MOMA trip was actually more inspiring than I thought. Although I have been to MOMA for so many times and seeing those painting so many times, I never had a chance to have someone stand next to me and explain its process.

We visited several paintings but there were few ones that caught my attention.

  1. Reflections of Clouds on the Water-Lily Pond by Monet:
    The presentation that MOMA presents is interesting. When I first walked into the space, my eye were invited to this three canvases and this immersive experience makes me feel like I’m inside the painting. Although this painting is broken down into three canvases; however, this concave setting is bring all as one that creates an interesting on point perspective.



2. Louise Rosenblatt Literature as Exploration: 
According to Katie’s explanation, this painting was painting during the war and is categorized under Surrealism. The perspective of this painting is quite interesting because there is sort of a layer or depth inside of this painting. There’s a painting inside a painting which is depicted that the present world is more complicated than we actually imagine. Also, seeing the wireframe of the building can be a metaphor of seeing the truth behind.


3. Shovel by Marcel Duchamp:
First when I see this as an art piece, I was like “why?”. But after hearing the story behind, I understand the reason and was actually quite impressed and thankful by him. So, being the pioneer of modernism, he submitted this shovel to an art competition and called this as art when everyone else sent in their beautiful paintings.















4. 3 standard stoppages by Marcel Duchamp:
System: Duchamp made fun of the system that our society comes up with. The reason why we have system today is because our brain can only process so much of information and that’s why we have system. The process he came up with this was also pretty fun. So, Duchamp dropped the meter randomly on the floor and found out the patterns. Then, he made his own system to challenge the systems that existed in our society.

Honestly, after viewing several modern artworks, I really think the most valuable part is not their artwork, but their concept and message behind. So, I think the best way to present these modern artwork is to include their production process with solid description of their ideas while creating the art. I think this is better than just showing the art itself.

5. Painting (don’t quite remember the full title and artist’s name)
This painting is not as well known or recognized compared to those previous ones. However, the presentation of this painting is interesting. The layers of paints somehow creates a 3-Dimensional space and turns the painting becomes more alive. Also, since the paints are layering up, it creates the digital form.





Einstein Dreams_interlude 1 through “29 May 1905”

I read the part from interlude 1 through 29 May 1905 with the topic of discovering time. In the beginning, I found no relationship between each chapter; however, I realized the author continues to discover and discuss from different perspectives, which I found really interesting and inspiring.

The chapter 8 May 1905 is talking about the world will end soon at a specific day in the future and everyone remains the same time to be alive, which the world becomes peaceful without crime or negative feelings. This immediately reminds me the idea of Socialism. Things are distributed equally to everyone in order to make everyone equal. However, this is still not a perfect solutions to make everyone happy with what they have. Also, people can suffer together but hard to share with others because greed has no limits. Today, people are trying hard to extend life and trying to live as long as possible even they are lying on bed in the hospital. But, there’s no happiness anymore; instead, with a limited of time, people are actually happier and enjoy the most of their life.

This also refers to another chapter, 14 May 1905, which talks about without time there’s no life. Some disagree but I do agree. With a fixed and frozen of life, is like extracting the best part of your life and store as memory to others and die with happiness.

The next chapters, 15 May 1905 and 20 May 1905, talk about memory. After reading these two chapters, it brings up a thought that if in the future should people have the ability to delete their memories or not. With memories, our lives become richer; without, we become like a robot. A lot of emotion will be lost; however, people might be more efficient, easier to be friends with, and even become better human being.

Assignment 4_Tracery+Markov Chain

This week, I decided to use the Sonnet file that I created with Tracery couple weeks ago. I wanted to test it if Markov Chain’s model can duplicate similar text generation behavior.

So, using Tracery library, I used artificially designed template to generate Sonnet. The rules of text generation were explicitly articulated in the form of a template. Thanks to the library, I can replace some text components in the sentence.

On the other hand, I want to explore the possibilities of using Markov Chain to achieve the same behavior. Based on a relatively large Sonnet text data set, which was generated by Tracery library, a Markov model was built. It captures the interrelationship between individual and multiple characters and words, using the concept of n-grams. I used a similar function definition from the class notes and the Markov model is essentially a dictionary or look up table. Different keys in the dictionary correspond to multiple text predictions. The text generation was based on the same idea as in the class notes.

After the experiment, my Markov model is able to automatically learn the text generation rules previous specified by an artificial template. At the sentence level, the Markov model achieves the same effect as Tracery’s template. The grammar and syntax remain intact. However, Markov model picks random starting point and the whole format of Sonnet cannot be guaranteed.

Week1_Future Storytelling

I was having conversations with one of my friend the other day about the format of videos that we are seeing and people are exploring nowadays. The media had developed from 2D, 3D and even now 4D, computer graphics, and virtual experiences. So, the format of storytelling has transformed to a non-traditional way.

We found a group of artists called AES+F, they are a group of artists create non-linear story but at the same time is super hyperrealistic and strange. It somehow contains the aesthetic of those ancient paintings. The video like this is another way of storytelling which does not have a clear storyline but somehow contains message behind. On the one hand, this video is somehow challenging people’s aesthetic, and their societal perceptions, as well as their understanding of the purpose behind the videos.

Assignment 4

When I think of a particular form of the poem, I directly thought of Shakespeare Sonnets. I chose the one that I studied before, Sonnet 20, as my template. However, I realized the format of Sonnet does not have a rule that I can easily replace words. So, I borrowed a general format that Sonnet has and implement my own form.

The Sonnet I chose has total 11 sentences and every three sentences is categorized as one section. The last two sentences are its own section and indented.

So, I rewrote the poem by adding my own rule. Every section’s odd and even number sentences are in the same sentence structures. As you can see from the image in below:








So, I picked most of my vocabularies from Corpora and some I just manually typed it myself. Later, I installed base_english so I can implement capitalization and third person verb form. But, I realized that base_english does not include the different verb tense such as V+ing so I can only add “ing” manually; therefore, the grammar is a little off.

After looking back to my poem, I found that is really different than what I initially thought..The meaning has totally changed and it does not make much sense. Also, within each section, those sentences seem not associate or coherent to each other…which I realize how challenging it is to redesign a sonnet and implement my own style in it. In order to make it perfect, I believe there must be multiple trials and to reconsider the word choices.

midterm progress

We worked on three different ideas:

First: the stairwell with a cover…

By looking up the stairwell, we used a piece of fabric to cover the opening above. This is to test if we can create an illusion of a floating ceiling which is similar to the mirror reflections. The only challenge would be the cost of the material.



Second: create a fake door…

To just try out, we use a projector to project an image of a door on a piece of white paper in the hallway. The effect seems pretty nice. We also tried out to project an image of the same background as the wall and resize the image a bit unaligned with the wall in the back and see how that works…the answer is…it’s pretty obvious to tell due to the different surfaces. The challenge for this idea is the lighting in the hallway, the material to use (tape, images, or applied wood?), and the freedom of using this space.



Third: a peephole on a door

We tested it with a door peephole and see through an image on an iphone. It seems working and we are more leaning towards this idea. We will have to find a larger device so we would be able to see through wither larger view. The only challenge now will be finding a spot to install and hide the device and makes the peephole seeing nothing will be hard.