Final Week: Unity - 'The World' Music Pattern

AR Project with Unity

I am very inspired by the examples of Foo/Skou that Gabe showed us in class with sound and viusal. I see opportunities of this physicallity and digital world by this "visual to music" concept. This also has always been what I am interested in (I made a simple project for a class 'Code of Music', which sort of demonstrates what I want to focus on > Drawing Beats). And from this I see a lot of opportunities for music industry or even music educations. First I was playing with the concept of visualizing lyrics. I use the lyric booklet of one of my favorite Taiwanese Band's album as a tracking target, and create some different scenes to specific songs:







At first, I wanted to track the lyrics instead of the photography, however... there were no stars on the lyric part... (and I tried to use the zero star ones to track... of course I failed) I wanted to add runing texts of the lyrics (like Kareokes) but importing CHinese characters seem to be way more difficult than I imagined, and I accidentally droped a font file into the assets folder that made me restart my entire project. I call it 'the most regretful thing Michelle had ever done in 2019':







Also, my shader and model building skills aren't good enough to support me building this 'beautifully'. I tried to follow a tutorial on making water ripple effect to mimic ocean, which been mentioned in the lyrics, to create a visual scenary of the song. The second scenary is inspired by the vocalist's talk during concert while singing this song--"take our your phone, call someone that you want the person to listen this song with you", at the time there was no smart phones, which the screen of phone was bright and tiny, creates a illustion of being surrounded by stars. I tried playing with particles in this scenary, but I figured that it is hard to play with physics with image tracking targets. The code I was learning from a tutorial didn't work in this environment.

When I was doing this, I thoug about this being a new form of music videos (I don't know if this awesome idea has already been done or not) But from this I see how AR is different from VR or other technologies. It supports the physical object while being support by it. Nowadays, due to the conviniency of just downloading a song through spotify/apple music, physical cd album is slowly dying (though, also due to eco-system issues). AR can bring the experience of owning a physical piece more magical. Unlike any other possible technologies, AR make the user feels like the digital elements stays with the physical object, which is physically own by the user, that feels very personal and one-and-only.

Then I thought about, what if when you are playing some sort of puzzle, and it will come up with different pattern of beats or melodies?

So first designed some simple patterns:







Idealy, the application can track each icon and generate music when it sees a pattern. However, zero stars again...







So, I made some trick to pretend that it is doing the smart job by using the stupid method... I create all the possible pattern (Not all apply yet, I'm still building this 'for loop').







and four small animation to show which is now active.







The video doesn't have sound, but I assign each possible pattern to a certain related ASMR sound. Now this doesn't work as a running music piece, but ideally, when the user changes the order of the icons, the order of the elements changes.

Week 3-4: Collage Animation "Dreamwalk" & Mirrorworld

AE Collage Animation

My group member and I want to play with mainly collage and greenscreen. Having a collage landscape and a character moving inside a fictional environment. After disscussing the entire mood of the cilp we wnat to convey, since we have a lot of stuff going on in the collage, we came up with a detailed storyboard with the assets we want to use and transitions between scenes beforehand so that we can save time.







We split up the work into three parts and set up a moodboard so that we work with the same style. And I am incharge of the first part. Until the person starts to walk in the long landscape.

For this specific project, we are relying on the music very much, so as I animates the assets, I first listen to the whole track and mark the beats, climax and attributes that happened in time.







The function that I like the most in AE is parenting. Most of the assets in my work perform well only if it is parent to another main body. However, for the head opening part, I figured that parent is only good for still images but not working as well for the footage. So I tried using the tracking function, but it never worked for my computer...







Fortunatly, this zoomin & opening up part isn't long so I can just manually fix the positioning of the landscape in the brain and the opened cap.

I really enjoyed making the landscape in the brain by creating a seperate composition and have muiltiple characters moving in the same frame. But the opening music isn't long enough so I can really have a long scene on this part.







And personally, my favorite scene that I've created is this scene of the earth in the univers with a red door sitting on it.







I think what can be improved next time is the way we work as a group. Eventhough we've talked through the storyboard many times, and looking at the mood of the collage that we want to carry together, some of us have the assigned part done the last minute and we didn't have time to go through the final piece together as a team and give each other feedback and help. Everyone had different experiences in AE, so we need to know who needs more time to figure out the time schedule and way to teach each other.

---

AR Will Spark the Next Big Tech Platform — Call It Mirrorworld

In the article by Kevin Kelly, talked about the two parallel world -- the real world and the digital world -- intersects, which driven by the augmented reality, as called the Mirrorworld. This article is really interesting and also he introduced a lot of existing projects. And really make me think about as a designer or artist or even just a consumer, what is the purpose of using AR and how far it can go.

The concept of "two parallel world in AR" make me think about, why would people want something digital to appear on a physical real world (in a digital device)? I think it has a great opportunity to become a prototyping tool for designers to cut down wastes and budget. For example, for architect, they can adjust the design in the program, to put it at the actual space they will be building it. Also for fashion designers, before production, dripping, pattern making, fitting can all be down in AR before the real production. Even for designs that required digital printing, can be applyed directly on to it. New job opportunities will be available (to create libraries): AR civil engineerer, AR pattern maker, AR fabric/textile designer and etc.

If possible, the customer end should also work like this: fitting using AR and adjust sizes in the program --> data sent to the factory --> ordermade tailoring, no premade and stock so no waste. Also as a customer, it will be a great opportunity for furniture companies to have a interior design AR app online, after the customer designed with the AR furnitures, they can order directly from the app.

Week 1-2: Stop Motion Animation "Love Cut" & Reading Response

Stop Motion Animation

For this week's project, we went through a lot of ideas and stories, and had a 3 mintue long story, which is already a simplified version of the actual story and we think that it will lose its meaning if we simplify it even more, so we instead, inspired by the materials we got: some colorful wools--maybe some hair story?

As we started making the characters, we build up the plot for them. I'm very into photography and always wanted to try doing some cinematography for a short story. Inspired by Toy Story 4 I feel like not only how the entire enviornment for the character feels real or not, but using similar cinematography that's used for actual movie/videos will also help building the characters more. Through this project, I feel strongly that this is sort of where my passion is and also my focus point when watching a anime film (other than the plot).

Also, we do not want to give to much identity to the characters, so we kept them without speaking any lines by only making some emotional sounds, just like Minions.



Even though I feel anxious when I was doing the project, but mostly enjoying it alot. The most struggling part for this project will be using Dragonframe is still kind of unfamiliar for us. We only realized (or remembered) that the 'zoom in' only works as a preview but not a crop until we export the video...

---

Understanding Comics: The Invisible Art

This is a very interesting but at the same time a difficult comic book to me. The words are simple but the it makes me read twice in order to understand concepts. Scott Mccloud starting from definding what exactly is comic as an art form, a medium of expression an idea, the 'closure' (design kid happy to see the old friend mentioned here) happened in the gutters, the time and space and etc... We might see similar concepts in other form of arts but all of them can only happen in comics. I'm a comic person, specially I read a lot of mangas. Though, like he mentioned early in the book, there are still a lot of people think it is a "cheap, disposable, kiddie fare". But like books and movies, there are already a lot of people inspired by comics, or experienced some sort of life that they never get to experienced. There are even educational Mangas with proffessional terminology that introduced some occupation, which is not as popular, and increased the interest of public towards that occupation. This I think also proved Mcclound's point on the readers tend to put themselves in the characters.

Below are some parts that I feel really interesting and could be used for other fields