Advanced Interactive Design

October 06, 2020

Advanced Interactive Design - Project 2

7.10.20 - 28.10.20 (Week 7 - Week 10)
Abigail Bea Chong Ka Yee (0333966)
Advanced Interactive Design
Project 2 - Augmented Reality Application

INSTRUCTIONS

PROJECT 2 - AUGMENTED REALITY APPLICATION

In this project, we have to create an augmented reality application using Spark AR Studio. We could make any application of our choice but we must include the three features below:

using ui button to change between options (for the 3 filters)
background replacement
using face interaction patch + face tracker

I then proposed my idea to Mr Lun before proceeding to the next stage.

After Mr Lun gave a thumbs up I started by creating the assets.

Making the assets in Procreate and Illustrator | Week 7 - Week 8 | 7.10.20 - 14.10.20

Firstly, I drew out my ideas for each of the mid-autumn stories in Procreate.

Fig 7.1 Sketching Ideas

Then I went to search for a portrait picture I could find to use as a face reference to place my assets. I removed the background so it was easier for me to get the whole look and aura I was going for.

Fig 8.1 Portrait guide for placement

For the first filter, I followed the story 'Chang E Flying to the moon'. I wanted to have symbolisation in the filter so I drew the 9 suns, clouds, moon, ribbon, and a rabbit.

Fig 8.2.1 Filter 1 (Progress #1)

Fig 8.2.2 Filter 1 (Progress #2)

I added more texture to the background and redrew the ribbon and added shadows. When a user raises their eyebrows the rabbit appears and disappears whilst clouds moving as an animation.

Fig 8.2.3 Filter 1 (Final Design)

For the second filter, I was inspired by 'Rabbit on the Moon'. I tried drawing different backgrounds to fit the elements tracked on the face but it wasn't working as well. The elements on the face would be the rabbit on top of the head and when a user blinks the type of lanterns would change (in figure 8.3.2 & 8.3.3).

Fig 8.3.1 Filter 2 (Progress #1)

I knew I wanted to have a border but was stuck using Procreate so I used Illustrator instead for the background in figure 8.3.4.

Fig 8.3.2 Filter 2 (Progress #2)

Fig 8.3.3 Filter 2 (Progress #3)

Fig 8.3.4 Filter 2 (Progress #3)

Later on, when I tried putting the elements together it was too much because of the flowers on the top right so I removed it. A small animation would be the rabbit blinking in the filter.

Fig 8.3.4 Filter 2 (Final Design)

Lastly, the third filter was about 'Wu Gang who chopped the tree'. I drew the components that had to be tracked on the face in Procreate and made the foreground and background in Illustator (in figure 8.5.3).

In figure 8.5.1 sihoullete of Wu Gang would act as a headband and the flower would be for the eyes.

Fig 8.4.1 Sihoullete of Wu Gang chopping the tree and a flower

The face interaction would be when a user opens their mouth a mooncake (in figure 8.5.2) would appear.

Fig 8.4.2 Mooncake

Fig 8.4.3 Filter 3 (Background)

Fig 8.4.4 Filter 3 (Final Design)

Moving the assets into Spark AR Studio | Week 9 - Week 10 | 21.10.20 - 28.10.20

When the assets were completed, I exported them into Spark AR Studio. It was tricky to figure out how the background replacement worked so I made canvases to place my background and foreground first.

Fig 9.1 Importing the foreground and background

Then I tried placing the face-paint using a face-mesh layer which were the 9 suns but it didn't go well together. I later searched up whether background replacement and face-mesh were okay to use together in their official website but turns out it wasn't advisable to do so. After reading the article, I created a second version of the file and went ahead with placing the face tracker assets such as the bunny and 9 suns.

Fig 9.2 Placing the face assests

Once that was done, I encourted another problem with the eyeballs. I managed to fix it by just putting the eyes I made in front of the eyes instead of behind our eyes where it should have been. This issue was caused by the background replacement.

After all the issues were solved, I placed the filters into null objects as they acted as folders in Spark AR Studio. The null objects helped make my life easier when using the ui buttons to change between all three filters.