Over the course of my internship at Cortina Productions Inc. during the summer of 2017, I worked on two projects, one of which is under an NDA. Not too much information can be divulged about the project under the NDA, apart from the fact that I worked as a Unity Programmer on the project and extensively used the Kinect, OpenCVSharp, Fluid Simulation and the Boids Algorithm.

As for the other project, it was a project that spanned 5 weeks – from initial brainstorming to the final product. I was a Unity Programmer on this project as well. I also, frequently, helped with the interaction design and sound design. Moreover, I was involved with the initial brainstorming sessions.

We used Unity and Vuforia for this project for the AR sequences.

The end product for this project was an interactive piece that made use of Mobile Augmented Reality to tell a story about “ghosts” through different, antiquated media devices (a newspaper, a radio and an old CRT-Television).

Trailer Video

Key Challenges Faced:

Having never worked with AR or Android deployment before this project, I not only had to learn how to do this within 5 weeks, I had to make anything I made easy-to-change. This is because our project required a lot of iterations and my code had to be flexible to accommodate this. Our timeline did not make this any easier on me.

Flow of Experience:

The flow of the experience was as follows:

1. Intro Video – tutorial explaining the premise of the experience
2. AR Tutorial – a tutorial explaining how ghosts are spawned by detecting AR markers, and then captured using electricity
3. Non-AR Tutorial – a tutorial explaining how to deal with Non AR scenes
4. AR Scene – Detecting and capturing the first ghost
5. Newspaper Puzzle
6. AR Scene – Detecting and capturing the second ghost
7. Radio Puzzle
8. AR Scene – Detecting and capturing the third ghost
9. TV Puzzle
10. Selfie Scene

 

Contributions:

• Developed a tool that helped speed up the conversion of multiple videos to make them alpha-capable inside the android device we were using for deployment. I used python and ffmpeg to make this tool, which was very similar to the one I made for Project Hindsight
• Rewrote a TV Static shader that I found online to make the static be of any color, and used it for multiple purposes – such as a screen-change-flash effect, and actual static on a TV (for a scene that used the old CRT-TV)
• Changed the marker detection logic such that the ghost that appeared out of the marker depended on the number of markers that had already been scanned
• Developed a tool that interpolated the color of a particle system in unity between two gradients
• Built all the AR elements of the project using Vuforia, and contributed heavily to the non-AR scenes of the project as well.