Week 2, Part 2: 23 Feb. - 2 Mar. [AR Prototype]

I began looking at the available AR platforms to identify the most appropriate. The most promising options were ‘ARKit’, ‘ARCore’, ‘Vuforia’ and ‘8th Wall’.

‘ARKit’ is only for iOS; ‘ARCore’ is only for Android. Both these options exclude too many potential users.

‘8th wall’ and ‘Vuforia’ are both cross platform and offer thorough documentation, however ‘8th wall’ doesn’t offer image tracking where ‘Vuforia’ does. ‘Vuforia’ is also officially integrated into unity 2017 and higher, making it my best option.

Using Vuforia within Unity is an intuitive experience. Vuforia allows users to upload images, creating a database reference image which can then be imported into Unity. After activating a Vuforia license key, any Unity function can be triggered when the device camera detects an image identical to the one used for the database image.

The first issue I found was that the model failed to load. After troubleshooting the problem and reading the Vuforia documentation, I found that the app works by detecting ‘features’ within images. It ignores colour and creates reference points by identifying intersecting areas of high contrast (non-repeating, high variance of features is preferable). Vuforia rate each reference image out of 5 to indicate how easily recognisable the reference points in the database image are. My picture was rated at 2 stars. 

After reviewing my reference image’s features, it was clear that the similar ‘orange’ tones throughout the image did not provide enough areas of contrast – nor did the airship body provide sufficient angular diversity for Vuforia to recognise. Both of these yielded a very low number of image features. 

After sourcing a more suitable 5/5 image (due to a far greater number of features) and repeating the process, the model was instantiated without issue. 
I will need to ensure future database images chosen take these criteria into account and are of 5 star rating in order to reliably trigger functionality.

Another issue was the model ‘shaking’ once instantiated. After further reading into the Vuforia documentation and browsing the forums, I found the model instability is caused by a combination of factors: flatness of image (creases hinder detection), finish of image (glossy images hinder detection), lighting of area (dark environment hinders detection), size of image (the smaller the image, the harder to detect) and camera mode (“CONTINUOUS_AUTO” smooths the 3D model).

Using a database image with these improved attributes greatly reduce the issue, though to completely solve the problem a script which updates the transform and rotation of the scene camera to that of a child object whose position and rotation is an average of the most recent position between the viewing camera and position of the instantiated model should be implemented.

Vertical Database

Horizontal Database

After solving my problems, I created a script to rotate the airship using an onscreen joystick, giving the user some very basic interaction. I then began testing detection of database images at different angles, as this will likely be required at the museum exhibition. This meant the axis about which the ship rotated had to be considered.

I then adapted the app to have the joystick become visible only when the database image is detected. Once this was implemented, I used the same approach to trigger information about the loaded model as it is instantiated. This approach can be used to start any visual effect or UI.

We then forwarded Basil a link to the below YouTube video which shows a prototype of the implementation, as without a reference the concept may have been misunderstood.

Video sent to Basil

The next stage will be to explore detection of 3D models to trigger further visuals as well as object permanence (3D model will not be destroyed when device loses database image).