Week 2, Part 2: 23 Feb. - 2 Mar. [AR Prototype]

-
I began looking at the available AR platforms to identify the most appropriate. The most promising options were ‘ARKit’, ‘ARCore’, ‘Vuforia’ and ‘8th Wall’.
‘ARKit’ is only for iOS; ‘ARCore’ is only for Android. Both these options exclude too many potential users.
‘8th wall’ and ‘Vuforia’ are both cross platform and offer thorough documentation, however ‘8th wall’ doesn’t offer image tracking where ‘Vuforia’ does. ‘Vuforia’ is also officially integrated into unity 2017 and higher, making it my best option.

Using Vuforia within Unity is an intuitive experience. Vuforia allows users to upload images, creating a database reference image which can then be imported into Unity. After activating a Vuforia license key, any Unity function can be triggered when the device camera detects an image identical to the one used for the database image.

The first issue I found was that the model failed to load. After troubleshooting the problem and reading the Vuforia documentation, I found that the app works by detecting ‘features’ within images. It ignores colour and creates reference points by identifying intersecting areas of high contrast (non-repeating, high variance of features is preferable). Vuforia rate each reference image out of 5 to indicate how easily recognisable the reference points in the database image are. My picture was rated at 2 stars. 
After reviewing my reference image’s features, it was clear that the similar ‘orange’ tones throughout the image did not provide enough areas of contrast – nor did the airship body provide sufficient angular diversity for Vuforia to recognise. Both of these yielded a very low number of image features. 

 After sourcing a more suitable 5/5 image (due to a far greater number of features) and repeating the process, the model was instantiated without issue. 
I will need to ensure future database images chosen take these criteria into account and are of 5 star rating in order to reliably trigger functionality.

Another issue was the model ‘shaking’ once instantiated. After further reading into the Vuforia documentation and browsing the forums, I found the model instability is caused by a combination of factors: flatness of image (creases hinder detection), finish of image (glossy images hinder detection), lighting of area (dark environment hinders detection), size of image (the smaller the image, the harder to detect) and camera mode (“CONTINUOUS_AUTO” smooths the 3D model).
Using a database image with these improved attributes greatly reduce the issue, though to completely solve the problem a script which updates the transform and rotation of the scene camera to that of a child object whose position and rotation is an average of the most recent position between the viewing camera and position of the instantiated model should be implemented.

Vertical Database
Horizontal Database

 After solving my problems, I created a script to rotate the airship using an onscreen joystick, giving the user some very basic interaction. I then began testing detection of database images at different angles, as this will likely be required at the museum exhibition. This meant the axis about which the ship rotated had to be considered.

I then adapted the app to have the joystick become visible only when the database image is detected. Once this was implemented, I used the same approach to trigger information about the loaded model as it is instantiated. This approach can be used to start any visual effect or UI.


We then forwarded Basil a link to the below YouTube video which shows a prototype of the implementation, as without a reference the concept may have been misunderstood.

Video sent to Basil
The next stage will be to explore detection of 3D models to trigger further visuals as well as object permanence (3D model will not be destroyed when device loses database image).

Comments

Post a Comment

Popular posts from this blog

Week 3: 2 - 9 March [Continued AR prototype]

-
Image
I have spoken with the university’s 3D technologist, who is happy to assist with printing the airship model, but does not have the capacity to do so for several weeks. Until I can print a model I will continue to develop the prototype started last week. Instantiating model versus instantiating model with shadow shader. To add an element of realism to the instantiated airship, I added a plane below the model so the light source within the scene would be blocked by the model, casting a shadow onto the plane. The shadow had very poor definition, not appearing to look even vaguely similar the model, and often went unnoticed against grey/white backgrounds. To improve this, I added a shader to the airship model, which casts a much more well-defined shadow. I also updated the components of the plane within Unity, setting its alpha value to 0, making it transparent. This still casts a shadow onto the planes surface, while making it appear to the user that the shadow is cast direc...
Read more

Week 7, Part 2: 30 March - 6 April [Leap Motion, Prototype]

-
Image
Thinking of possible uses of the Leap Motion in the application, I ruled out a pop-up menu as this could only be used to trigger scene changes – not the level of engagement the project requires. Despite this, I was still able to draw inspiration from the ‘Hover-UI-Kit’ menu I experimented with in the previous blog entry, in Unity. Using a gesture to activate a ‘mode’ then more intuitive movements within the mode for further functionality. Using the hand to select parts of the airship to then display related information was the idea I decided to develop. This could be done with relative ease through raycasting out from the hand prefab to detect colliders on a ship model. I used the last bone in the index finger for the origin of the ray, casting the ray forward in the direction of the bones Z-axis. Once this functionality was implemented it needed to be made toggleable, triggered by a user gesture. Leap Motion provides an example script to detect predefined hand positions. Th...
Read more

Week 1: 16 - 23 Feb. [Meeting with Museum Director]

-
Image
While I was intrigued by both the energy efficient car and the museums interactive experience, the prospect of working with new technology directly related to the content of the degree, where people outside of the university would be the end users made the museum’s project far more appealing. Diss museum had reached out to the university for assistance with their upcoming exhibition, to commemorate the successful double Atlantic crossing of the R34 airship. No brief or requirements were specified in the museum’s request, so after learning of the project I contacted the museum manager, Basil Abbott, to arrange a meeting to further clarify what they hoped to include in their exhibition and available resources. From initial email exchanges it was clear that the intention was to involve the community with a hands-on experience(s) rather than a static exhibition, the implementation of which the museum has left to our discretion. To prepare for the meeting it was necess...
Read more