Week 7, Part 2: 30 March - 6 April [Leap Motion, Prototype]

-
Thinking of possible uses of the Leap Motion in the application, I ruled out a pop-up menu as this could only be used to trigger scene changes – not the level of engagement the project requires.
Despite this, I was still able to draw inspiration from the ‘Hover-UI-Kit’ menu I experimented with in the previous blog entry, in Unity. Using a gesture to activate a ‘mode’ then more intuitive movements within the mode for further functionality.

Using the hand to select parts of the airship to then display related information was the idea I decided to develop. This could be done with relative ease through raycasting out from the hand prefab to detect colliders on a ship model.
I used the last bone in the index finger for the origin of the ray, casting the ray forward in the direction of the bones Z-axis. Once this functionality was implemented it needed to be made toggleable, triggered by a user gesture.
Leap Motion provides an example script to detect predefined hand positions. The script works by tracking each digit’s tip, relative to the palm and wrist positions to detect whether fingers are extended. Each fingers position is tracked continuously and its position used to update an enum for each finger (extended, not extended, or either). The fingers’ states are then used to detect gestures. The states enum is set to public, which can then be selected from a drop-down menu in the inspector.
Initially toggled the ray by making a fist, but through testing this was later improved to the point gesture itself (triggers when middle, ring and little finger are not-extended), removing the need for a different toggle gesture.

I began entertaining the possibility of using leap motion when I saw there was no explicit requirement of connection to a computer as I had expected. By using a USB-C/micro USB to female USB adapter, Leap Motion can be connected to the android device itself.
However, after securing the device to my phone in this way, I was forced to reconsider the use of Leap Motion while physically mobile at the exhibition. All other users would also need to secure the device to their phone/phone case with non-reusable adhesive. This isn’t ideal as it may damage their property, will be relatively expensive and not reusable and if not done properly may result in device damage. These factors make the Leap Motion an impractical choice as an application peripheral.

As an alternative to physically moving around the scene, I considered using the hand prefabs position to translate the camera.
Initially, I assumed it would be necessary to get a vector between parallel joints to calculate the rotation, pitch and yaw of the hand prefab.
Luckily, I consulted the Leap Motion documentation before implementing this approach as it showed that the hands rotation and pitch can be accessed from the instance of the hand prefab. It was then a matter of comparing these values to defined thresholds to trigger movement of the camera.

After implementing this, I found the camera would often not respond immediately to input, or move along or rotate about unintended axis. After writing the position values of the hand to the console to locate the issue, I found that even when the user holds their hand as still as possible, erratic value outliers are returned.
I altered the thresholds to increase the size of the deadzone. This separated the range of motions to a suitable degree, preventing hand position from triggering an alternate command.

Using Leap Motion to steer and select
If I were to continue development using the Leap Motion, I could improve the reliability of data given by the Leap Motion by averaging the data to eliminate noise. Averaging the data will add latency, so testing would be needed to balance the results accuracy/latency.
An alternative implementation would be to apply smoothing to the data. Creating a weighted average function, where the further the new data is from the previous value, the less weight it is given.
While Leap Motion proved to be an effective method for obtaining user input, unconsidered issues regarding the practicality of its use will make it incompatible with the project I am building.
Using the device would also restrict the number of potential simultaneous users to the number of Leap Motions brought to the exhibition.

Raycasting out to detect interactable objects was a far better option than the slow transition of steering the ship to trigger UI, and could be made to work with my recent Vuforia build which allows the player to physically move about the scene.
I will attempt to create a similar functionality using Vuforia next week.


References:
https://developer.leapmotion.com/documentation/v2/unity/unity/Unity.HandController.html

https://answers.unity.com/questions/547020/unity-and-leap-motion-help-with-jerkiness-of-input.html







Comments

Post a Comment

Popular posts from this blog

Week 3: 2 - 9 March [Continued AR prototype]

-
Image
I have spoken with the university’s 3D technologist, who is happy to assist with printing the airship model, but does not have the capacity to do so for several weeks. Until I can print a model I will continue to develop the prototype started last week. Instantiating model versus instantiating model with shadow shader. To add an element of realism to the instantiated airship, I added a plane below the model so the light source within the scene would be blocked by the model, casting a shadow onto the plane. The shadow had very poor definition, not appearing to look even vaguely similar the model, and often went unnoticed against grey/white backgrounds. To improve this, I added a shader to the airship model, which casts a much more well-defined shadow. I also updated the components of the plane within Unity, setting its alpha value to 0, making it transparent. This still casts a shadow onto the planes surface, while making it appear to the user that the shadow is cast direc
Read more

Week 1: 16 - 23 Feb. [Meeting with Museum Director]

-
Image
While I was intrigued by both the energy efficient car and the museums interactive experience, the prospect of working with new technology directly related to the content of the degree, where people outside of the university would be the end users made the museum’s project far more appealing. Diss museum had reached out to the university for assistance with their upcoming exhibition, to commemorate the successful double Atlantic crossing of the R34 airship. No brief or requirements were specified in the museum’s request, so after learning of the project I contacted the museum manager, Basil Abbott, to arrange a meeting to further clarify what they hoped to include in their exhibition and available resources. From initial email exchanges it was clear that the intention was to involve the community with a hands-on experience(s) rather than a static exhibition, the implementation of which the museum has left to our discretion. To prepare for the meeting it was necess
Read more

Week 4, Part 1: 9 - 16 March [Understanding Kinect functionality]

-
We decided early on in the week that the three of us working on the R34 centenary project should separate, so that we can explore a range of methods to produce interactive experiences. Of the ideas presented to Basil the three that he was most responsive to were mobile augmented reality, Xbox Kinect and Oculus Rift VR as they each offer simultaneously accessible experience to many users, or deep and meaningful experience for solo users. Henry will continue work into augmented reality, Elliot into VR and I will develop a desktop application using the Kinect. I chose to work with the Kinect v1 (Xbox 360), rather than the Kinect v2 (Xbox One). The latter offers significant improvement over v1, but is itself more expensive and also requires a highly priced adapter to make it USB compatible – making it less suitable for the museum. While I am familiar with the top-level functionality of the Kinect, I have very little understanding of how to use the peripheral to achieve my own
Read more