Tag Archives: Unity

Roames: Virtual Asset Management With Unity

Roames is a Unity-based 3D data visualisation platform created by Dutch company Fugro for the purpose of asset management. In particular it has been used to visualise LiDAR point clouds as the basis for the management and maintenance of power networks. In the video below Glen Ross-Sampson and Peter O’Loughlin from the Fugro Roames team discuss the challenges involved in creating a platform for geospatial data in Unity.

Behind the scenes Amazon Web Services (AWS) are used to provide scalable computation for processing large amounts of point cloud data. Algorithms are run on the AWS clusters to classify and extract different types of features from the point cloud. These include power lines, poles, vegetation and buildings. Further business rules can then be applied and visualised to help users make decisions. In this case they helped Ergon Energy in Australia assess the risk of damage to overhead power cables caused by growing vegetation. The benefit of this kind of ‘virtual asset management’ is that it allows clients like Ergon to make assessments about the assets they manage without having to send crews to inspect every site. By prioritising those sites most at risk they can expect to make significant savings.

Roames was the outcome of a five year project. Unity was chosen as the visualisation client because commercial GIS software didn’t provide the performance they required. They also wanted to be able to customise the interface and experiment with simulation. Using Unity enabled the team to prototype without having to build low level functionality from scratch.

The system allows the user to explore the scene in real-time. Data are streamed in to the scene and unloaded dynamically with the aid of memory and hard-disk caches. Changing level of detail (LOD) is used to support zooming the view from out of space all the way in to a ground level view. As the user zooms in points are replaced by a voxel representation. All of this is achieved using Amazon S3 for cloud storage.

As well as discussing the motivation behind Roames’ and their technical stack the talk does a great job of discussing some common problems and solutions in working with large spatial data sets in Unity.

Technical notes:

Regionation – Map data in Roames is structured according to the Tile Map Service (TMS) specification developed by the Open Source Geospatial Foundation (OSGeo) and served via a Rest API endpoint. Tiles of different LOD are provided based on proximity to the camera. This is also used when the camera is tilted to ensure lower levels of details are used for objects that are further away.

Floating Point Precision – Unity using 32-bit single precision floating point numbers to store the positions of assets. This gives an average accuracy to 7 significant figures. The need to map data across the whole globe to millimeter precision. However, on the scale of the globe accuracy to the nearest metre alone requires 8 significant figures. The spatial uncertainty this introduces is visibly represented by an onscreen spatial jitter. This was resolved by storing the positions of objects with 64-bit double precision and using a floating origin. The floating origin was achieved by setting the position of the main camera to the Unity world origin (0,0,0) each frame and moving the other objects relative to that position rather than moving the camera.

Manipulating large numbers of objects – Manipulating the positions of thousands of objects is computationally expensive and reduces the frame rate. The Roames team used a number of evenly distributed empty game objects or ‘terminal nodes’ as references that other objects could be parented to. This meant Instead of updating the positions of all objects in the scene they just had to update those of the terminal nodes.

Memory Management – As objects are loaded and removed from the scene their are spikes in activity caused by lags in Unity’s automated memory management or ‘garbage collection’; the process by which unused memory is freed for reuse. These issues were resolved by reusing existing objects to avoid allocating more memory and making those objects static where possible. Use of for loops or enumerators was recommended over foreach loops which allocate memory internally. Reducing the amount of string manipulation is also recommended.

Scratch Arrays – Roames introduced their own ‘Scratch Array’ pattern to resuse commonly sized arrays.

Binary Formats – Rather than use KML which is a verbose, text-based XML format, Roames uses the Binary PLY format which performs much better. This reduced file sizes and improved load times and garbage collection allocations.

In order to display the points efficiently they are batched into single meshes on 65,000 vertices. They also lower the density of their clouds prior to loading.

The Core Engine and the other aspects of the product like the user interface were separated to make the project easier to handle. This enabled the 14 developers on the project to work more efficiently. It also meant that other custom tools could be quickly developed in separation from the main project.

The team’s goals going forward to get the product working across the web, to open the Core API to developers, and to start using Unity physics for simulations.

Advertisements

A/B: Participatory Navigation with Augmented Reality

Imagine navigating the city with an augmented reality app, but where the choice of route is determined by a crowd and the decision floats in front of you like the hallucinations of a broken cyborg. A/B was an experiment in participatory voting, live streaming and augmented reality by Harald Haraldsson. Created for the digital art exhibition 9to5.tv the project allowed an online audience to guide Haraldsson around Chinatown in New York for 42 minutes. This was achieved through a web interface presenting the livestream from an Android Pixel smartphone.

The smartphone was running Haraldsson’s own augmented reality app implemented with the Unity game engine and Google’s ARCore SDK. At key points Haraldsson could use the app to prompt viewers to vote on the direction he should take, either A or B. ARCore enabled the A/B indicators to be spatially referenced to his urban surroundings in 3D so that they appeared to be floating in the city. Various visual effects and distortions were also overlaid or spatially referenced to the scene.

More images and video including a recording of the the full 45 minute can be found on the A/B project page here.

Thanks to Creative Applications for the link.

One Man Game Jam: HTC Vive Basketball

HTC Vive BasketballLast week CASA finally received the HTC Vive. Everyone in the office had great fun exploring Valve’s demo experience The Lab. During the week the Longbow emerged as a particular favourite and caused several of us to discuss which sports might work in VR as viable training simulations. Wanting to get to grips with the HTC Vive hand controllers I decided to take up the challenge by creating a basketball simulation for the Vive in Unity.

I started by downloading a SketchUp model of a basketball court from the 3D warehouse. The model had no walls and a lot of reversed faces so I quickly fixed it up in SketchUp with the aid of the S4U Material plugin, ThomThom’s Material Tools and ThomThom’s excellent CleanUp³ plugin. I also obtained a royalty free basketball model from TurboSquid.

Basketball Court SketchUp

As the Unity importer for SketchUp had failed last time I used it I exported the model from SketchUp in Collada format, and converted it to FBX out of habit using the Autodesk FBX converter. After importing the models into Unity I downloaded the SteamVR plugin and added the CameraRig prefab to my scene to handle the basic Vive interaction.

Basketball Court Unity

Trigger colliders were placed in the basketball hoops with a C# script attached to count the score. The Steam scripts for TestThrow and Teleporter were then added to the hand controllers and modified to enable the player to navigate the entire basketball court and to spawn and throw the ball. The ball physics were handled with a simple Unity physics material which was surprisingly effective.

Using the Vive hand controller works well with two qualifications: Firstly it isn’t possible to apply back spin to the ball; secondly there is a high risk of throwing the hand controller out of the window. Risk of breakage in injury aside the final game is really challenging but great fun. I thought I’d actually got the drop on Basketball games in VR but it looks like HelloVR are adding a basketball experience to their social VR platform Metaworld. Could be fun!

Unity 5 Released!

Great news today from the Games Development Conference in San Francisco. After almost a year since its first announcement the latest version of the Unity game engine has just been released.

From a real-time visualisation perspective the newly enhanced graphical features are the most exciting:

  • Realtime Global Illumination
  • Physically-based Shading – Makes materials look more true-to-life under a range of lighting conditions
  • Reflection probes – Adds better

Jointly these help enhance the realism of Unity scenes by more effectively simulating the way light interacts with materials and bounces off of them. This is demonstrated to great effect in Unity’s ‘Viking Village’ example:

These new features have been used to great effect by Alex Lovett in the development of his ‘Divine Shrine’ concept visualisation below. You can read more about the making of the project here.

Additional enhancements for Audio, in game UI and the Editor mean that Unity 5 is a massive leap forward over previous versions in all regards. This is brilliant news for the scores of indie developers who continue to make the most of Unity’s favourable pricing structure and ease in deploying projects to a wide range of platforms.

Over the coming months I expect to be working with the Unity engine extensively and can’t wait to explore Unity 5’s new features. Aside from the obvious benefit of the graphical enhancements I’m looking forward to experimenting with the ability to deploy direct to the web using webGL which should spare users the need to download an additional web plugin.

Unity 5 is now available for download from the Unity website here. Enjoy!

VUCITY: Approaching real-time city information

On Tuesday I was invited to visit Wagstaffs Design in London to take a look at their latest product VUCITY. Powered by the Unity game engine, VUCITY offers an interactive 3D model of London that can be deployed to a touch screen table, video wall, tablet or desktop computer as required. The standalone application enables users to rotate and view the entire scene and zoom down to the scale of individual buildings.

VUCITY Table

Currently VUCITY covers 80 square kilometres of Central London, from Earls Court in the west to the ExCeL exhibition centre in the east, and from Old Street in the north to Battersea in the south. The project is a joint venture between Wagstaffs and Vertex Modelling who are able to provide high detail 3D models of the London area. Created from high resolution imagery the models boast an average accuracy tolerance of 40mm compared with full measured surveys. This degree of accuracy is particularly important for viewshed analysis and visualisation proposed as a possible application in Wagstaffs’ promotional video:

Over time Wagstaffs are seeking to integrate a range of data including data on demographics, property prices over time as well as live data streams for transport. The inclusion of real-time data is possibly the most exciting aspect of the project but also holds the biggest challenge. This is a fantastic project and I’ll be following it keenly. I thoroughly recommend heading over to the Wagstaffs’ website to check out VUCITY and their other great projects today.

Virtual Architectures at the UCL Coding Club

At the end of July I was fortunate to be invited to contribute a workshop on Virtual Reality and the Oculus Rift so the series of coding workshops that UCL have been running over the summer for teenagers at Stratford Library.

This was a fantastic opportunity to give young people hands-on experience of the latest technology and help them engage directly with the possibilities it affords for participatory design. To that end I chose to offer a workshop in which I’d lead participants right through the workflow for creating a 3D scene and generating an interactive walk-through. In doing so I wanted to demonstrate the relative ease with which a 3D environment of any kind could be created with accessible free software including Trimble’s Sketchup Make and the powerful Unity game engine. My hope was that after trying these tools participants might be encouraged to start experimenting for themselves.

Virtual_Architectures_at_the_UCL_Coding_Club_00

Following a brief outline of the session and a presentation about the Oculus Rift we jumped right in to the process of building a model in SketchUp. I began by demonstrating the user interface and walking participants through the creation of a basic room using the rectangle, line, extrusion and offset tools.

Virtual_Architectures_at_the_UCL_Coding_Club_01

Once we had our room we added basic textures and created an array of windows.

Virtual_Architectures_at_the_UCL_Coding_Club_02

After turning the room into a component I demonstrated how the same array technique could be used to quickly turn one floor into an entire building. The ability to copy and array parts of models is fundamental to speeding up work in 3D. It can be tricky to understand and get it right at first so it was really rewarding to watch as the penny dropped. After adding a roof and door to our models we stopped for a break.

Virtual_Architectures_at_the_UCL_Coding_Club_03

During the break it was really encouraging to see that some of the participants were keen to start work on models of their own. The influence of Minecraft was very apparent.

Once we were back at the computers I quickly demonstrated how to export a model from SketchUp and use the free Autodesk FBX Converter to prepare it for import into Unity. As we were running slightly behind at this point I had to move through the process of working in Unity more quickly. After demonstrating the interface I imported the warehouse model we’d created in SketchUp, being careful to note the importance of setting the model scale and position correctly.

Virtual_Architectures_at_the_UCL_Coding_Club_04

A directional light was added for the sun and I imported a model of a town square that I’d prepared in advance of the workshop. The group were really surprised and excited when I told them that I’d been able to create the scene from scratch in about 45 minutes in SketchUp using the same techniques I’d demonstrated earlier.

Virtual_Architectures_at_the_UCL_Coding_Club_05

A skybox was added to complete the scene and the Unity first person controller was added to provide the ability for the player to interact with the scene.

Virtual_Architectures_at_the_UCL_Coding_Club_06

I built the scene and had a working first person experience, in minutes, all within the free version of Unity.

Virtual_Architectures_at_the_UCL_Coding_Club_07

After explaining the process for integrating the scene I’d just created with Oculus Rift we moved to another room for the final demonstration of two Oculus Rift headsets running demos I’d built previously with Unity Pro. Alongside the CASA Urban Roller Coaster I also showed off my City Run driving demo which I’d created with new sample assets from Unity. The latter demo was particularly popular amongst the gamers in the group as it provided the freedom to explore the city for jumps and stunts using an Xbox controller.

[vimeo 105501647 w=625 h=351]

The session was really challenging but we had a great time and managed to achieve a lot. I’d like to share a big thank you with the staff at Stratford Library and Kim and the Public Engagement team at UCL for making this workshop possible. I’d also like to thank CASA for their support.

The making of the CASA Urban Roller Coaster

In May 2014 Virtual Architectures was invited by CASA to create a virtual reality exhibit for the Walking on Water exhibition that was partnered with Grand Designs Live at London’s ExCeL. While CASA spend a lot of time thinking very seriously about cities it was quickly decided that a fun and novel way to engage the 100,000 or so expected visitors would be an urban roller coaster ride using Oculus Rift.

The first tool chosen for this project was the Unity game engine because it provides a very simple means of integrating the Oculus Rift virtual reality headset into a real-time 3D experience. Initial tests were made in Unity with a pre-made roller coaster model downloaded from the Unity Asset Store. However, rather than simply place that roller coaster in an urban setting I wanted to create a track that would be unique to this experience and feel like it might have been part of the urban infrastructure. Due to time constraints it was not possible to model the urban scene from scratch. Instead I decided to generate it procedurally in Autodesk 3ds MAX using a great free script called ghostTown Lite.

Making_Of_CASA_Roller_Coaster_01

Although I like to use SketchUp for 3D modelling wherever possible 3ds MAX was much better suited to this project as it allowed me to quickly generate the city scene, model the roller coaster track, and animate the path of the ride, all in the one software package. After generating the urban scene I used the car from the Asset Store roller coaster as a guide for modelling my track in the correct proportions.

Making_Of_CASA_Roller_Coaster_02

The path of the ride through the city was modeled using Bezier splines, first in the Top view to get the rough layout and then in the Front and Left views to ensure the path would clear the buildings in my scene. The experience needed to be comfortable to users who may not have experienced virtual reality before so it was agreed to exclude loop-the-loops on this occasion. It was also important to avoid bends that would be too sharp for roller coaster to realistically follow. Once I was happy with the path I welded all the vertices in my splines so that the path could be used to animate the movement of the roller coaster car along the track later.

Making_Of_CASA_Roller_Coaster_03

Next sections of track were added to the path I’d created using the 3ds MAX PathDeform (WSM) modifier. As the name suggests this modifier deforms selected geometry to follow a chosen path. Using this modifier massively simplified the process by allowing my pre-made sections of track to be offset along the length of the path and then stretched, rotated and twisted to fit together as seamlessly as possible. This was the most intricate and time consuming part of the project.

Making_Of_CASA_Roller_Coaster_04

In order to minimise the the potential for motion sickness with the Oculus Rift I was careful to keep the rotation of the track as close to the horizontal plane as possible. Supporting struts were then arrayed along the path of the track and positioned in order to anchor it to the rest of the scene. When I was satisfied a ‘Snapshot’ was made of the geometry in 3ds MAX to create a single mesh ready for export to Unity. At this point the path deformed sections of track could be deleted as Unity does not recognise the modifier.

Making_Of_CASA_Roller_Coaster_05

To create the movement of the roller coaster car along the track a 3ds MAX dummy helper was constrained to the path I’d created earlier. This generated starting and ending key frames on the animation timeline. The roller coaster car model was then placed on the track and linked to the dummy helper. It is possible in 3ds Max to have the velocity and banking of the dummy calculated automatically, but I found that this did not give a realistic feel. Instead I controlled both by editing the animation key frames using a camera linked to the dummy for reference. This was time intensive but gave a better result. The city scene, roller coaster and animation were exported as a single FBX which is the preferred import format for 3D geometry in Unity.

Making_Of_CASA_Roller_Coaster_06

Having completed the track and animated the car it was time to assemble the final scene in Unity. First I generated a terrain using a great plugin called World Composer. This enables you to import satellite imagery and terrain heights from Bing maps to give your backdrops a high degree of realism.

Making_Of_CASA_Roller_Coaster_07

The urban scene and roller coaster were then imported and a skybox and directional light were added. The scene was completed with various assets from the Unity Asset Store including skyscrapers, roof objects, vehicles, idling characters and a flock of birds.

Making_Of_CASA_Roller_Coaster_08Making_Of_CASA_Roller_Coaster_09

To prepare the Oculus Rift integration the OVR camera controller asset from Oculus was placed inside and parented to the roller coaster car. In my initial tests with the Asset Store roller coaster I’d found that OVR camera would drift from the forward facing position. This would disorientate the user and contribute to motion sickness. To prevent it with a quick fix I parented a cube to the front of the roller coaster car, turned off rendering of the cube so it would be invisible, and set the camera controller to follow the cube.

Making_Of_CASA_Roller_Coaster_10

In order to ensure the best possible virtual experience it is really important to keep the rendered frames per second as high as possible. As the Oculus Rift renders two cameras simultaneously, one for each eye, you need to aim to render 60 fps in Unity so as to ensure the user can expect to experience a frame rate of 30 fps.

CASA Urban Roller Coaster

In order to achieve this I took advantage of occlusion culling in Unity Pro which prevents objects being rendered when they are outside the camera’s field of view or obscured by other objects.

Making_Of_CASA_Roller_Coaster_11

I also baked the shadows for all static objects in the scene to save them as textures which saves the processor calculating them dynamically. The only objects casting dynamic shadows are the roller coaster car and animated characters.

Finally two simple java scripts were added. The first would start the roller coaster and play a roller coaster sound file upon pressing the ‘S’ key. The second closed the roller coaster application upon pressing the ‘Esc’ key.

UCLProvost

The reception of the CASA Urban Roller Coaster ride at Grand Designs Live was fantastic and I’m really pleased to have participated. It was a great project to work on and an excellent opportunity to learn new techniques in 3ds MAX and Unity. Having my first VR roller coaster under my belt I’m looking forward to building another truly terrifying one when I get the time, hopefully for the Oculus Rift DK2 which has just arrived at CASA.

On a last note I’d like to thank Tom Hoffman of Lake Earie Digital whose excellent YouTube tutorials on creating roller coasters in 3ds MAX provided a great guide through the most difficult part of this challenging project.

Update 26/08/2014 – This article has been featured as a guest post on the Digital Urban blog here.

Update 05/09/2014 – Versions of the roller coaster for use with Oculus Rift DK1 are now available here for Windows and for Mac.