The Rise of Extended Reality (XR) Technology in Construction

It’s critical for construction professionals to be able to visualize a completed project in advance. Fortunately, we have tools that can provide detailed and thorough plans and schematics  –– these allow owners, architects and general contractors to see a project as it will be before ground is even broken. But there has always been a limit. Until recently, you could only represent a three-dimensional building on a flat surface. But the rise of extended reality (XR) changes everything and gives construction professionals a true 3D model to work from. 

Today’s episode features Amy Peck, the Founder and CEO of EndeavorVR, a strategy and consulting firm focused on augmented reality, virtual reality, and symbiotic frontier technologies. Amy sits down with our host Mike Merrill to discuss how extended reality is changing the pre-construction process forever.  Amy explains what XR is, how it is being used on job sites, and how contractors can implement and see the benefits from XR right away.

 

Key Takeaways:

  1. XR technology comes in many forms. XR covers a number of technologies that give the user more information than their natural senses can give by themselves. Augmented Reality (AR) is on one end of the spectrum, and it gives a heads-up display on a phone or tablet interacting with an image the tablet is capturing. Virtual reality (VR) is on the opposite end of the spectrum. Here, the entire experience is projected from a helmet requiring no physical interaction with the environment. The most useful technology for contractors is Mixed Reality (MR) which is a blending of the two technologies.
  2. MR technology gives immediate feedback on the job site. MR can be used to give information to workers on the ground instantly and with confidence. One example is mixed reality can reveal the internal composition of a wall, revealing a BIM construct overlaying the actual field of view of the user where they can mark out digital readings on the physical environment. This effectively eliminates mistakes during construction and allows easy and efficient maintenance after the building is complete. 
  3. Start with AR when getting into XR. XR doesn’t require prohibitive amounts of investment to get started. AR can work on any modern smartphone or tablet – all it needs is the right software to process the environment and start giving a heads-up display of information that keeps up with what the tablet is looking at. From there it is all about ROI. What part of the tech are you getting the most out of? What could you do with an MR environment? Let those questions dictate where your investment goes from there.

Subscribe to the Mobile Workforce podcast to receive alerts as the new episodes post on Apple Podcasts, Google Podcasts, or Spotify.

Click Play to Listen to the Podcast Now:

Episode Transcript

Mike Merrill:

Hello, and welcome to the Mobile Workforce Podcast. I am your host Mike Merrill, and I am here today sitting down with Amy Peck. Amy is the founder and CEO of EndeavorVR. What they do is they have a strategy consulting firm that’s focused on augmented reality tools and that symbiotic frontier type technology. Super cool stuff. Really excited about this conversation today. Hello, Amy, and thank you for joining us.

Amy Peck:

Hi Mike. Thanks for having me.

Mike Merrill:

Absolutely. There’s all these Rs. This podcast might be our R-rated, I guess, because all the reality going, or you’re a reality star. I don’t know what to really call this, but anyway, really cool, exciting stuff, and I think the listeners will really enjoy tuning in to some of the things that you can shed some light on. But to start off, what is XR technology just for those that may not know.

Amy Peck:

You said the R’s and I think it’s interesting that in this industry our R stands for real reality, but we now have really four acronyms for the different realities. Just to level-set augmented reality, so if you’ve ever played Pokémon GO, it’s basically just like a heads-up display. It’s just putting like plunking a digital object in your field of view, and then you typically would hold up your phone. Eventually, we’ll get to the wearables that look like this, but I used to turn off the AR functionality on Pokémon GO because it was just pasted on. It didn’t have any context to the real world, and so it was more annoying than anything else.

Amy Peck:

And then on the completely opposite end of the spectrum is virtual reality. So you’re actually in a headset, and you can’t see your surroundings, and it’s a completely controlled environment. Then in the middle is mixed realities. That would be MR. That’s where I think it gets really interesting because what  that is, is really a blending of the physical and digital. We’re not quite there yet, but for example, relative to construction. If you were able to hold up your phone and then be able to see BIM data, for example, through the wall, giving you x-ray vision. Then it has context to the actual location, and we’re moving towards that. There are some mobile devices that are capable of showing that interaction. And then we talk about XR, which or extended reality, that’s the umbrella of all of the realities. And then, of course, we can’t forget IRL in good old in real life.

Mike Merrill:

Is that even a thing anymore?

Amy Peck:

Apparently, it’s not because all we do is this all day long.

Mike Merrill:

It’s so outdated. Well, cool. Well, lots of really cool tech topics. If we were to focus in on one, let’s start with maybe XR technology. How is construction taking advantage of XR technology today?

There’s mobile AR, and then there’s the HoloLens 2, which is a Microsoft product, which is a true mixed reality wearable, and then Magic Leap, which is similar. They’re a smaller company, actually, not that small, because they’ve taken in a lot of investment, but technically a start-up based in Florida. Those are your mixed reality and augmented reality devices, and then there’s things like RealWear. Of course, Google Glass is still out there, believe it or not, and Musics and a few others. It’s very difficult to bring this technology into a construction site, mainly because there are some inherent hazards in a construction site. Often there’s not connectivity, which is one of the big challenges. You’ve got workflows that if it ain’t broke, don’t fix it. It’s working, and though I think this technology will be really impactful in the future on construction sites and there are lots of examples of how it is being used today.

Amy Peck:

I would say that for most companies, you may want to start if you’re just looking at this technology in the design and BIM management phase and then the as-built phase, whether it’s around again, viewing BIM data, interior design, maintenance, all of that. I can talk about some of the use cases throughout that process, but imagine remote teams being able to collaborate. That’s one of the great benefits of virtual environments. This can be done in both a mixed reality or AR environment or virtual reality. But if you have remote teams who need to look at either the same 3D asset or the same Revit models. You could do that remotely. We could actually be in virtual reality right now looking at plans for a particular site or being able to take again, the HoloLens I actually have a Magic Leap here somewhere. But you can HoloLens, and you could actually walk just in an empty construction site or say, you’re doing a big retrofit. You could actually walk through the building, put on the glasses, or use your iPad, and you could actually see what the transformation’s going to be.

Mike Merrill:

Like you’re walking through the building, basically. Right?

Amy Peck:

Yeah, and that even extends. You can see how it can easily extend to even interior design, and looking at how you’re going to actually build out the building for a prospective tenant to go in and say, look, these are the five different options that you could have tomorrow. They get to see everything so that we don’t get a million change orders.

Mike Merrill:

Sure. Well, that’s a great tool right there for sure if it could eliminate some of those. One of the areas that I commonly hear it being utilized by more companies today. An easier entry point is maybe wearables for safety. I mean, what can tell us about what’s going on there?

Amy Peck:

There again, HoloLens, you can actually wear safety glasses underneath. Magic Leap are ANSI certified, so really depends on what environment you’re in. They do need to be ruggedized, and there are some great applications in the construction site. Again, you do need wifi connectivity or some server to be able to deliver an experience unless it’s just a singular experience. There are a number of companies who are using it for LiDAR scan visualization so point cloud visualization.

Amy Peck:

There are some companies and very smart companies that are actually using it to trigger payments, and that will certainly ensure that your contractors are actually going to use it because what it does is if you’ve got, let’s say, a smart contract and there are certain tranches you have to hit in terms of completion to be able to trigger that next payment, you can actually go do a LiDAR scan, and you can have your team go and actually do a pass-through video where you are visualizing the LiDAR scan over the current state of the construction, and that can all be recorded and sent back and then verified by the company who owes you money. So that is one of those areas where it’s when anything triggers a payment people pay attention.

Mike Merrill:

It’s funny because I was going to ask you all right. So practically, how does a company actually tie this to revenue, or how can they see this as a cost-benefit as opposed to an expense of cool tech?

Amy Peck:

Yeah.

Mike Merrill:

Love that. This keeps running through my mind, but I just have to ask, how did you get on this path? I mean, are you just a nerd that just loves this stuff and can’t keep away from it or what? I mean, it’s really actually pretty amazing.

Amy Peck:

It’s funny. I’ve always been a little bit of a geek in terms of how do things work? I remember I had a summer job sweeping the tennis courts when I was growing up at the local little tennis club, and the Coke machine broke, and part of my job was also refilling the Coke machines. Of course, I had the keys, and of course, I opened it up. I took it all apart. I remember the tennis pro came and just rounded the corner, and there I was sitting on the floor with a screwdriver and a bunch of parts around me, and he just looked at me, and he just shook his head, and he turned around, and he walked away and but I put it back together, and I fixed it.

Amy Peck:

I was always the one who was if you need the stereo wired… Growing up in school, I was the one who would rewire the stereo, and I’ve always been fascinated by how things work. I think I’ve always been fascinated by just technology in general. I’ve always really enjoyed it. It took a circuitous journey to it, and it wasn’t really until I moved to San Francisco, I don’t know, about eight years ago from New York. I was working with JP Morgan Chase, but we were working on a data product. It was my first time really working with data scientists. It was right when data was becoming a thing, and everyone was talking about big data. It was a really interesting place to be, and that just started my curiosity and that there’s a lot more out there and how do you blend data and technology?

Amy Peck:

Again, I think my focus has always been on how do we do things better. I wasn’t ever really interested in how do we play with it or how do we use it in games. It’s like, how do we just do things better, and how do we improve our processes? In doing so, giving ourselves time back, and I think that does actually improve our lives.

Mike Merrill:

No question. Well, and in this day and age, obviously with the pandemic and all these other things, mobility and the ability to work from anywhere and collaboration remotely has become just critical, I mean, mission-critical. I know companies haven’t totally gotten on the bus yet for all of these cool things. What are some technologies that are surprising to you that are maybe mobile solutions that companies have been able to leverage so far?

Amy Peck:

I have found it interesting. There’s a company called Spatial, and there’s several of these types of companies out there. The reason I like Spatial is that they’re cross-platform, and it’s a web-based collaboration solution. You can join in a Zoom 2D interface like we’re doing right now, or you can join in the headset. You can join on your mobile device. You either show up as a 3D avatar, or you show up as a 2D screen, but you can all look at the same thing. You can import 2D and 3D assets. Right now, if I wanted to show you a digital 3D asset, I’d have to share my screen. You wouldn’t really be able to interact with it. It’s interesting because pre the thing that shall remain nameless that we’ve all been working through for the past year and a half.

Amy Peck:

I travel 200,000 miles a year, so I was already in this. How do we work remotely? I was always traipsing around hotel rooms and looking for wifi in conferences, and it became second nature, so when everything changed, it was fairly simple. But what I found was very interesting is that most companies never really had this remote collaboration construct. Everyone was scrambling, and what it did for our industry was just opened up a lot more interest and proof of concepts. Again, the industry is still technically a fairly young industry. We also saw on the solution side companies really starting to grow and scale because now we have users. We have people out there using the technology saying, wait, this works, this doesn’t work. Oh, wouldn’t it be great if we had X, Y, Z?

Amy Peck:

We need to hit that critical mass on both sides. The technology’s there. If you go out and you buy yourself a Quest 2 headset, regardless of what you feel about Facebook, you’re going to get whatever it is you can get one for $299. You’re going to get a pretty incredible experience, and you’ll really get to see what VR is. But what we need is really a very broad swath of consumers, not just gamers, not just tech-forward people, but consumers looking at the technology and then saying, oh, wait, I could totally do this at my job.

Mike Merrill:

Use it for this too.

Amy Peck:

It’s the same thing that happened with mobile. You remember we used to carry around two phones you get one for work and one, and then finally the phones became so indispensable to us that we got to this, bring your own device construct. Now it’s like, we have customized these devices so that they help us do everything. It’s going to be that same cycle with augmented and virtual reality until we really get some utility and a broad swath of people in it. It’s still going to be those bleeding-edge types of companies that are adopting it.

Mike Merrill:

There are, and it is interesting. You bring up pre-pandemic, and that shall remain nameless. I can even think of probably three years ago. We were on doing a web presentation, product presentation software overview with a company that was well over a billion dollars a year annually. They were having a really hard time logging into the web meeting. I mean, it was 20 minutes of I couldn’t believe what was happening. I’m thinking you guys are how big, but they were just doing it old school and kept growing. I think now everybody knows how to get on a Zoom meeting. It’s not a big deal. It’s not a barrier of entry to anything. That’s just the way we do things now.

Mike Merrill:

It’s interesting too, I think. What I have found is that even some of the most stayed companies now are really they’re in this digital transformation phase. It was a forcing function of what’s gone on in the last couple of years. But I think now, and ironically, I think that some of those companies will have been saved by the fact that we were forced to look how technology could keep us moving and working together, and collaborating remotely. That hopefully has now sparked a continuation because we’re in a really interesting time where it’s not just AR and VR. I mean you look at blockchain, blockchain is ridiculous, with NFTs, and that’s all blowing up.

Amy Peck:

And then artificial intelligence and robotics and automation and all of these technologies, 5G, edge compute, all of this is moving at such an incredibly fast pace that companies who are just sitting back and going, well, we like our ERP systems that we’ve had for the last 25 years and it’s all working fine. A lot of those companies really run the risk of being displaced and not necessarily going out of business, but being pushed off the Fortune 500 and being displaced by companies that are more nimble that are embracing technology. The other really big consideration that companies need to consider is that you’ve got a next-generation workforce coming in. They’re going to go with the companies that have the cool tech. They’re not going to want to go to company that has these arcane systems. I always urge companies. I know that sometimes there’s a stasis in this industry and I understand it because it’s a very, very big industry. The margins can be great, but one mistake can take out profitability on a project.

Amy Peck:

It’s really, really hard to change those workflows, but the beauty of doing it now and again, trying it on the book-ended sides. So in the design phase, in the process phase, even in training, there are a lot of things you can do offsite that can improve processes. In doing that, what it does is it, you get to dip your big toe in and start to make incremental change, and then as these devices start to populate on the consumer side, which they already just saw, I don’t know if you saw it. Facebook recently announced these Wayfarers. It’s funny because I’ve been referring to them as magic Wayfarers forever, and now they actually really are. It’s not the device that we’re going to be using at work.

Amy Peck:

But again, what they’ve done is they’re saying, look, we just need to get these devices on the market. This doesn’t have the functionality that mixed reality does. It’s spatial audio, so it’s really, really good audio. And then they have outward-facing cameras, which creeped everybody out when Google Glass came up the first time, but now we’re all living out loud. We’re so inured to being filmed everywhere. I think that those are some of the ways that companies should look at just trying something, whether it’s AR or VR.

Mike Merrill:

I like that. Speaking of that, what is a minimum cost or investment? I mean, how complicated is it to get something started for even a small to medium size organization?

Amy Peck:

It’s hard to say. I mean, honestly, I would just, if you don’t want to invest in the devices, I would find there are a lot of local universities that have devices, and I would call local university. I would talk to economic development in your city and say do anything about these devices? Does anyone have any? Could we get a demo? Call the companies. Call Microsoft. Microsoft might be tough for maybe one of the development houses that works with them. Call Magic Leap and say, we want to try, we want to try these devices and get a demo. Those are much pricier devices, but you can get like I said, you can get an Oculus for $299. Get one for the office. Even looking at the games, believe it or not, can just give you a hint as to how you would use it.

Amy Peck:

There’s a company called The Wild, and they can take any 3D model, they can take it, and you can ingest your Revit models. Then you can have remote teams all in the same environment, looking at the same models, annotating it, and then recording it and then shooting it back out so that you can share with your colleagues on 2D screens. There are some really lightweight solutions that already exist that if you can’t get a free license or a demo license, they’ll at least walk you through what it does. You can spend a couple of hundred dollars for three or four months to get a temporary license to give it a shot and try it in your processes.

Mike Merrill:

That’s a good idea. I think a good comparable, at least that comes to my mind. I hear of lots of contractors that, in fact, I heard one the other day say, I think we bought five drones. We’re not using them yet. We don’t know how to use them, but we bought them because we think we probably should have them, and so I don’t think a few hundred bucks is a big deal for anybody that’s in business.

Amy Peck:

No, and I think I would even say before even doing that I think just start with one and train somebody up. I think it’s really important before you buy technology, it’s always important to identify what problem are you trying to solve? Because technology is meaningless in a vacuum. If you think about some processes, and that’s why I say get a demo first, then go back and say, all right, now that I understand, because it’s impossible to explain what it’s like to be in a virtual environment. You think you know, and there’s levels too. I think some people maybe have tried mobile VR. So it’s where you take your phone, you put it in a headset, you look really, really cool. No, you do not.

Mike Merrill:

Not stylish.

Amy Peck:

The graphics weren’t great. It wasn’t a great experience. But these new devices, Pico, which was actually just acquired by the company who owns TikTok and the Facebook Quest. HTC has a professional standalone version called the Focus 3. That one’s a little pricey. That’s probably around I think, $500. Again, get demos, go to conferences. There’s one called Augmented World Expo coming up in Santa Clara, California. It’s one of the largest XR conferences. There will be a lot of companies there showcasing different types of visualization tools, design tools, even actually using on the construction site again for that point cloud visualization. There are a lot of touchpoints, and it’s just educating yourself and don’t be afraid of the technology. I think that’s my main message.

Mike Merrill:

Oh, I love that. Don’t be afraid. Dip a toe, do one toe first, right?

Amy Peck:

Exactly.

Mike Merrill:

Maybe not even the big toe.

Amy Peck:

Exactly, exactly.

Mike Merrill:

Obviously, like anything else, you just have to start with something. So your advice is, borrow one, get one, look at someone else’s whatever you have to do to try and get the ball rolling and then ease into it.

Amy Peck:

Reach out to me. Find me on LinkedIn. I know we have lots of people who are experts in every single aspect of building lifecycle management. They can just give you some advice of what to try and some of the software that you could use. We just want to promote the industry. It’s not about trying to get a gig. It’s about pushing the industry forward and just helping people find a path to start bringing this technology in.

Mike Merrill:

I love that. I think that’s great advice. So tell me this, just switching gears a little more personally. What’s something that you are super grateful for in your professional life?

Amy Peck:

Oh wow. I’m actually grateful for my entire professional life. I mean, I joke that I stumbled upwards from day one, but that I’ve been really fortunate to be able to follow a really fascinating and rewarding career path that from the outside makes absolutely no sense to most people. They’re like what you were in the music business and then commercial production, and now you’re doing this. It all made sense to me because it was all a learning experience, and also it was fun for me, and what was exciting, and this industry is really, really exciting for me. I was lucky to be one of the earlier people in the world that I inhabit, which is XR for business and for enterprise and I get to work with really amazing companies and people and technology every day. Pretty lucky.

Mike Merrill:

Very cool. Love to hear that. You’re grateful for your entire professional life. That’s cool. Having said that, what is Amy Peck’s superpower? When you put that Cape on, what are you getting ready to do?

Amy Peck:

Well, aside from always getting really, really good parking spots, it’s amazing, like the people laugh. Someone really drive away right as I pull up, but that’s my general superpower. I think my business superpower, and I think I have the decoder ring between the technical people and people who are less technical. Again, I think I can help people focus on the how and the why and not get so wrapped up in, oh, we should get some drones or we should get a headset. We should look into blockchain, or what are NFTs? It’s more like, how do you want to improve the way you do things, and what are the things that really bug you at work? And then now, let’s look at the technologies that could actually help you do all of that better. So I think I can sit in the middle there and bridge that gap between the super tech talk and a practical use case.

Mike Merrill:

Well, it’s super exciting. I don’t know how you at night with all this cool tech going on, all that stuff. Your brain’s got to be going a hundred miles an hour all the time. I’m sure.

Amy Peck:

It’s pretty fun. It is pretty fun.

Mike Merrill:

Cool. Well, thank you so much for joining us today. I had so much fun and learned some things. And now I got some other homework. I got to go check out and read up on this stuff you’re talking about.

Amy Peck:

There you go. Whenever we’re in the same place at the same time, I’ll put you in a VR headset and-

Mike Merrill:

I love to do that. That sounds amazing.

Amy Peck:

I’ll challenge you with Beat Saber.

Mike Merrill:

Okay. All right. Let’s roll. I’m ready.

Amy Peck:

All right. Let’s do it.

Mike Merrill:

Cool. All right. Well, thanks again. We’ll have to connect up down the road. I’m sure there’s a lot of other great stuff you could talk about, but I’ve surely enjoyed today’s conversation.

Amy Peck:

Thanks so much.

Mike Merrill:

All right. Thank you to the listeners also for joining in on the Mobile Workforce Podcast today. If you enjoyed the conversation that Amy and I had today, or you were able to learn some helpful tips or tricks or some hints that you can help improve your business, please share this episode with your colleagues and friends. Of course, we love it when you hit the subscribe button and, of course, give us those five-star rating and reviews so that we can continue to bring these valuable episodes to others. Of course, our goal here is always not only to help you improve your business but your life.