What is Augmented Reality and How to Apply It to Mobile | Tapptitude

Augmented reality often loses the battle for the spotlight to its cousin, virtual reality. While both technologies make us think of science fiction movies, we’d like to show you just how accessible AR really is.

Not only accessible, AR is becoming an integral technology in more and more industries. Maybe the most uplifting fact is that the tech is still evolving at a fast pace. It’s in a “wild west” phase of trial and innovation.

We don’t want to just gloss over such an interesting subject, so we’re going to give you a quick rundown of what this article is about below. Feel free to jump to any section that interests you, and let’s begin.

What is augmented reality (AR)?

What does augmented reality mean, really? Augmented reality is the technology that superimposes different types of data onto the real world, mixing real life objects with virtual ones. Imagined over a century ago and prototyped more than two decades ago, AR has long been hailed as the way of the future.

Unfortunately, its evolution was slow at times and things seemed bleak in those first years of development. One of the first ever augmented reality systems was developed in 1992 for the U.S. Air Force. It needed a special helmet and a full upper-body exoskeleton to work. It was cumbersome, expensive and it couldn’t even run Pokemon Go.

Things have changed dramatically in the last decade as technology caught up to our imagination. Augmented reality was embraced by multiple platforms, including mobile.

The new age of AR

Among the first new devices to leverage this groundbreaking technology were wearable augmented reality glasses. This device had its ancestor’s feature – they were worn on the head, like normal glasses. Through their lenses, the user could see virtual objects that looked pretty real for that time.

Even more noteworthy is the great software evolution that augmented reality was undergoing: the announcement of the first frameworks dedicated to the technology. Among them was Qualcomm’s Vuforia, released in 2011, which was coming to multiple platforms. Vuforia developers proved the power of mobile augmented reality, and that acted as the starting point to develop native AR libraries for mobile OS’s.

Seen as a thriving technology, augmented reality started to gain traction and tech giants scrambled to develop native frameworks for multiple platforms. Apple unveiled in 2017 their new and powerful AR framework, ARKit for the iOS. Then Google founded ARCore, the Android rival of ARKit. Wikitude continued its development, and EasyAR, Maxst, and DeepAR became powerful competitors aiming to bring the best AR experience.

Augmented reality on mobile

Both Apple’s ARKit and Google’s ARCore had humble beginnings. You’d run into small bugs and the apps were huge power hogs. With time, the frameworks became more stable and helped developers deliver amazing experiences with just a few lines of code.

Nowadays more and more businesses worldwide are enhancing the user experience by implementing AR-powered interactions. We’ve had the pleasure of working on such projects ourselves, and we’re going to go through one of our favorites. 

Case study: Artfare

One of our clients here at Tapptitude was Artfare, an ecommerce platform that connects artists and collectors, facilitating the buying and selling of artwork. The app was focused on New York City, and that geographical delimitation allowed us to integrate more features without bloating the app. One of those features was augmented reality. So, how did we do it?

The problem

Art is highly subjective. Artworks have personal meaning to us, and they gain emotional value. That’s why buying a painting, for example, is a lot more difficult than buying a nightstand.

You don’t always know where art galleries are and even if you do, who’s to tell if you’ll find something that speaks to you on a deeper level?

Maybe you want to know a thing or two about the artist before making a purchase. Maybe you’re not sure if a certain painting goes well with your curtains because you can’t see them together without buying. In that case, what are you to do?

The solution

You open the Artfare mobile app. The app contains a map with all art galleries in New York and, more importantly, it shows you the paintings displayed at each artist studio or gallery. So, you know where you’re going and what you’ll see.

Once there, you can scan the paintings with the app and get more info right on your phone. No tags or plaques required.

Lastly, you can use Artfare to browse artwork and superimpose art pieces on any wall, anywhere. That means no more second guessing if the piece will be a good fit for your home.

How Artfare uses AR

As you might have guessed, placing virtual artwork on your walls is augmented reality in action.

Besides getting an idea about the aesthetic fit, another big advantage is the fact that you don’t have to try to visualize the size of the painting in relation to your home. Gone are the days of reading measurements and then busting out the ruler to see them in real space. 

Artfare proves how augmented reality can come in handy in an individual’s life, helping art lovers eliminate various steps in the art buying or selling process. This is just one example of what augmented reality can be used for in mobile development, and of how it can improve a user’s life. Still, mobile AR isn’t perfect, and the technology still presents some weak points.

The weak points of mobile AR

Augmented reality as a whole is making great strides, and that includes mobile AR, but things aren’t perfect. When building an AR experience around a mobile device, you run into three major hurdles, which we’ll present below.

1. Accuracy

The sensors used for detecting real-world anchors are liable to bias (as in “this table is actually the floor”). This means that the precision of placed virtual objects can decrease in real-time. Even the newer, sharper mobile device sensors are not without flaw.

2. Battery

The most energy-consuming hardware on any mobile device includes:

• orientation and direction sensors;
• image recognition systems;
• camera;
• GPS;
• display.

Every single item on this list is an indispensable part of mobile AR. In other words, you’ll need to keep a charger handy when playing around with AR on your mobile device. 

3. Accessibility

When the AR frameworks for both OS versions were released, people celebrated. Then, they learned the catch-22 of these frameworks: their device coverage. The need for a powerful processor makes mobile AR apps available only on a limited number of devices – usually those released after 2015.

TappTips for mobile augmented reality

If the list of AR weak points has made you reconsider adopting AR technology for your next mobile app, don’t. Those problems are at least partially hardware-related, but there are plenty of opportunities that lie in UX and software development.

Let’s focus on what you do have control over and cover the best practices of mobile augmented reality development.

1. Plan for the mobile devices’ constraints

An easy mistake to make when integrating AR in a mobile app is to focus on a single scenario and to build around that. For example, tablets have bigger screens so they offer a better AR experience, but a “tablet first” mentality can lead to a lackluster smartphone experience.

When designing the user interface, focus on flexibility. Consider the different screen sizes, then model and scale the UI accordingly. Here are two common aspects:

• Virtual object size – one size does not fit all! When designing an object (or button), think in percentages. The subject should occupy X% of the screen, regardless of screen size.

• Methods of interaction – are you supposed to hold the device in one hand, both, or none? A phone might fit nicely into one hand, but a tablet won’t. Buttons may become hard to reach on large screens or hard to see on small ones, so build an UI that offers the same ease of use for all types of devices.

2. Counter the counterintuitive

On the one hand, augmented reality apps thrive when the user is in motion, looking at virtual sprites from different angles and places. On the other hand, people are used to using two-dimensional apps, i. e. the kind that usually don’t care if you’re moving or not.

This disconnect can result in users not using AR to its full capability and coming away with a subpar experience. It’s not their fault for not knowing, it’s your fault for not telling them. Your objective is to bridge the gap between functionality and habits.

A straightforward way to avoid this issue is to simply let the user know. Have tips pop up on the screen and explain how to use the app and how to access its AR-powered features. Be aware, however, that this method lacks subtlety, and it could hinder immersion, progression and even come across as condescending. 

Another solution can be found in the form of intuitive cues to guide users in the right direction – literally. For example, your virtual object could go on the move, making the user follow. Other visual effects, like a “glow” coming from a direction offscreen, or even an arrow, can help. Another cool idea is to use audio signals like sound effects from the right/left, but that requires the use of headphones, so it’s a less common scenario.

The important thing is to get creative. Find ways to guide your audience in a way that would work for you. That’s an excellent way of making a memorable augmented reality experience.

3. Plan for the space constraints

At the moment, diving into augmented reality isn’t seamless. The first (unavoidable) step is scanning the room in order to define the “action space.” That is also known as world discovery.

Not all users know about this step and you shouldn’t expect them to. Offer a clear guide on world discovery (how to do it, why it’s necessary). The advantages are that you make sure the user doesn’t abandon the app before the experience really starts, and that you can more easily manage their expectations.

Another constraint could be the size of the space available to the user. Depending on the type of experience you’re developing, you might want a large open space, close proximity to a wall, or neither. Generally speaking, tight spaces are the enemy.

Tailor the app to work well in as many conditions as possible. When you run into limits – and you most likely will – explain to your users what is required of them to improve performance.

Some of the worst conditions your AR app will run into are: low-light environments, tight spaces, and rooms where the colors of the walls, floors, or furniture are very similar. These types of factors take their toll on the sensors of mobile devices.

Why do you even have a white, unfurnished room?

As we’ve said, you may already know all this, but your audience might not. Communicate these facts to them and minimise the risk of them giving up in frustration.

4. Show and tell

Sometimes digital objects can seem so real, we can’t tell what’s really there or not. That sounds great for mobile AR apps, but it’s really a double-edged sword. If the objects you create don’t stand out in a meaningful way, they fade into the boring, real-life background. Then, users ignore them. Or, the opposite could happen: users think something real is actually part of the augmented reality experience.

Other times, the app and the users don’t agree on what constitutes a viable plane. That happens when the user places an object on a plane (a wall, for example), but the app doesn’t recognize that plane as such. As a result, the object hovers without being anchored and is liable to move in unintended ways or simply disappear.

Technically, these two negative scenarios are different. But they have the same solution!

Use contrasts and highlights strategically. When placing an object, lighten recognized planes and darken everything else. If you want the user to interact with the object, give it a highlight, an animation, something to grab attention. You can also do all that through text pop-ups at the right moments.

There are millions of ways of telling the user to look at something. Brainstorm ideas, try them out and see what works and what doesn’t.

What’s next for augmented reality?

Augmented reality is making waves in industries like manufacturing, telecommunication, and energy – where the workforce is usually thinly spread over remote areas. In this kind of situation, AR is used for communication and learning.

In addition, educational and healthcare organizations rely on AR and VR as the most engaging and easy-to-use technologies for final users. As an example, healthcare companies are creating AR tools for the operating room, while students experience interactive classes using AR.

Moving forward, AR development will bring out many new features, which will lead to:

• Developers creating even more complex interactions;
• Businesses finding new ways to engage with their clients through augmented reality.

Building a better reality

The tech world we live in is always changing and innovating. Lately, we’ve been getting new technologies that are so good, they’ve become a must-have on the market. AR is one of them.

We constantly hear questions like “Is AR the next big thing in…?”. That applies to gaming, health tech, education, sports, manufacturing, commerce, and many more industries. Competition is extraordinarily fierce on the cutting edge, but AR’s star is still rising and we’re sure we’ll see more amazing developments in the future.

The appeal of AR on the mobile platform is so strong, many users try apps just to experience the AR features. That’s because our need to discover and learn has never been as strong as it is today. AR helps fulfill this need in an immersive way. It brings both fantastic and realistic representations of real world objects in a half-virtual world.