Key Considerations When Developing for Multiscreen: Mobile, Tablet, and TV

Hao Chen
Hao Chen

What developers need to know about multiscreen development.

Brand recognition is crucial when a product is distributed in many forms. Having a unified user experience across a fragmented device market ensures brand consistency. It’s why cross-platform application design (e.g. Android, iOS) is so popular in the developer community. But, the conversation around building apps beyond mobile is few and far between. Multiscreen options such as TV-connected devices, game consoles, and Smart TVs are having a renaissance right now, which makes telling a brand story on all screens a must.

This article describes key considerations for front-end development beyond traditional mobile apps.

mutliscreen_cross-platform_devices

Platform vs Form Factor?

Cross-platform design refers to an architecture that uses a single codebase (or a majority of the codebase) to build an application on multiple platforms. This is one of the most common topics for front-end developers. However, if you look into current user behaviour, you’ll notice that end-users use many devices of different form factors throughout the day. As an example, I have an Android phone, an iPad, an Amazon Fire TV stick, a PlayStation 4, and more.

mutliscreen_cross-platform_form factors

The platform dictates the OS but the form factor refers to:

  • Screen size, resolution, aspect ratio (720p, 1080p, 4K, 4:3, 16:9, etc)
  • Screen distance (handheld, TV)
  • Interaction method (touch, remote control, game controller)

A developer might say:

Since each form factor is different, we should design and build separate applications to suit individual needs.

With that in mind, here’s what developers need to consider. Let’s use building a simple VOD app as an example.

The Thought Process

Screen Layout

Consideration 1:

Mobile phones and tablets allow the user to choose portrait or landscape mode. TV is always in landscape. How are we going to design a screen layout that accommodates all?

mutliscreen_cross-platform_orientation

The orientation of mobile and tablets (source: khef.co)

The short answer is you can’t. The long answer is that we can achieve this with advanced planning, a UI framework that can iterate designs quickly, and sophisticated plans for implementation. More on this in another article.

Let’s assume we decided to lock the screen to landscape. You’ll hit the next question.

Consideration 2:

For each category of device, there are different aspect ratios: older phones are 3:2 and 5:3, while newer ones are 16:9 and 16:10. Tablets have a common aspect ratio of 4:3, while others are 8:5 and 16:9. Finally, TV screens are 16:9. How do we account for different aspect ratios?

mutliscreen_cross-platform_screen resolutions

Aspect Ratio for various screens (source: dvxuser.com)

The short answer is to use a responsive layout. Thankfully, a popular cross-platform framework, React Native, brings the Flexbox and CSS from web development to iOS and Android applications.

With You.i Engine One, our SDK supports core React Native functionality, enabling consistent tooling across all devices. For more details, please see here.

Consideration 3:

Now that our app is responsive, how do we account for different screen resolution and density? Also, some elements look good on mobile but odd when viewed at a distance for TV. How do we resolve these discrepancies?

mutliscreen_cross-platform_DIP

Image Resolution, Scale Factor and Pixel Density (source: developer.apple.com)

The answer is Device Independent Pixels aka, DIP. DIP is a form of abstraction allowing developers to work screen space measurements independent from screen resolution and pixel density. This enables developers to display the same size graphics without custom calculation for each display hardware. The underlying framework converts the abstract measurements into real pixels for the particular device.

Consideration 4:

After some prototyping, the design team is suggesting a different experience on large screen devices. Great.

Now we’re talking. Today, the layout is created by a designer but implemented by a developer. Although it might have responsive layout properties, such as alignment and wrapping behaviour, a different look and feel cannot be achieved in an app without code changes. It’s frustrating to constantly communicate back and forth between design and development. Oftentimes, both teams have to compromise.

At You.i TV, we can actively realize a design team’s vision for a brand through our You.i Engine One technology. Our engine abstracts the concrete UI elements from the developer’s code. The developer writes code to describe business logic, such as what happens when the user interacts with a button. The look and feel (motion, visual, etc.) of said button is abstracted away. For example, a user can scroll a list vertically or horizontally. This doesn’t affect the logic of the button action, which is activated through click, tap, or press.

User Interaction

As a mobile app developer, this is straightforward. Users can perform actions such as a tap or a swipe. But how does this translate to large screen devices such as TVs, set-top boxes, and game consoles? Would you consider the following flow?

  1. Get up from the couch
  2. Walk up to the TV
  3. Touch the screen
  4. Go back to the couch

First of all, in the consumer market, there are very few TVs with a touch-enabled screen. Secondly, TV remotes were invented to control the TV from a distance. The screen size and viewing distance of a TV affects the user experience significantly. The original TV remote was designed with the following functions:

  • Change channels (Up/Down)
  • Change volume (Up/Down)
  • Enter channel number
  • Navigate and change TV settings

mutliscreen_cross-platform_remotes

Recommended reading: Architecting a Front-End Stack for Multiscreen Pay TV Services

Modern TV remotes use a standard Directional Pad (D-Pad) input. Users have the option of four directions (up, down, left and right) to navigate two-dimensional space on a large screen application.

mutliscreen_cross-platform_remote

A seasoned application developer might be thinking:

Wait a minute, did you say two-dimensional space? Does that mean you cannot build an application that includes three-dimensional elements such as overlay, pop-ups, etc?

Of course, you can. Keep in mind that you can only navigate in two-dimensional space. What this means is you’ll have to use X and Y coordinates to navigate, and ‘Enter’ or ‘OK’ buttons to select options (similar to a tap or a click on a mobile device) but there is no Z directional navigation.

mutliscreen_cross-platform_focus

At You.i TV, we solve this problem via a concept we refer to as Focus Root (aka Focus Trap). It is the Focus Management System’s ability to limit the next focus within a subset of the elements. The details can be found here.

So What?!

Taking into account all above, after 900+ words, you might be thinking;

Why can’t we just create an app for each form factor? This is too much work. We can still achieve a unified user experience across all form factors by tweaking each app.

I have to admit, this process takes dedicated resources and skill. However, implementing designs and changes across all form factors one-by-one takes a significant amount of time and resources. iOS and Android developers are not familiar with the complexities that come with developing for Xbox One. Just like, Windows developers are ill-equipped to navigate Roku’s native BrightScript language. Not to mention the maintenance of each platform and subsequent updates that will need to be implemented one-by-one.

Our You.i Engine One SDK allows users and customers to achieve the goal of a unified, branded user experience despite device fragmentation. Similar to cross-platform design on mobile, You.i Engine One abstracts common functionality between apps on any OS and form factor. Introducing You.i Engine One to your development team saves considerable time and resources via the paradigm of “Code Once, Deploy to All.” For more information, watch our overview series or request a one-on-one demo with an expert.

Here’s something similar we think you’ll enjoy.