HEIF and HEVC – new formats of multimedia

CameraDo you remember HD-ready TVs? And later Full HD resolution? In that times, it was something amazing. But time flies, we got 4K resolution, HDR technology, and the main “issue” – much increased bitrate, which is critical to use services like streaming movies via Internet provided by services like Netflix. High data demanding tasks like streaming 60 FPS 4K videos take a lot of data resources from video servers and it affects your Internet connection speed too. Something similar applies on photos taken by our mobile smartphones and tablets. In this case, it’s not just about size of photos, but problem is also image format itself – JPG. This is reason, why new technologies appeared – HEVC for videos, HEIF for photos. Can HEIF replace JPEG? And what is HEVC? If you are not tech-savvy and these terms are new to you, just sit and I will explain it for you by simple speech.

First, I want to say that this article is not for IT professionals, I just want to explain these new technologies in multimedia world by way understandable for common computer user.

Let’s look at HEIF:

Many of us are taking photos by smartphones or by digital cameras. Most users just use default settings and they do not care about what is JPEG, PNG, TIFF or even RAW format. Most consumer cameras just support JPEG (and some higher-end models also RAW) and the same applies to cameras in iPhones and Android smartphones. At App Store/Google Play, many alternative apps for taking photos are present with additional features like manual ISO, aperture and shutter speed (of course, limited by hardware capabilities) and other features like popular photo filters and simple image editor with some effects and textures for services like Instagram and Facebook. In short, we (almost) all are using JPEG. So why HEIF?

JPEG became standard in 1992. It’s a very old lossy format with obsolete compressor engine. There is no support for alpha channel, depth channel and it can’t be animated (you have to rely on GIF with all of its limitations and problematic license). If you are an owner of supported iPhone/iPad, you may know that JPEG does not natively supports Live Photos – Apple was forced to do workaround to get Live Photos work.

HEIF got many improvements. Let’s start with compression. As common user, you may notice that your photos take significantly less disk space. It’s because of new compressor engine, which is up to 2x better compared to JPEG. Also, this is a reason, why HEIF photos are sent faster with lower data usage from your limited mobile carrier plan. And as bonus, HEIF photos do not suffer as many artifacts as in case of JPEG while you are using very high compression.

Next feature you may not notice, but advanced users will: HEIF format is based on containers, so it can store multiple variations or additional representations of a file as auxiliary images, including metadata. This is reason why HEIF photos can natively support Live Photos (as photo with included video) taken on compatible iPhones and iPads. More advanced photographers now can enjoy native support for burst photos.

And compatibility?

This is little tricky. If you are a Windows PC user, you need to use Windows 10 with upcoming Spring Creators Update installed. HEIF support will be added to system’s API, so all 3rd party apps using these APIs will support HEIF automatically.

If you are a macOS user, you will need to upgrade your Mac to macOS High Sierra, if you still haven’t do that. If you are unsure which Macs are compatible with macOS High Sierra, here is list:

  • MacBook Pro (Mid 2010 or newer)
  • MacBook (Late 2009 or newer) – including all Retina MacBooks
  • MacBook Air (Late 2010 or newer)
  • Mac mini (Mid 2010 or newer)
  • iMac (Late 2009 or newer)
  • Mac Pro (Mid 2010 or newer)

If you are an iPhone or iPad user, you need to upgrade your device to iOS 11. For taking photos in HEIF (and capturing videos in HEVC), you need device with A10 Fusion SoC (or newer) running iOS 11. Supported devices are:

iPhones:

  • iPhone X (also supports taking HEIFs and HEVC videos)
  • iPhone 8 Plus (also supports taking HEIFs and HEVC videos)
  • iPhone 8 (also supports taking HEIFs and HEVC videos)
  • iPhone 7 Plus (also supports taking HEIFs and HEVC videos)
  • iPhone 7 (also supports taking HEIFs and HEVC videos)
  • iPhone 6s Plus
  • iPhone 6s
  • iPhone SE
  • iPhone 6 Plus
  • iPhone 6
  • iPhone 5s

iPads:

  • iPad Pro 12.9-inch (2017) (also supports taking HEIFs and HEVC videos)
  • iPad Pro 10.5-inch (also supports taking HEIFs and HEVC videos)
  • iPad Pro 12.9-inch (2015)
  • iPad Pro 9.7-inch
  • iPad Air 2
  • iPad Air
  • iPad 6th generation (2018) (also supports taking HEIFs and HEVC videos)
  • iPad 5th generation (2017)
  • iPad mini 4
  • iPad mini 3
  • iPad mini 2

Note: please check in iOS 11 Camera Setings (Settings -> Camera -> Formats) to set your prefered format. “High Efficiency” means taking photos in HEIF format and videos using HEVC (H.265), “Most Compatible” means taking photos in JPEG and videos will use older H.264 standard. HEIF compression relies on HEVC, so that’s reason why HEIF photos taken on iOS 11 got “.heic” file extension.

If you are an Android smartphone/tablet user, you have to wait for upgrade from Google with codename Android P, which is coming later this year. Thanks to penetration of Android operating system on mobile platforms, this could be good future-proof of HEIF as format on mobile devices, but due to fragmentation of Android platform, it’s not clear how many devices will get Android P upgrade.

Another feature HEIF format benefits from is support to hardware acceleration and parallel operations.

Now look at HEVC:

HEVC is a codec used for HEIF photos and H.265 videos. Streaming 60 FPS 4K video (and I’m not even talking about 8K) is very resources-heavy task for both server and your Internet connection. So new compression standard was created. It features up to 40% better compression compared to older H.264, so you can have same video with smaller size or video with same size, but better quality. Unlike HEIF format, videos will use the same file extensions (like “.mp4” and “.mov”).

For hardware acceleration, you need supported hardware. Some of supported CPUs are listed here:

Desktops and laptops:

  • Intel 6th-generation “Skylake” Core processors or newer
  • AMD 6th-generation “Carizzo” APUs or newer

Smartphones and tablets:

  • Apple A8 SoC or newer
  • Qualcomm Snapdragon 805/615/410/208 SoC or newer (support varries from 720p for low-end to 4K for high-end)

Some newer GPUs are capable of HW acceleration too, some with full support up to 8K and some limited to lower resolutions only with less features. Please read carefully tech specs of GPU before you buy your graphics card.

On Apple platform, when you try to send HEIF photos or videos using HEVC via AirDrop, it will check compatibility. If receiver uses macOS Sierra (or older) or iOS 10 (or older), it will convert media to JPEG/H.264 before sending to ensure compatibility. If you share HEIF or HEVC video via e-mail or other sharing capabilities, it will always re-encode to JPEG/H.264.

Liked it? Take a second to support Lukáš Raynor Majer on Patreon!

Leave a Reply

Your email address will not be published. Required fields are marked *