Responsive Images 201: Client Hints

September 21, 2016 0 Comments images, browser, server, webp, markup, responsive

Responsive Images 201: Client Hints

 

 

Last year, I wrote a ten-part series called Responsive Images 101 that introduced the new responsive images syntax. Today, I want to continue that series by talking about Client Hints.

As the title indicates, this is an advanced responsive images topic. However, despite being potentially complex to implement, Client Hints can make the day-to-day task of creating responsive images much easier for designers and developers.

Client Hints define a way for browsers to tell the server what the types of content that the browser prefers. It provides this information via HTTP request header fields.

Every request that a browser makes to a server contains HTTP request headers. For example, the headers that Firefox sends to request our home page look like this:

GET / HTTP/1.1 Host: cloudfour.com User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:47.0) Gecko/20100101 Firefox/47.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Cookie: _ga=GA1.2.333160365.1465672418 Connection: keep-alive 

You can see HTTP request headers in most developer tools in the network pane.

Client Hints add additional HTTP request header fields containing information about the the browser. One of the main uses of these additional headers is to send information about the size of images required for the page.

There are three Client Hints that are relevant for responsive images:

  • DPR - Device pixel ratio (e.g., 1x, 2x, etc.)
  • Viewport-Width - The size of the viewport.
  • Width - The width of the image in the page.

Of these three, I find DPR and Width the most useful. As I explained in the Responsive Images 101 series, viewport size can be a poor substitute for the actual size of an image.

What could we do if the browser provided width and DPR information to the server and the server knew what to do with it?

Let's take a look at a code sample from the Sizes section of the 101 series:

<img src="cat.jpg" alt="cat" srcset="cat-160.jpg 160w, cat-320.jpg 320w, cat-640.jpg 640w, cat-1280.jpg 1280w" sizes="(max-width: 480px) 100vw, (max-width: 900px) 33vw, 254px"> 

The reason for the addition of the srcset attribute in the markup is to provide the browser with different image sizes that the browser can request. However, if the server knew the size of the image in the page, it could simply provide an appropriately-sized image to the browser.

That's what the Width Client Hint provides. It adds the width of the image, calculated using the information in the sizes attribute, to the HTTP headers that are sent to the server.

GET cat.jpg Accept: image/webp,image/*,*/*;q=0.8 DPR: 2 Viewport-Width: 1024 Width: 508 

Therefore, if the browser and the server both support Client Hints, the markup can be simplified to be:

<img src="cat.jpg" alt="cat" sizes="(max-width: 480px) 100vw, (max-width: 900px) 33vw, 254px"> 

Making this change takes all of the complexity of picking responsive image breakpoints out of the hands of designers and developers and puts the onus on the server.

We've already made some big improvements in our markup by using Client Hints, but there is more that can be done if we couple Client Hints with another HTTP request header: Accept.

The Accept header is used by the browser to tell the server what types of media the browser considers acceptable for the current request. For example, when Chrome requests an image, it adds the following Accept header:

Accept: image/webp,image/*,*/*;q=0.8 

This tells the server that Chrome prefers WebP image formats, but if WebP isn't available, then send a file with a mime type that starts with image ( image/*). If you don't have that, then send whatever you've got (*/*).

The last little bit, q=0.8, tells the server how much the browser prefers the specified format. You can find more on how the quality preference parameter works in the specification.

Adding the Accept header allows us to simplify responsive images markup used to support multiple image formats. Here is the "before" code:

<picture> <!-- serve WebP to Chrome and Opera --> <source media="(min-width: 50em)" sizes="50vw" srcset="/image/thing-200.webp 200w, /image/thing-400.webp 400w, /image/thing-800.webp 800w, /image/thing-1200.webp 1200w, /image/thing-1600.webp 1600w, /image/thing-2000.webp 2000w" type="image/webp"> <source sizes="(min-width: 30em) 100vw" srcset="/image/thing-crop-200.webp 200w, /image/thing-crop-400.webp 400w, /image/thing-crop-800.webp 800w, /image/thing-crop-1200.webp 1200w, /image/thing-crop-1600.webp 1600w, /image/thing-crop-2000.webp 2000w" type="image/webp"> <!-- serve JPEGXR to Edge --> <source media="(min-width: 50em)" sizes="50vw" srcset="/image/thing-200.jpgxr 200w, /image/thing-400.jpgxr 400w, /image/thing-800.jpgxr 800w, /image/thing-1200.jpgxr 1200w, /image/thing-1600.jpgxr 1600w, /image/thing-2000.jpgxr 2000w" type="image/vnd.ms-photo"> <source sizes="(min-width: 30em) 100vw" srcset="/image/thing-crop-200.jpgxr 200w, /image/thing-crop-400.jpgxr 400w, /image/thing-crop-800.jpgxr 800w, /image/thing-crop-1200.jpgxr 1200w, /image/thing-crop-1600.jpgxr 1600w, /image/thing-crop-2000.jpgxr 2000w" type="image/vnd.ms-photo"> <!-- serve JPEG to others --> <source media="(min-width: 50em)" sizes="50vw" srcset="/image/thing-200.jpg 200w, /image/thing-400.jpg 400w, /image/thing-800.jpg 800w, /image/thing-1200.jpg 1200w, /image/thing-1600.jpg 1600w, /image/thing-2000.jpg 2000w"> <source sizes="(min-width: 30em) 100vw" srcset="/image/thing-crop-200.jpg 200w, /image/thing-crop-400.jpg 400w, /image/thing-crop-800.jpg 800w, /image/thing-crop-1200.jpg 1200w, /image/thing-crop-1600.jpg 1600w, /image/thing-crop-2000.jpg 2000w"> <!-- fallback for browsers that don't support picture --> <img src="/image/thing.jpg" width="50%"> </picture> 

There's a lot going on in that example-multiple image formats, numerous image sources, and art direction. It is a bit of an extreme example.

With Client Hints and the Accept header combined, it can be simplified to:

<picture> <source media="(min-width: 50em)" sizes="50vw" srcset="/image/thing"> <img sizes="100vw" src="/image/thing-crop"> </picture> 

Amazing, right?

Notice that the paths for the images in the simplified markup do not have a file extension. That's because the Accept header will tell the server what image formats the browser accepts and what preference it has. It is then up to the server to pick the best image format to send back.

Some of you may be wondering why you've spent so much time learning srcset, picture and the rest of the responsive images syntax if Clients Hints makes it so simple. But there are reasons why Client Hints are a Responsive Images 201 topic.

At the moment, only Chrome and Opera support Client Hints. They are under consideration by the Microsoft Edge and Firefox teams.

HTML can be used without a server. Perhaps you're building an eBook or something that works primarily offline. In those scenarios, you may need responsive images markup.

The server needs to know what to do with the Client Hints HTTP request headers. We may eventually see servers like Apache and NGINX add support for Client Hints, but it will take some time because it will require them to not only support the HTTP headers, but also add image processing capabilities.

In the short run, the way to utilize Client Hints is by signing up for an image resizing service that supports the specification. I know that Cloudinary, imgix and ScientiaMobile all support Client Hints.

There may be additional services that support Client Hints that I'm unaware of, so I've added a column for Client Hints to the spreadsheet of image resizing services so we can track support. If you know of any services that should be updated, please let me know.

Like many image-related topics, supporting Client Hints on the server sounds easy at first. The browser tells the server the size of the image in the page. The server returns the correctly-sized image.

But doing that would cause all sorts of problems as Eric Portis pointed out on the Cloudinary blog:

To deliver perfectly-scaled resources, our servers must render and store hundreds, maybe thousands of alternate versions of your image, which is both computationally expensive and terrible for your pages' performance. More, alternate resources means fewer cache hits, and a hit at any level - in the local browser cache, on the CDN, or server-side - will always be significantly faster than a miss.

You probably don't want small adjustments in the size of the viewport, or a rotation event, to cause the image that has been downloaded to no longer look good. This is one of the side benefits of downloading larger responsive images than needed-resizing the browser doesn't necessarily mean the image will look bad.

You will also want images to get cached. And you definitely don't want to pay exorbitant amounts to cache dozens of image sources on your CDN.

This is where the various image resizing services can set themselves apart. If we give them Client Hints, how smart are they about providing the best possible image size and format?

Because browser makers don't want to add excess HTTP headers to every request unnecessarily, you have to explicitly opt in to Client Hints by adding a <meta> tag:

You should only list the Client Hints that you're going to use in the comma-separated list to avoid excess HTTP headers.

Opting in via the <meta> tag presents a conundrum for your markup. When the browser requests the HTML document from the server, the browser won't supply any HTTP header information that indicates to the server that the browser supports Client Hints because the browser hasn't yet seen the <meta> tag declaring browser support.

The server can add the <meta> tag into the HTML document that it returns to the browser, but what does it do with the markup for responsive images within that same HTML document? The server doesn't know yet if the browser will support the <meta> tag request for Client Hints.

I asked Ilya Grigorik, the Google engineer who has led the charge for Client Hints, if browsers don't declare their support before the HTML is delivered, do you have any suggestions on how to support browsers that support client hints and those that don't?

Ilya replied:

In the meantime, I guess you can run a simple UA check (Chrome 46+) if you want to customize the HTML markup - yes, I recognize this is a bit ironic given that CH is all about removing the need for sniffing the UA string :-)

Before you throw your keyboard out the window, there are good reasons why Client Hints work this way.

Using HTTP request headers to define what content the server should send back is commonly referred to as Content Negotiation. Browser vendors have a long-standing aversion to Content Negotiation and for good reasons.

The opt-in nature of Client Hints is one of the ways that Ilya has tried to address these concerns and get buy-in from browser vendors. Instead of adding HTTP headers to every request whether or not the server requires it, the opt-in limits it to servers that are ready to use the information.

If the server doesn't know if Client Hints are supported before generating the markup, how do we know what markup to generate. Assuming we want to support Client Hints, there are two possible solutions:

  • Server-side user agent detection
  • Supply both Client Hints and responsive images markup

I confirmed that if you supply both the normal responsive images markup and the <meta> tag for Client Hints, that the browser will evaluate the responsive images markup and request the appropriate image source using the Client Hints HTTP headers.

So the simplest solution in the short run may be to use both Client Hints and the responsive images markup.

In reality, whatever implementation makes the most sense is likely going to be dictated by whatever image resizing service you're using. I'd recommend talking to your service provider about what they provide.

Client Hints provide a glimmer of a promising future for responsive images. When Client Hints have broad support, the only responsive images syntax you need will be sizes. This is why I consider sizes to be the superhero of responsive images.

Client Hints also remind us that humans shouldn't be in the business of processing images or picking image breakpoints. As I stated at the end of my Responsive Images 101 series, "The goal for most organizations should be to centralize image resizing and processing and automate as much of their responsive images as possible."


Tag cloud