---
layout: post
title: Improving Core Web Vitals on the Mail.ru home page resulted in an average 10% increase in conversion rates
subhead: |
  Several months of work to improve Core Web Vitals on the home page of Mail.ru resulted in a 60% increase in the 75th percentile in Cumulative Layout Shift (CLS), boosting average session time by 2.7% and conversion rates of core sections by more than 10%.
description: |
  Several months of work to improve Core Web Vitals on the home page of Mail.ru resulted in a 60% increase in the 75th percentile in Cumulative Layout Shift (CLS), boosting average session time by 2.7% and conversion rates of core sections by more than 10%.
authors:
  - denisstasyev
  - svenmay
date: 2021-11-15
updated: 2022-07-18
hero: image/dB6B4Sr8kaaT0KZujRBFC303oFR2/rhX18pRJWKzMdDNFzxr7.PNG
alt: Google Core Web Vitals + Mail.ru Home Page.
thumbnail: image/dB6B4Sr8kaaT0KZujRBFC303oFR2/7lLVG6IeMKZYyhLanvXc.PNG
tags:
  - blog
  - web-vitals
  - case-study
---


## Where we started

Mail.ru is one of the leading e-mail services on the Russian-speaking Internet and is [in the top 5 sites in Russia in terms of traffic](https://www.alexa.com/topsites/countries/RU). Mail.ru is an important resource for many people. It receives several hundred million visits per month, and is a portal from where people can access email, news, social media, performance internet searches and more.

Mail.ru wanted to provide its visitors with a high quality user experience, so work began to improve Core Web Vitals. Before discussing our optimization strategy, a few technical details of the Mail.ru home page should first be noted.

Though the project had long been developed using our in-house templating engine [Fest](https://github.com/mailru/fest), we began to migrate to [Svelte 3](https://svelte.dev/) in 2019.

{% Aside %}
You can find out more about Fest and our reasons for changing the technology stack to Svelte in [our team's presentation](https://youtu.be/H2l3pK42f38?t=2913) at Svelte Russia Meetup \#1.
{% endAside %}

Svelte implements reactivity in a way that [doesn't use Virtual DOM](https://svelte.dev/blog/virtual-dom-is-pure-overhead), which makes it less resource-intensive. [Svelte's approach removes unused functions from production bundles](https://svelte.dev/blog/frameworks-without-the-framework) because the code implementing them isn't generated by the compiler if the functions aren't used. Unused code is removed during compilation, resulting in smaller bundles. This may help reduce [Total Blocking Time (TBT)](/tbt/) during page startup.


## Tracking performance metrics

Before optimizing Core Web Vitals, it's helpful to [evaluate performance in the field](/vitals-field-measurement-best-practices/). Before Core Web Vitals, we tracked other metrics, such as [First Contentful Paint (FCP)](/fcp/), in our internal performance dashboard.

Our metrics collection script was modified to collect Core Web Vitals and transmit them to our performance dashboard for visualization. In line with Google's recommendations, our script uses [PerformanceObserver API](https://developer.mozilla.org/docs/Web/API/PerformanceObserver) to obtain metrics, which is part of [the universal frontend "Platform"](https://github.com/mail-core) inside Mail.ru.

The dashboard displayed the following metrics for users (mean values for the week of 15-21 March 2021):

<div class="table-wrapper">
  <table>
    <thead>
      <tr>
        <th colspan="2" width="40%">Metrics group name</th>
        <th colspan="3" width="30%">Core Web Vitals</th>
        <th colspan="3" width="30%">Other Web Vitals</th>
      </tr>
    </thead>
    <tbody>
      <tr>
        <td colspan="2">Metric name</td>
        <td>LCP</td>
        <td>FID</td>
        <td>CLS</td>
        <td>FCP</td>
        <td>TBT</td>
        <td>TTI</td>
      </tr>
      <tr>
        <td rowspan="3">Share of users in accordance with Core Web Vitals thresholds</td>
        <td>good</td>
        <td>52%</td>
        <td>92%</td>
        <td>33%</td>
        <td>35%</td>
        <td>42%</td>
        <td>43%</td>
      </tr>
      <tr>
        <td>needs-improvement</td>
        <td>19%</td>
        <td>5%</td>
        <td>23%</td>
        <td>38%</td>
        <td>16%</td>
        <td>25%</td>
      </tr>
      <tr>
        <td>poor</td>
        <td>29%</td>
        <td>3%</td>
        <td>44%</td>
        <td>27%</td>
        <td>42%</td>
        <td>32%</td>
      </tr>
    </tbody>
    <caption>Metrics for the week of 15-21 March 2021</caption>
  </table>
</div>

<figure>
  {% Img src="image/dB6B4Sr8kaaT0KZujRBFC303oFR2/ZL7y3ZCpbMcqlOv9mLpV.png", alt="Core Web Vitals before optimization show roughly 1/3 of users in the poor bucket.", width="800", height="479" %}
  <figcaption>Web Vitals values before the improvements.</figcaption>
</figure>

## Improving Core Web Vitals

While plenty of guidance exists for improving Core Web Vitals, every project has unique challenges. For the Mail.ru home page, the following opportunities were identified:

- Implementing placeholders for ad banners to reduce [CLS](/cls/).
- Using server-side rendering (SSR) to reduce [Largest Contentful Paint (LCP)](/lcp/).
- Code splitting to reduce LCP and [First Input Delay (FID)](/fid/).

## Skeletons for CLS improvement

CLS was one of the worst performing field metrics for the Mail.ru home page. Subsequent profiling of this page in the [**Performance** panel](https://developer.chrome.com/docs/devtools/evaluate-performance/) of Chrome's DevTools revealed that ads were the source of the problem. To improve layout stability, our team decided to use placeholders to reserve space for ads before they load.

When implementing placeholders, the first step is to determine the dimensions of the content that will replace them. Luckily, the desktop version of Mail.ru home page has strictly documented sizes for ads. After talking with the design team, SVG-animated UI skeletons were used as placeholders as [they reduce the perceived load time of the content](https://uxdesign.cc/what-you-should-know-about-skeleton-screens-a820c45a571a).

{% Aside 'caution' %}
SVG animations are not supported in Internet Explorer 11, consider using CSS if legacy browser support is essential to your application.
{% endAside %}


## The return of SSR

To ease the transition from Fest to Svelte, we incrementally rewrote the existing project rather than start over. By March 2021, we had migrated most of the frontend to Svelte, and eventually brought SSR to our production application after triaging and fixing backend performance issues.

{% Aside %}
SSR was not used for the entire application, as [it can be an expensive process](https://malloc.fi/performance-cost-of-server-side-rendered-react-node-js) that can delay [Time to First Byte (TTFB)](/ttfb/), which [may affect LCP](/optimize-lcp/#optimize-your-server). We decided to use SSR for crucial content that the user will see on initial render, and offload rendering of hidden content on the client. At the moment, Mail.ru is experimenting with [Islands Architecture](https://jasonformat.com/islands-architecture/) to see if further performance gains can be had.
{% endAside %}

After implementing SSR, the team discovered the cause of CLS regression that initially went unnoticed: the news section was not inserted at the moment of rendering the first content on the page. There was a delay between the initial painting of the page markup provided by the server and the insertion of news section on the client. This behaviour resulted in an ad skeleton shift, which worsened CLS.

Although Chrome's DevTools showed Layout Shift events, we couldn't find the reason for it at first. Though SSR itself wasn't the problem, it helped in discovering the solution later on. Fixing the code responsible for the painting delay improved layout stability of the news component.

<figure>
  {% Img src="image/dB6B4Sr8kaaT0KZujRBFC303oFR2/X5ZgXiNPGEqz6oOlvY5v.png", alt="Active JavaScript just shows an empty page in the news section, hidding the layout jumps.", width="800", height="443" %}
  <figcaption>Finding the news painting problem with JavaScript disabled.</figcaption>
</figure>

<figure>
  {% Img src="image/dB6B4Sr8kaaT0KZujRBFC303oFR2/aXnxfjZCk4oZN8s6ag5o.png", alt="Disabling JavaScript revealed layout shifts, previously hidden from human eyes.", width="800", height="448" %}
  <figcaption>Fixing the news painting problem with JavaScript disabled.</figcaption>
</figure>

Another effect SSR can have on CLS is the movement of components before and after hydration, which can lead to further layout shifts. We encountered this on the mobile version in particular and it required paying special attention to the hydrated component markup. A good solution to this problem was transferring as much display logic from JavaScript to CSS when possible.

## Code splitting and unused polyfills

To improve the perceived page load speed, work was required to decrease LCP and FID values. One way to achieve this is through [code splitting](/codelab-code-splitting/). In addition to the Mail.ru home page itself, our team is developing a widget for portal navigation. It is currently embedded in [many of our company's projects](https://vk.company/en/projects/).

For historical reasons, the widget is inserted at the very beginning of the page as a synchronously loading script. The share of polyfills in this script grew over time. To limit the negative performance effects of loading these polyfills, we implemented code splitting for both modern and legacy browsers.

We decided against the [`module`/`nomodule` pattern](https://philipwalton.com/articles/deploying-es2015-code-in-production-today/) for loading JavaScript bundles for modern or legacy browsers, as the `<script>` element's [`type="module"` attribute](https://developer.mozilla.org/docs/Web/JavaScript/Guide/Modules) didn't target browsers that were modern enough for our needs. To address this, Mail.ru uses an in-house tool for identifying modern browser versions on the backend, and can adapt to those browsers accordingly.

{% Aside %}
The [Browserslist Useragent](https://github.com/browserslist/browserslist-useragent) package is a publicly available alternative to custom tooling for comparing a project's `.browserslistrc` to the User-Agent string in an application backend. However, our team strongly recommends considering using [User-Agent Client Hints](https://github.com/WICG/ua-client-hints#differential-serving) (for example, [Sec-CH-UA header](/user-agent-client-hints/#user-agent-response-and-request-headers), as Mail.ru does in our in-house tool.
{% endAside %}

Once browsers could be identified in the backend, we implemented code splitting for modern and legacy browsers. The result was a 43.3% reduction in size of the synchronously-loaded JavaScript widget for modern browsers. This practice has been applied to some other portal scripts as well.

In addition to bundle size reduction and positive effects on Core Web Vitals, code splitting improves the developer experience as well. Only 3.5% of our users use legacy browsers and that share is on a downward trend, so implementing code-splitting allowed our developers to use the latest browser APIs without introducing the polyfill bloat necessary for legacy browsers to all users.

## Results

After the optimization effort, we observed the mean values for the week of 24-30 May 2021 in our field data:

<div>
  <table>
    <thead>
      <tr>
        <th colspan="2" width="40%">Metrics group name</th>
        <th colspan="3" width="30%">Core Web Vitals</th>
        <th colspan="3" width="30%">Other Web Vitals</th>
      </tr>
    </thead>
    <tbody>
      <tr>
        <td colspan="2">Metric name</td>
        <td>LCP</td>
        <td>FID</td>
        <td>CLS</td>
        <td>FCP</td>
        <td>TBT</td>
        <td>TTI</td>
      </tr>
      <tr>
        <td rowspan="3">Share of users in accordance with Core Web Vitals thresholds</td>
        <td>good</td>
        <td>58% (+6%)</td>
        <td>93% (+1%)</td>
        <td>93% (+60%)</td>
        <td>43% (+8%)</td>
        <td>49% (+7%)</td>
        <td>51% (+8%)</td>
      </tr>
      <tr>
        <td>needs-improvement</td>
        <td>18%</td>
        <td>4%</td>
        <td>3%</td>
        <td>34%</td>
        <td>17%</td>
        <td>24%</td>
      </tr>
      <tr>
        <td>poor</td>
        <td>24%</td>
        <td>3%</td>
        <td>4%</td>
        <td>23%</td>
        <td>34%</td>
        <td>25%</td>
      </tr>
    </tbody>
    <caption>Metrics for the week of 24-30 March 2021</caption>
  </table>
</div>

<figure>
  {% Img src="image/dB6B4Sr8kaaT0KZujRBFC303oFR2/7FIE2St8w6EEowUIqV9K.png", alt="All metrics in the good bucket improved by at least 1%. CLS even by 60%.", width="800", height="376" %}
  <figcaption>Comparison of Web Vitals before and after (change in 'good' group is shown in brackets).</figcaption>
</figure>

The graphs below show changes in web page performance metrics values according to the "Platform". Note the two important dates on the graphs:

- 23 March 2021: the release of iteration with the last page sections migrated to Svelte;
- 19 April 2021: the release of iteration with returned SSR and layout modified to correct CLS regressions.

The decrease in values from May 1 to May 10 is due to May holidays in Russia.

<figure>
  {% Img src="image/dB6B4Sr8kaaT0KZujRBFC303oFR2/8wq9d0uj3EJp5vK6kI49.png", alt="LCP from March to 1 June 2021 showing small improvments over time.", width="800", height="344" %}
  <figcaption>LCP graph in 'Platform': 16 March to 1 June 2021.</figcaption>
</figure>

<figure>
  {% Img src="image/dB6B4Sr8kaaT0KZujRBFC303oFR2/bxANzHIw78TvdS1RFyWG.png", alt="FID from 16 March to 1 June 2021 showing tiny improvements on a high level.", width="800", height="345" %}
  <figcaption>FID graph in 'Platform': 16 March to 1 June 2021.</figcaption>
</figure>

<figure>
  {% Img src="image/dB6B4Sr8kaaT0KZujRBFC303oFR2/7woJXpiHLZU3l7PCdspi.png", alt="CLS from 16 March to 1 June 2021 showing huge improvements starting at April 23rd.", width="800", height="344" %}
  <figcaption>CLS graph in 'Platform': 16 March to 1 June 2021.</figcaption>
</figure>

Results obtained using the "Platform" are in line with the growth of metric values in [Chrome UX Report (CrUX)](https://developer.chrome.com/docs/crux/).

<figure>
  {% Img src="image/dB6B4Sr8kaaT0KZujRBFC303oFR2/41Czys52EyEMyYv3fI2w.png", alt="LCP metric from CrUX showing increase from 51% to 58% in the good bucket.", width="800", height="499" %}
  <figcaption>LCP metric change in CrUX in 2021.</figcaption>
</figure>

<figure>
  {% Img src="image/dB6B4Sr8kaaT0KZujRBFC303oFR2/c8LWvsC52Q58uvKOIZXc.png", alt="FID metric from CrUX showing slight improvment in FID from 91% to 93% in good bucket.", width="800", height="498" %}
  <figcaption>FID metric change in CrUX in 2021.</figcaption>
</figure>

<figure>
  {% Img src="image/dB6B4Sr8kaaT0KZujRBFC303oFR2/ilAn6vKt93uh4ioqGKj1.png", alt="CLS metric in CrUX showing hugh improvements from 46% to 98% in the good bucket.", width="800", height="492" %}
  <figcaption>CLS metric change in CrUX in 2021.</figcaption>
</figure>

A comparison of mean user session duration values a week before the roll-out of initial improvements and a week after the roll-out shows 2.7% growth. Moreover, there is an overall significant increase in conversion in most sections of the page. In particular, conversions to the Mail.ru email app increased by 11.6%, the conversion of the news section increased by 13.5%.

<div class="stats">
  <div class="stats__item">
    <p class="stats__figure">181<sub>%</sub></p>
    <p>Boost of share of good CLS threshold</p>
  </div>
  <div class="stats__item">
    <p class="stats__figure">2.7<sub>%</sub></p>
    <p>Higher mean session duration</p>
  </div>
  <div class="stats__item">
    <p class="stats__figure">13.5<sub>%</sub></p>
    <p>Increase of news section conversion rate</p>
  </div>
</div>

The most unexpected result we got was a 17.4% increase in the Click-Through Rate (CTR) of the marketing banner (its rendering time was significantly reduced by the introduction of SSR and preload tags).

{% Aside %}
An unintended consequence of our work was that we saw a decrease in the conversion rate of some sections on the page. The search section in particular saw a decrease of 6.7%. Our team associates this with a change in the rendering of the page, as earlier on, this component was one of the first to render. We were able to reverse this decrease by introducing new product features into this section.
{% endAside %}

After analyzing the rest of the sections on the page, we noticed significant performance improvement in the vast majority of them. Even for sections such as Weather and Coronavirus&mdash;which are not key on our page&mdash;we see an increase in conversion by 9.6% and 9.5%, respectively.

## Conclusion

Improving performance is challenging in that the work involved may be prolonged. You should regularly monitor changes in metrics over time and ensure that all new product features don't cause regressions in Core Web Vitals. To achieve this, we monitor changes in Core Web Vitals in our [performance budget](/performance-budgets-101/).

Most importantly, we stressed the importance of Core Web Vitals to all members of our product team, from managers and designers to testers and QA. Each team member should be aware of performance metrics and be empowered to improve them. We also incorporate performance optimization objectives into our business processes on a regular cadence. Successfully providing a high-quality user experience is only possible through a joint effort by all team members.
