If you want to build a headless WordPress theme, you need to get your SEO right. In this talk, Reyes and Luis walk you through all the SEO challenges you need to consider when building a headless WordPress theme and how to deal with them.
Table of contents:
Main SEO challenges
- Search Engine Requirements
- Important Performance Metrics
What is a headless CMS?
Let’s start by explaining this concept as it will help us understand what comes later. These illustrations explain it quite well:
On the left, you can see a traditional CMS, which is a database-driven structure where the front and backend functionalities are deeply coupled. It can only push content to web-based applications.
On the right side, you can see a headless CMS architecture, which is sometimes also referred to as decoupled. This is API driven and comes without a frontend out-of-the-box. By decoupling the frontend from the backend functionalities, you can manage both separately.
On the one hand, a headless CMS allows you to push content to different channels and devices at the same time. For example, a wearable device, a website, or anything connected via Internet of Things.
Example of a headless WordPress installation
This is a typical headless WordPress installation:
On the left: content creators go to the WordPress dashboard to add or modify content. That’s a PHP server running WordPress.
On the right: visitors go to http://www.site.com and that’s a separate Node server running React. Data from WordPress is retrieved using the REST API.
Benefits of headless WordPress themes
The first one is performance. A headless WordPress theme can be optimized to get the best possible performance and improve your website?s speed.
A fast website is important for SEO (as page speed is a ranking factor for Google searches) but for the user experience as well.
No one likes to stare at an empty screen for more than 3 seconds. That?s why pages with a longer load time tend to have higher bounce rates, lower average time on page and have been shown to negatively affect conversions.
Conclusion: performance is a key factor to make sure users come back to your website and engage with your content.
Modern UI libraries
The second benefit is that this approach sets developers free from the structures of the backend and allows them to use modern UI libraries like React or Vue for the frontend, which are becoming essential to rich user experiences.
Security and scalability
In addition, by building a headless WordPress theme, the frontend is more scalable and the backend can be locked down to improve security.
Lastly, a headless WordPress theme it?s future-proof as integrates easily with new technology and devices (as long as the API endpoint is available).
What about SEO?
We have talked about a headless CMS structure and the benefits of building a headless WordPress theme, but what happens with the SEO?
Traditional CMS platforms like WordPress work well with today?s SEO because they are mainly HTML-based. Search crawlers, or spiders, were originally designed for crawling through HTML.
In addition, these platforms have made it easier to optimize a website by installing a number of plugins which tell you how ?SEO friendly? your web page is.
With a headless CMS, since it is as a backend-only solution, you don?t have the standard SEO functionality that you typically get with a traditional CMS. For example, meta tags are not automatically output (you would need to do this process manually).
This means that if you use WordPress as a headless CMS, you will need to do some additional development to implement SEO.
But don?t worry, there is good news too! First, most traditional SEO rules don?t change. Things like keyword research, link building or regular posting of high quality content, they still apply.
And second, if you are building a headless WordPress theme don?t give up because of the SEO. There are challenges of course, but there are solutions too, and a lot of things that can be implemented to keep your SEO up.
Main SEO challenges
There are 3 important challenges to keep in mind when developing a headless WordPress theme. We’ll go over each of them in detail:
1.1. How does Google index your site?
Well, yes, but no. Let us explain that.
This (see image below) is the normal crawler that’s been around for decades. It only understands HTML and is pretty fast.
This is a normal PHP theme, living in a PHP server:
This approach works but is not ideal because you are sending an empty HTML, which is bad for SEO and not good for the user experience either.
Before we move on, let’s explain this further from the perspective of the servers and the concepts of server side rendering and client side rendering.
1.3. Server Side Rendering & Client Side Rendering
First, let’s see what a static server looks like.
This is the simplest server possible:
- There’s one HTML file stored for each URL of your site.
- It is fast, but you may want to use a CDN to be as close as possible to the end user.
- And what happens when the HTML gets to the browser? Since it’s HTML it works fine, nothing fancy here.
But this approach has two drawbacks:
- First, when the user navigates through your site it needs to start the process all over again: throw away all the DOM > white screen again > wait for the new HTML > and finally render the new page.
- Second, it doesn’t work well with dynamic content stored in a database.
To solve that, somebody said “Hey, let’s improve this”, and they invented server side rendering.
Server Side Rendering
Here, instead of HTML files, we have code in the server, logic that gets data from a database and generates an HTML file for each URL.
Of course, this is slower than serving static files. But if you add a CDN, it is as fast as the static server.
Drawback: This works great for dynamic content but the process still has to start all over again each time users want to visit a new URL.
To solve that, again, somebody came and said “Hey, we want websites to be like native apps”. This is where the client side rendering comes in.
Client Side Rendering
To make websites be like native apps, all the logic must be downloaded and executed in the client, no server code, only client code.
Server & Client Rendering
Finally, what about server side rendering AND client side rendering? Can we do both?
Mission accomplished, right? Well, no…
You are now probably like “OMG! This is super complicated stuff!” ?
Gatsby is a bit special because it’s a static site generator and it works with a data query language called GraphQL.
It’s great because you end up with a static server, which is cheap. But it has the drawback that you need to recreate all the HTML files each time you update content in WordPress.
This framework is not focused on WordPress but they have great support for this CMS using the GraphQL language.
Next.js is a server side rendering solution. Technically, it also has static exports, but there’s no easy solution to use that with the dynamic content of WordPress.
It doesn’t have anything specific for this CMS but you can connect manually using the REST API.
It is not as cheap as a static server but it’s quite affordable.
Frontity does server side rendering and works with serverless as well. Its differential value is that it’s focused onWordPress, so everything regarding this CMS works out of the box.
Nuxt is very similar to Next.js, but made for Vue.js.
2. Search Engine Requirements
This is the second SEO consideration to keep in mind. As part of these requirements we have included metadata, robots.txt and sitemaps.
Metadata is all the information about a page that you send to a search engine that is not visible to your visitors. It is one of the most important practices to optimize your site for SEO.
However, not all metadata is relevant for SEO. Let’s see some of the elements that can influence your rankings. For example, the basic meta tags that go inside the <head> section of your site.
- Meta title: this is the main title of a page and one of the most important elements for SEO. As search users, titles are the first piece of information that we see and we use them for deciding whether a page is relevant to our query. As people can use different devices to view your website, remember to keep your <title> descriptive and short.
- Meta description: this is a short description of the page and can be displayed below the title in Google’s search results. Sometimes Google prefers to display another snippet from your page to match a specific search query. However, this does not mean that you should forget about it. It is important too.
- Canonical URL: when multiple pages have similar content, search engines consider them duplicate versions of the same page. You should always use this element to prevent duplicate content and point Google to the original source of that content (or tell the search engine which version to crawl).
- Social metadata: it allows to control the way your articles or pages are shared on social media networks by defining how titles, descriptions, and more appear in social streams.
Platforms like Facebook, Linkedin or Pinterest use a protocol created by Facebook called Open Graph for social-share-preview. While Twitter has its own meta tags. They are similar to the Open Graph protocol, but they use the ?twitter? prefix instead of the ?og? one.
There are a lot of Open Graph and Twitter meta tags, but you don?t have to use all of them. Here you can see the basic ones that you should keep in mind:
If you want to learn more, you can find a complete list of all the Open Graph and Twitter meta tags in the links below:
How to add metadata manually
As meta tags are not automatically output in a headless WordPress theme, you have to do this process manually.
In Next, Frontity and Gatsby:
You do it inside React with a component called <Head> (or <Helmet> in Gatsby).
It’s very simple. You can add that component wherever you want in your app, and it moves whatever you put inside to the <head> tag of your HTML.
You have to use their head() method with an array of the meta you want to include, it is that simple.
In case you want to use Yoast SEO:
You need to install an additional plugin called REST API ? Head Tags so the metadata from Yoast is exposed in the REST API.
- Then, in Gatsby and Next, you can use the same <Head> component.
- In Frontity, you can use the @frontity/head-tags package and everything is added automatically for you.
- And in Nuxt, you can just use the same head() method again.
Update: If you’re using Yoast SEO 14.0 (or a newest version of the plugin) in Frontity, you can now install the @frontity-yoast package, which is designed to automatically get and render all the tags that Yoast exposes in the REST API.
The robots.txt file is one way of telling a search engine where it can and can?t go on your website, or in other words, which URLs on that site it is allowed to index.
- Gatsby: you can use their gatsby-robots plugin.
- Frontity: you can add the robots.txt file at the root of your theme (next to the frontity.settings.js file) and when you build and deploy your app for production, it will be automatically picked up by the Frontity server.
- Next and Nuxt: you need to create a robots.txt file in the /public or /static folder.
As you can see, adding robots is pretty simple!
Sitemaps are another important requirement for SEO because they help search engines to easily understand your website structure while crawling it.
- Gatsby: it has good support for sitemaps with their gatsby-sitemap plugin.
- Frontity: we want to make use of the WordPress sitemap plugins. There is actually a conversation going on about including the sitemaps in the WordPress core, so we will release a package for sitemaps taking into account all these scenarios.
- Next and Nuxt: as far as we know, there’s no simple way to generate a dynamic sitemap like this, which means that you will have to create one yourself.
3. Important Performance Metrics
As previously mentioned, performance is very important, not only for the search engine optimization but for the user experience as well.
3.1.FCP, FMP and TTI metrics
As we can see from the image below, one of the most important performance metrics after the blank screen is the First Contentful Paint (FCP).
The First Contentful Paint (FCP) is triggered when any content from the DOM is painted. This could be a text, an image, a SVG, or a canvas render. This metric is important for users as it provides feedback that the page is actually loading.
Then we have the First Meaningful Paint (FMP). This is the time when the primary content of a page appears on the screen.
Definitions of primary content differ depending on pages. For example, for blog articles, it would be the headline + the above the fold text. But if an image is critical to your page (e.g. if you have an e-commerce product page), then the First Meaningful Paint requires that image to be visible.
This is going to be our primary metric for the user-perceived loading experience.
Finally, we have to consider the Time to Interactive (TTI) metric, which measures how long it takes a page to become interactive. Or in other words, when the users get the real thing and can start using it.
3.2. How to improve the First Meaningful Paint
Technically, the First Meaningful Paint is the time between the moment a user enters the URL of your site, or clicks on a link that points to your site, and the moment that the browser renders the HTML of your site.
If you do proper server side rendering, your contentful and meaningful times should be pretty similar.
There are two things you can do to improve them:
- First, use a CDN to serve your HTML files as fast as possible, that’s a must.
- And second, make sure your HTML is ready for render. What do we mean with this?
HTML files should be ready for render
- Use server side rendering to generate proper HTML files (we hope that’s clear now after this talk!).
- For CSS, you may want to use a library like Emotion or Styled-components because they put all the CSS in the <head> tag of your HTML files. This way the browser doesn’t need to download any additional CSS file. You may say “Well, that sounds kind of tricky”, but that’s what Google itself is doing in their own Google AMP format. Believe us, it works and it is fast.
- Fonts: try to avoid them as much as possible. If you absolutely can’t, use font display: swap so you don’t block the HTML render.
3.3. How to improve the Time to Interactive
What techniques can we use to reduce the bundle size?
- First, remove all the unused modules. We’ll show you later how you can inspect your modules.
- And finally, use code-splitting. Let’s see this point in detail.
Then, once everything is loaded, if the user navigates to other parts of the site you can load that code. Actually, you can even prefetch that code if you think a user is likely to move to another part of the site.
Again, this looks like super complicated. But all these tools are included in the main frameworks with APIs that are easy to use and understand.
React Concurrent Mode
Finally, there’s another thing coming that we believe will help with Time to Interactive: React Concurrent.
Right now, the current React (called React Sync) does its rendering in a single chunk. But React Concurrent divides the work in smaller chunks, so they never block the CPU.
The React team has been working on this for a while now, so it will be probably released later this year.
We’re sure Vue will eventually get something like this too.
3.4. Useful tools
To conclude this talk, these are some of the Google tools which you can use to analyze and optimize your metrics:
Luis went more in detail over these tools during our talk, feel free to check it out (starts at 30:29).
We hope this post gives you a better understanding of the SEO challenges that come along with building a headless WordPress theme, how to solve them them, and most importantly, how to keep your SEO up with the right tools and knowledge base.