In today’s column, I will discuss a frequently ignored aspect of WordPress behavior: the handling of http 404 ‘not found’ errors. I will explain the implications of this problem and I will show you what I do to fix the issue.
The issue with handling 404 ‘not found’ errors is that there are two very different situations where such errors can occur. These need to be handled differently — and they aren’t.
The problem is compounded by the fact that WordPress itself does not handle the 404 ‘not found’ problem at core level, but instead leaves it to theme developers to provide error handling via a specially created 404.php template. While there are many fantastic WordPress themes, I have yet to find one that comes with a functional 404 page.
So it ultimately becomes the responsibility of the website designer or manager to create a working 404 page that keeps visitors on board instead of making them leave the website and monitor for 404-inducing errors.
The Two Different Causes for 404 ‘Not Found’ Errors
Web servers use 404 ‘not found’ errors to signal that they cannot fulfill a request for a specific resource (file). In WordPress context, it is nearly impossible to force a real 404 error, because the CMS is coded to intercept all kinds of file queries and either returns some kind of output or blocks them.
Technically, WordPress should never cause server-level 404 errors. However, there do exist two types of conditions that can still cause 404 errors.
The first — and by far more popular — case is to request content that has existed before but has been deleted or moved. These errors are reproducible: as long as somebody follows a link that points to nothing, the 404 error will appear.
The second, much rarer case is to pass to the browser incorrect URLs (either typed by hand with a typo, not copied/pasted in its entirety, or otherwise mangled via social sharing, etc.). These errors are intermittent: if visitors realize they have mistyped a URL, they could fix the issue and reach the content (however let’s be honest: not many people will bother).
Regardless of the root cause for the 404 ‘not found’ error, everybody agrees that 404 pages are very bad for UX and webmasters should do their best to prevent users from experiencing them, ever.
404s are not the kind of errors that can sink your SERP rankings, but they silently work to destroy your visitors’ confidence in your ability to provide them with up-to-date, quality content.
If a visitor sees a 404 ‘not found’ error at any time during their visit to a website, they are unlikely to report it. Rather, they will close the page and move on. Maybe even conclude that if you have 404 errors on your website, then who knows what else there is outdated!
So what is the best way to handle both causes of 404 errors? This is what the rest of this column is all about.
Handling 404 Errors for Changed/Moved/Deleted Content
Moving, editing, and deleting content is by far the most common cause of 404 errors in WordPress. Thankfully, it is also very easy to manage and monitor.
Possible error 404-causing situations include:
– renaming a page and changing its slug for SEO purposes;
– relocating/reclassifying a page within a category, which can cause a change of the canonical URL in case the permalink structure includes categories in the page URLs;
– retiring a product that is no longer available;
– intercepting an incorrectly posted but valuable link from a social media post or a 3rd party website.
In order to manage such content, you need to equip your WordPress website with a redirection monitor. The best solution that I recommend is Redirection, which is free of charge, reliable, and trustworthy (2+ million installs, 12+ years in existence).
Side note: Redirection can work alongside the free versions of popular SEO plugins such as RankMath, All in One SEO, and Yoast SEO. The premium versions of all these SEO plugins however ship with similar functionality that supersedes what Redirection offers.
The principle in these premium tools is the same, but they offer extra convenience for repeating tasks which might justify their price alone. For example, RM Pro will create an automatic redirect when you edit the post slug or change the canonical URL of a product by moving it into a different directory. All of them will ask if you want to create a redirect when you delete a page, blog post, or product, instead of relying on you to remember to do so.
If yours is a highly dynamic website, you might consider a paid solution. But Redirection will work even then; you just need to learn how to use it. This is what I am about to show you next.
How to Configure Redirection
Redirection automates two important activities that will help you avoid creating 404 ‘not found’ errors when you manage your content by renaming it (changing the slug), relocating it (changing the canonical URL), and/or deleting it.
You can configure Redirection to watch for such changes and log them. In order to enable this functionality, you need to navigate to the plugin settings page (Tools → Redirection), scroll down to URL Monitor, and select any or all of the possible post types:
Note: If you are wondering what ‘bitform pages’, ‘funnels’, and ‘steps’ are, these are custom post types added to this WordPress instance by different 3rd party plugins that I have installed there (a form creation plugin, a sales funnel plugin, and an automation plugin). It serves to show you how flexible Redirection is in its scope of operation.
I recommend that you select all types of URLs for monitoring. For a simple demonstration, I will only enable ‘Monitor changes to pages’ and then proceed to create three pages and delete one of them to see what happens:
If Redirection is enabled and configured to keep track of deleted pages, as soon as I delete the ‘Delete Me Page’, it will show up in the list of trashed pages maintained by the Redirection monitor:
However it just sits there doing nothing, and if I try to open the page, WordPress will load its 404 page and tell me it couldn’t find the page I was looking for. If you hover with your mouse over the table row for the Delete Me page (or tap with your finger if you are using a touch-enabled device), you will notice that two buttons will appear below: Delete and Enable.
The Delete button will remove the entry from the list and make Redirection ‘forget’ that there ever existed a Delete Me page that was then removed. You can always create a new redirection rule by hand but I recommend that you don’t remove entries from this list unless you really intend to ignore the issue and not create a redirection.
The Enable button activates the redirection and allows you to make changes to the default behavior. Unless you don’t change it, the redirection will point towards the home page of your website. More often than not, this will be a mistake.
At best, such redirection behavior isn’t optimal. It makes much more sense to redirect the URL to a similar blog post or Woocommerce product; or at least towards a similar blog or product category.
Redirecting a customer to your home page confuses both the customer (who was expecting to land on a specific page) and Google (because there is a difference between the indexed contents for the original URL and the new page).
That is why I will first click on Enable and then hover/tap over the same area and select Edit to change the redirection target:
You then need to perform a couple of operations in the dialog panel that opens. The first one is to click on the cogwheel to the right of the Save and Cancel buttons in order to open the Advanced view for the redirection. Then you can start typing the name or slug of an existing page (or type a completely different URL, it could even direct to a different website).
In our case, I want to redirect the Delete Me page to the Save Me page. Notice how the plugin recognizes the page as soon as I start typing its name.
The other thing you need to consider is the type of redirection to choose:
Although Redirection allows you to choose between 6 different redirect codes, in practice your choice should be between 301 – Moved Permanently and 307 – Temporary Redirect. Which one you will choose depends on how you want to represent the redirection to search engines.
When to Use 301 – Moved Permanently
301 – Moved Permanently instructs the search engine to update its index and pass the SEO ‘link juice’ from the old URL to the new one. This is the one to choose in case the old URL is invalidated due to a change in the slug for SEO purposes or a change of the canonical URL due to relocation of the page in a different category. In other words, this is the setting to choose when the new target URL is a good substitute for the old URL.
When to Use 307 – Temporary Redirect
307 – Temporary Redirect, as the status name implies, is a temporary redirection. It signals search engines to keep the old URL in their index and does not pass link authority from the old URL to the new one. You should use this type of redirection sparingly.
The rule of thumb I use stipulates that using a 307 redirect to a different page is almost certainly guaranteed to be better for visitors than sending them to the home page. Putting that 307 in place will serve as a temporary measure until you write a new, more relevant post or decide to ‘kill’ the page by manually redirecting it to the site 404 page.
Do Not Forget to Clear the Site Cache
Once you choose a redirection type and save it, there is one more important step to take: clear your site cache. Chances are that the caching plugin you are using is configured to cache missing pages as well, and if you or another visitor has already requested the Delete Me page after it was deleted but before the redirection was configured, other visitors would be served a cached copy of the 404 page.
Final Thoughts on Redirection
So far I only touched on what Redirection can do. This is a very mighty plugin that comes with a ton of other helpful options. Going through all of them is beyond the scope of this article as it concerns itself with redirects only, but when you have some time to spare, I invite you to review the documentation and see how to take full advantage of it.
If you don’t want to burden yourself with too much information, it’s OK. The plugin will do its stuff and you only need to remember to go to the control panel after you’ve made a URL-breaking change to a post or page or have deleted some content and enable a 301 or 307 redirection from the old URL to a new one.
Monitor for 404 ‘Not found’ Errors
By configuring Redirection as described in the previous sections, you have created an easy-to-use solution to monitor for 404 errors from incoming visitors. However, there is a different group of visitors that you should pay attention to: search engine crawler bots. These have specific behavior and can do more ‘damage’ over the long term.
Unlike real-life visitors who visit your website irregularly following either bookmarks, search engine result pages, or social media posts, crawlers like Google Bot visit websites more or less regularly, and usually attempt to load multiple pages per website visit.
During the brief periods search engine crawlers are doing their job, even a tiny intermittent website error can prove fatal. Potentially dozens or hundreds of previously crawled URLs can be flagged as ‘not found’ and could quickly be dropped from the search engine index.
Even if you can’t prevent this from happening, you need to know Google Bot is getting errors so that you can figure out the reason and fix it immediately. To do this, you need to register your website with Google Search Console.
Once you do that, you will begin receiving notifications about any problems with the indexation of your website, including 404 errors. If you react on these quickly (within 24-48 hours) you should be able to keep the affected pages in the Google search index.
Hint: while on the subject of website monitoring and Google tools integration, you may consider also adding Google Site Kit to your website. This plugin allows you to monitor data from Google Search Console, Google Analytics, Google PSI and AdWords.
The final suggestion I have is to consider testing proactively for 404 errors via a third-party tool such as ScreamingFrog SEO Spider, Ahrefs Site Explorer or SEO Power Suite Website Auditor.
ScreamingFrog SEO Spider operates a lot like Google Bot — it follows every local link and crawls every local page it can find. The difference with Google is that it doesn’t have a crawl budget (a limit on the number of pages it scans at one session) and can be configured to crawl the complete website within a single pass.
Crawling is a server-intense process that might reveal issues with your host which may otherwise stay unnoticed for a long time and appear at the worst possible moment (for example, when Google Bot is crawling your site…) If you see a lot of timeout errors while crawling your website with ScreamingFrog SEO Spider, you should consider increasing your web host resources.
Handling 404 Errors for Non-Existing Content
By doing everything so far (setting up a solution to monitor when existing content has been deleted or has had its URL changed, and making redirects), you have solved probably 90% of all potential cases of 404 ‘not found’ errors.
It is now the time to bring that number up to a full 100.
To do this, we will next concentrate on how actual 404 errors are handled. If everything has been done to avoid such errors from getting into the search engine index, the chances are that such errors will primarily come from human faults like somebody failing to copy/paste a full web page URL.
When such a person visits your website and sees a ‘not found’ error, there is nothing you could have done in advance to prevent it; there is no way to plan ahead for such intermittent, user-caused errors. However, there is still one thing we can do to retain such a confused visitor.
A properly designed, helpful 404 page can do wonders to retain such visitors on site and keep them engaged. The last part of this column is dedicated to how to solve that issue.
How 404.php Works
Out of the box, WordPress does not handle 404 errors itself. For better or worse, the WP Core team decided a long time ago that it should be the job of a WordPress theme developer to provide a 404 page template.
Unfortunately, my experience is that even the best modern themes usually come with some really generic 404 pages. This literally drives me mad; asking people to edit the PHP page templates by hand when we have in our hands powerful page building capabilities in the face of Gutenberg, Elementor and other popular page/theme builders such as WP Bakery Visual Composer, Divi et. al. is simply insane.
In the last part of this column, I will show you how you can design and use a 404 page without the help of additional plugins, and using your favorite page building tool.
I will not be showing you specific design ideas; I will simply show you how to make things work and leave it up to you to come up with the 404 solution that serves your visitors’ needs best.
We are almost finished. Let’s do this!
How to Create and Use a Custom 404 page using Gutenberg or your Preferred Page Builder
To make a custom 404 page work, you need to do two things:
(1) Create a 404 template for your existing theme, or modify the existing 404 template so that it redirects internally to the custom 404 page that you will create;
(2) Design a page (that is correct, a normal WordPress page) that your visitors will see when they experience a 404 error.
Create or Modify the 404.php Theme Template
Using the WPX File Manager or your preferred FTP Client, go inside your website root folder and open
wp-content → themes → {your theme or theme child folder}
Once inside, look for a file called 404.php. This is the file that our ancestors used to modify by hand during the Stone Age. If such a file exists, rename it to something else, like 404.php.original; if it doesn’t exist, make a new one, and open it for editing.
Inside that file, place the following lines of code:
<?php
header("HTTP/1.1 404 Not Found");
header("Location: " . bloginfo('wpurl') . 'page-not-found/' );
exit();
Note: in the above example the assumed slug for our custom 404 page is /page-not-found You can choose a different slug, but you must put that one in the code. Save the file and close it.
What this small PHP snippet does is send the user’s browser a 404 ‘not found’ code and an instruction to redirect the visitor to a different web page, whose address is constructed by getting the website root url address (https://example.com) and adding the custom page slug to it (https://example.com/page-not-found).
Create and Design Your Custom ‘Page Not Found’ Page
This is the part of the task I leave entirely to you. I don’t want to leave you hanging at the very end, so here is the simple design I created using the Gutenberg editor. Remember: I was aiming for functionality, not for looks.
If you do a Google search for ‘best examples of 404 pages’ you will find many large collections of 404 pages. Go through a few of them and you will soon realize that they aren’t actually doing a very good job.
Most of the designs you will discover are beautifully made but still act as brick walls: they apologize to the visitor for the error but offer them no way to move forward. The only choice a visitor has is to go back or close the page. Aren’t both choices really bad?
I’ve never been able to appreciate a design purely by its aesthetics. To me, a good design is functional and prompts interaction. By the same logic, a good 404 should not leave your visitors wondering what to do next. It should provide a way to move forward.
The whole purpose of this article is to show you — my fellow WordPress site maker — how to take advantage of the very same tools you are using to build other parts of your site and create a beautiful and functional 404 ‘not found’ error handling page that makes sense and helps your visitors move forward.
Recap
By following my advice in this column, you will have ensured you’re fully proof against both types of 404 ‘not found’ errors caused by trying to open relocated/deleted content linked from search engines and other websites, and handling mistyped URLs.
- You installed a redirection plugin that logs address-breaking changes to various WordPress post types, and know how to create a redirect as soon as you relocate or retire a post/page/product.
- You also set up Google Search Console and linked it to Google Site Kit so that you will know immediately if the pages indexed by Google start returning 404 errors for whatever reason.
- You also know how to proactively scan your website for 404 ‘not found’ errors so that you can respond and fix things even before Google suspects there is something wrong with your website.
- And, finally, you improved your retention rates for visitor-caused errors and hopefully managed to retain these visitors and guide them toward other parts of your website instead of making them leave.
Congratulations! You have made an important step towards making your website more user-friendly and useful, and have brought down to a minimum the chance for a user to reach a ‘not found’ resource.
