What is expired content?
Expired content pages are which no longer serve the content which it was previously hosting. It is like an empty shelf in a supermarket. There are multiple reasons such a thing could happen on your website.
Importance of dealing with expired content right
There are 2 problems your site might run into if it does not handle expired content right.
- Bad User Experience – A user lands on a page from search engine expecting his answers solved. When she sees a page which doesn’t serve that content it might lead to frustration. The user might be lost while browsing your site and might never find the product she is looking for. This can lead to short click or a loss of brand reputation.
- Lack of efficient crawling – Search engines crawl from one page to another looking for fresh content. When search engines repeatedly hit 4XX pages or expired content pages which offer no value you are depleting your site’s assigned crawl budget. This loss of crawl efficiency can cause few of your other pages not get crawled and indexed.
How to check if you have handled expired content right?
Handling expired content should not be a redundant SEO task. It is a set of instructions you code on your server to make it both friendly for users and search engines. It is a set of practices your team follows when your site runs into such situations.
Situation 1: E-commerce Site which has products out of stock
If you are running an e-commerce store chances are that few of your products are temporarily out of stock. Although these pages serve content to the search engines. Users might feel frustrated if they see that the product they were looking for is unavailable. This is a kind of “temporary expired content” and needs to be dealt with from both search engine and user perspective.
Collect email from interested buyers – Collect email IDs of users who want to know when the product will be back in stock. This way you are still keeping the buyer interested and you have a way of reaching out to him when the product is back in stock.
Inform users about other relevant products – There might be other products which are highly relevant to the products which are out of stock. You have an overlay or a banner mentioning the same. Interested users will have an option to continue shopping with you and might end up buying your products.
Use a temporary Redirect – A 302 redirect to the most relevant product in the category or the parent category will give users a non-obstructive way of discovering other relevant products. This should be implemented only when you are certain that the user wouldn’t feel that she has landed on a completely different page.
Situation 2: E-commerce Site which discontinues products
When products are discontinued your e-commerce site might show up a 404 page or even worse show a 404 page to the user and an HTTP 200 code to the search engine. Since the page or product is discontinued permanently this is straightforward expired content.
Solution for small and medium websites
If there is a relevant or alternative product – In this case, the best approach is to 301 redirect the product URL to the alternative product.
If there is no alternative product – In this case, you should redirect the product to its parent category. In other words, the subcategory which it belongs to.
Solution for large websites
Since there might be a massive number of products which get discontinued redirects might not be the best approach. However, you can make this SEO and user-friendly by implementing –
Show the user other options – Mention that you have discontinued the product and no longer plan to sell it. Also, give the user a list of other relevant product options to choose from.
Unavailable_after Tag – If you have a website which discontinues hundreds of products every month like then use an unavailable after meta or robots tag on each of those pages. X-Robots-Tag: unavailable_after: “Event End Date” “Event End Time” “Time Standard” Example: X-Robots-Tag: unavailable_after: 1-Oct-2018 15:00:00 IST. You can implement the same using the meta tag in the <head/> section of the page. More on this tag
Situation 3: Business pulling their profile out of a classifieds website
This kind of expired content is applicable for websites which lists people and businesses publically. When a profile owner decides to delete his or her own profile it might lead to an expired content. If your data team realizes a particular page is unverified or a duplicate profile it might bring a page down. The problem is such actions are not usually performed with SEO in mind. Sometimes these pages hardly show any content but throw a 200 OK page instead of a 404 page.
- If you have launched profiles or listings on your site which are not meant to be then you might want to remove them. This expired content can be dealt easily using a simple “merge and redirect” function. To do this first you must implement a code which merges all data in duplicate profiles or listings under a single profile. Then you must 301 redirect all unverified or duplicate profiles and listings to this original URL.
- However, for large websites, the number of redirects can be very high. This might not be ideal for your performance. In such cases, you can simply remove the URL and serve a 410 HTTP status code. This tells search engines that the page has been removed on purpose and can be considered as a page which is never going to be back up.
Situation 4: Paginated URLs with no content
In certain cases, the number of products or pages under a category might reduce for various reasons. So a category which lists products under a paginated series of 10 might be listing now under 8 pages. This results in the last two pages in the series which is page 9 and 10 do not have any content to serve. This is a subtle way expired content shows up on your site.
There are 2 strategies you can apply in such cases.
- One is to detect such pages in the paginated series via code and throw a 404 page instead of a 200 OK page. This approach should only be employed when you don’t actually know when you will add more products to your categories.
- The other approach is to use a 302 redirect pointing towards the new last page in the paginated series. This should be applied if you are sure that you will be adding more products to the category in the near future. The problem with this approach is the time it takes to implement is much more and the fact that you have to revoke the redirect once the paginated series is back.
Situation 5: Pages or content which is pulled down
You might decide to pull down content from your site. It might be outdated content or content which you no longer appropriate for your brand. Such situations are best dealt with a 410.
If page or content is gone permanently use a 410 code.
The page doesn’t exist temporarily use a 404 code.
Situation 6: Expired Event Pages
If you list events or webinars on your website those pages have a temporary value because no user wants to be on it after the event is over. Since it offers no value to the user you don’t want users to land on it. You would rather direct him to upcoming events. This is a subtle way expired content shows up on your site.
This kind of expired content is harmless from SEO point of view. However, you don’t want these URLs to compete against your upcoming events. You also want your users to land on your upcoming events pages.
To handle this simply let the page be without doing anything in particular. To make it better for the user you can add a notification above the main content suggesting users check out other upcoming events by the organizer.
How to find expired content using DeepCrawl?
Although there is no straightforward way of finding expired content on your site. There is a possibility you find expired content in the following sections in your deep crawl dashboard.
Indexation > Non-200 Status – Check for 4XX pages
- Do any of these pages need to be redirected instead?
- Should any of these URL throw a 410 instead of a 404 code?
- Are any of these URLs not expired content and yet showing a 4XX code?
Content > Body Content > Empty Pages
Find list of pages which offer very little content. These could be a list of soft 404s which don’t serve any content.
Indexable Pages > Duplicate Pages
Check for pagination URLs in this list. Paginated series which are duplicates of each other are usually expired content showing up as duplicate content.