300% site indexability increase in 1 year. Preply.com. Case study

Search Engine Optimization - SEO

Updated on Oct 17, 2019

View more like this | Visit MONTCLAIR, NJ | Contact JetOctopus
How the global tutor platform Preply.com achieved an increase of
indexable pages from 20% to 60% by optimizing their internal linking structure
What’s inside:



1. Short introduction.
2. Pages with noindex tag. Why links to these pages harm your crawl budget.
3. Number of internal links impacts page’s visibility. Confirmation of hypothesis.
4. Key takeaways.




The story behind:
When we embark on technical SEO, we usually set internal linking optimization as a high priority task. There are a number of articles about building and analyzing internal linking structure. Each SEO influencer should create a few related pieces of content on this matter (for instance, we conducted a webinar.)
Image for 300% site indexability increase in 1 year. Preply.com. Case study with ID of: 3806203
The main questions after exploring such content are, “How can my website benefit from these recommendations? What results will I get after implementing changes in my links - moderate optimization, or measurable improvement?”

In general, each task in search optimization is aimed at increasing the number of profitable pages in Google’s index while eliminating non-profitable indexable pages. In that way search bots will show your profitable pages that meet users' needs. We increase traffic and conversions, whereas users get value.

However, if your website contains 1M+ pages (there 1.5M URLs on Preply.com) it’s impossible to get all your content indexed - search bots don’t want to spend so many resources on a single website.

That's the real challenge for SEOs - to increase the crawl budget for useful pages as much as possible. For this objective we have chosen a data-driven approach: formed hypotheses, made monitoring segments of URLs and conducted experiments. Only then we implemented best practices on the website as a whole. To hasten your way to an SEO boost, we are pleased to share our most profitable insights.



Links to pages with noindex tags harm your crawl budget.
Here is why
There are some pages that shouldn’t be indexed on our case study website. We can hide them from the search bots with noindex tags.

In practice, to commit this directive, firstly the bot needs to crawl the page, see the noindex tag and only then it drops that page entirely from search results. In this way, the bot wastes a portion of the crawl budget on non-profitable pages. That’s why the main priority was to find and eliminate links that point to pages with noindex.
In Indexation section we click on Blocked by Meta Tag to go to the list of blocked pages with 200 HTTP response:
Image for 300% site indexability increase in 1 year. Preply.com. Case study with ID of: 3806215
In the Data table we add the filter "Count of internal links from indexable pages > Greater than 0" to see only those blocked pages to which at least 1 link points:
Image for 300% site indexability increase in 1 year. Preply.com. Case study with ID of: 3806223
Add "In Links from Idx. Pages" column to the data table to see the total number of links from indexable pages:
Image for 300% site indexability increase in 1 year. Preply.com. Case study with ID of: 3806225
So, we created the table with blocked pages by noindex tag and also URLs where links to these pages were found. We eliminated those links, and as a result, the crawl budget was increased:
Image for 300% site indexability increase in 1 year. Preply.com. Case study with ID of: 3806237
Dynamic of not-indexed pages over 5 months

Source: analytics of preply.com​




Correlation between internal links and SERP performance
The hypothesis is as follows: the more internal links point to a given page, the more times this page will appear in SERP.

If this is true, we can improve the performance of profitable pages by inserting more relevant links and, conversely, reduce the number of links pointing to second-tier pages. To prove the hypothesis, we took the following steps.

Add pages that got at least one impression in the SERPs from Google Search Console to the data table:
Image for 300% site indexability increase in 1 year. Preply.com. Case study with ID of: 3806254
Calculate the number of internal links on these pages with impressions:
Image for 300% site indexability increase in 1 year. Preply.com. Case study with ID of: 3806271
Calculate the number of links with pages with no impressions:
Image for 300% site indexability increase in 1 year. Preply.com. Case study with ID of: 3806261
Data we got from this analysis revealed where SEO opportunities lie. We increased the number of relevant links on the most profitable pages, and as a result, the number of indexable pages increased from 20% to 60%.
Image for 300% site indexability increase in 1 year. Preply.com. Case study with ID of: 3806285
Dynamics of pages in Google’s index. Source: analytics of preply.com​




Key takeaways
1. Bots waste crawl budget on pages with noindex tag. Eliminate links to these URLs.
2. Interlinking structure is important for bots. Implement more relevant links on profitable pages to increase indexability of those pages.

SEO is a demanding and ongoing process. Tactics that worked well two years ago might not work well now. That’s why it’s crucial to conduct experiments, measure the results and implement the best practices across the whole website.


goodideazs, LLC is not affiliated with the authors of this post nor is it responsible for its content, the accuracy and authenticity of which should be independently verified.