Table of Contents
- Introduction
- How Google Finds Websites To Rank in the SERPs
- How to Check Whether or Not Your Website is Already Indexed
- Getting Web Pages Indexed By Requesting A Crawl
- Getting Website Submitting A Sitemap
- Conclusion
Introduction
Google and other search engines’ ability to send targeted organic traffic to websites makes them one of the highest quality sources of traffic on the web.
That’s perhaps why search engine optimization is one of the most popular marketing activities undertaken by business websites. However, before you can start optimizing your website to secure higher positions in the search engine result pages (SERPs), you must first ensure that your website is “indexed” by Google.
In simple terms, indexing means getting your website listed in the Google database. After this, Google’s algorithms can decide what keywords your website should be ranked for, and based on their 200+ ranking factors, decide the rank of your website for various relevant keywords.
Let’s learn a bit more about how indexing works.
How Google Finds Websites To Rank in the SERPs
Google’s algorithm powered crawling bot, known as the GoogleBot is used to crawl billions of web pages every day. GoogleBot begins by crawling through the list of URLs generated by websites covered in previous crawls and then adds to the list from the sitemap data it pulls from Google Search Console. When this “crawl” is being conducted, GoogleBot discovers new web pages and websites and updates them to its list.
Once this crawling process is complete, the data is fed into Google’s index and new web pages and websites are put into the correct categories.
Then, when a user conducts a search, Google’s algorithms use data from web page elements like content, meta descriptions, meta titles, alt tags, and many others to find relevant results and display them to the user.
The process described above is automatic and takes place around the clock. This means, there is no real need for you to take manual action to get your website indexed. If it exists on the web, it will be automatically found and indexed by GoogleBot.
However, trusting Google’s algorithms with automatically indexing your website and web pages has a downside, you have to depend on Google’s timeline of getting things done.
What does this timeline look like?
An experiment conducted by Hubspot determined that it takes Google roughly an entire day just to crawl a website. In this case, there was no manual action taken by the owner of the website.
In comparison, it took Google just 14 minutes to crawl a website when a sitemap was submitted by the website owner. This means that if you take the time to manually ensure that your website is being indexed by Google, you can potentially start getting organic traffic from the day you launch your website.
Since crawling is also an automatic process, it is worth your time to check if your website has already been indexed by Google before you start taking manual actions to make it happen.
How to Check Whether or Not Your Website is Already Indexed
To check if your website or all of the pages of your website have been indexed by Google, all you need to do is conduct a Google search with the following search term:
Site:Yourwebsite.com
Replace “yourwebsite” with the actual name of your website.
If your website is indexed, you will see something like this:
If, on the other hand, your website has not yet been indexed, Google will tell you that your search did not return any matches.
If that happens, manual action is needed to get your website indexed with Google quickly or in other words, you will have to “submit your website to Google”. There are two ways to go about this. Let’s explore each one in detail:
1. Submitting Your Website To Google
When you decide to manually submit your website to Google, you have two options- requesting a crawl or submitting a sitemap.
Before we get into how you can execute both the processes, let’s spend a minute understanding the difference between the two methods. This difference lies in the purpose of both methods.
2. Requesting a Crawl
usually makes sense when your website is already indexed and you have updated it with a brand new page. When you want a new page indexed alongside your website, requesting a crawl makes more sense. This method is also sometimes known as “Fetch as Google”.
3. Submitting a Sitemap
On the other hand, submitting a sitemap makes more sense when you have a brand new website that you want indexed with Google. As the name suggests, this process involves you creating a sitemap that helps search engine crawlers better understand what it’s about and then submitting the sitemap to the search engines.
Now that we have understood the difference between the two, let’s look at how each process can be executed. However, there is one thing that you must do before you can start executing any one of the processes- get your website registered with the Google Search Console.
4. Registering With The Google Search Console
Google Search Console is absolutely free to use and allows website owners an intuitive way to control how their website interacts with search engines like Google.
To start the registration process, visit the Search Console website. When you log on to the website you will see a page that looks something like:
On the left-hand side, enter your website URL and click on the “Continue” button. When you do, you will be asked to verify domain ownership. The message will appear in a dialogue box like this:
Click on “START VERIFICATION”.
After this, Google will share a code that you will need to enter into the DNS record of your website. Don’t be discouraged if all this sounds technical and complex, the process itself is very easy to execute.
Simply copy the code and log in to your hosting account using cPanel. Find the option labelled “Advanced DNS Zone Editor” and paste the code as a TXT record on the resulting page.
Once you are done, all you need to do is save the updated settings and wait for a couple of hours for the verification process to be completed on Google’s side.
If, however, you find yourself stuck while executing this process, you can seek help from your hosting provider. Most hosting providers will take care of the verification process on your behalf. Those that don’t will be happy to walk you through the process in real time.
Once your domain ownership has been verified and you have registered with Google Search Console, you can begin the process of getting your website indexed by Google.
Getting Web Pages Indexed By Requesting A Crawl
As mentioned earlier, this process is to be used for instances where you want to get a new page of your website indexed.
Step 1: URL Inspection
To do this, log in to the Google Search Console and select the “URL Inspection” option from the menu on the left hand side of the screen. Then, enter the new URL that you want to get indexed.
Once you hit enter, you will see a pop-up with the title “Retrieving data from Google Index”.
If your URL is not already indexed, Google will not find it, and that means it is time to move on to the next step of this process.
Step 2: Request Indexing
In the same dialogue box that said Google could not find your URL in their index, you will find the option to “Request Indexing” and the bottom right corner.
Simply click on it, sit back, and relax. You have just requested Google to crawl the new page on your website and start showing it in the search results for relevant search queries.
Keep in mind that the process isn’t complete when you have executed both the steps. GoogleBot still needs to crawl your new website page. While this is usually done within a matter of a few hours, it may sometimes take a few days. Once the process is complete, you will be able to see and track the performance and user behaviour data of that page in your Google Search Console interface.
Getting Website Submitting A Sitemap
A sitemap is submitted when you have a new website that you want Google to notice and index.
Submitting a sitemap is also a fairly simple two step process:
Step 1: Generate A Sitemap Report
To execute this step, log in to your Google Search Console account and select the desired web property.
Once you are logged in, open the sidebar menu and find “Index”. Under “Index” you will find “Sitemaps”. Click on “Sitemaps” and the Search Console will generate a report containing all the information about the submitted sitemaps of the web property you selected earlier.
The report may contain specific information like the sitemap type, the current status of the sitemap, the number of URLs recorded through the sitemap, and the date it was last updated.
Step 2: Submit A New Sitemap
In order to submit a new sitemap, you will first need to generate one. While it is absolutely possible to create one manually, I recommend the XML Sitemap tool. XML Sitemaps are one of the most widely used types of sitemaps and generating a sitemap is free through the XML Sitemap Tool (unless your website has more than a few dozen pages).
Once you have your sitemap, you have to go back to the sitemap report generated in step 1. At the bottom of the report, you will find the option to add a new sitemap. When you click on that option, you will be asked to enter the URL of your sitemap.
Simply enter the URL given to you by the XML Sitemap tool and click on “Submit”.
There you go, sitemap submitted.
Keep in mind that even with a sitemap, it may sometimes take Google upto a few days to crawl and index your website.
Conclusion
You Have Indexed Your Website And Still Not Seeing Organic Traffic?
Submitting your website is only half the battle won. Once you have ensured that your website is indexed by Google and other search engines, you must start optimizing it to appear higher in the search results for the search terms relevant to your business.
If that sounds complex, it’s because search engine optimization (SEO) is truly complex. Surprisingly, the complexity is usually not the biggest challenge with SEO, it is the patience required to get results.
While SEO experts do develop that patience, it usually means that their clients have to wait weeks and months before they realise whether or not the SEO expert they have hired will be able to deliver the results they promised.
That’s why you need someone that not only has the required technical and creative expertise, but also the track record to back it up. At Digital Web Solutions, our SEO strategies have generated over 6 million leads for our valued clients- and you can be one of them. Contact us to find out how we can help you put your business on the first page of the SERPs.
FAQs
How Do I Submit A URL to Google?
There are two ways to submit a URL to Google:
- Submit an updated sitemap on Google Console. This tool allows you to monitor and manage your site presence on search engines. You must open an account on Google Search Console and verify your website ownership. Then, you can submit your URL for indexing.
- You can also submit an existing URL sitemap using the Google ping service. It allows you to notify Google's search engine about changes or updates on your website so that Google can crawl and index the new content faster.
How Do I Request Google To Index?
You can request Google to index your website using the Google Console tool.
- Open a Google Search Console account and verify your ownership.
- Once you have logged in and verified your website on the Google Search Console, use the URL inspection tool to request the indexing of a specific webpage.
- After requesting indexing, there will be two options: crawl on this URL and crawl on this URL and its direct links. Choose the option that fits you the most.
- Submit your request to Google Console.
- Google crawlers will now start indexing your URL. You can also check the indexing status on the URL inspection tool.
How Many URLs Can I Submit To Google?
If you submit individual URLs, you can submit 50 URLs per week. However, if you submit URLs with all linked pages, you can submit only 10 per month. Submitting URLs to Google is very important for indexing the web pages. Only after your URLs have been indexed will they appear on search engine result pages when an individual searches for information with relevant keywords.
