Register

If this is your first visit, please click the Sign Up now button to begin the process of creating your account so you can begin posting on our forums! The Sign Up process will only take up about a minute of two of your time.

Results 1 to 10 of 10
Like Tree1Likes
  • 1 Post By integral

Thread: Which is better Robots.txt or Meta Robots Tag?

  1. #1
    Member
    Join Date
    Feb 2017
    Location
    India
    Posts
    34
    Member #
    56118
    Liked
    2 times

    Which is better Robots.txt or Meta Robots Tag?

    Have you any idea which is better and what are the functionality they differ each other.

  2.  

  3. #2
    Junior Member
    Join Date
    May 2017
    Location
    Pune
    Posts
    11
    Member #
    56620
    Meta Robot tag needs to be put in HTML code of every webpage. Whereas robots.txt file is uploaded on root path is applicable to complete domain. From my point of view robots.txt file is better, bcoz it gets applied at one stroke.

  4. #3
    Junior Member Kinopio's Avatar
    Join Date
    May 2017
    Posts
    10
    Member #
    56706
    Meta Robots Tag is much better as it helps in forcing the search engine crawlers not to index and display the hidden pages in server.

  5. #4
    Junior Member
    Join Date
    May 2017
    Location
    Cromford
    Posts
    1
    Member #
    56720
    Liked
    1 times
    Its also easier to stop robots indexing a whole folder with robots.txt with the disallow command. You may have a folders of say pdf downloads you don’t want Google to index in "downloads" so you can Disallow: /downloads/ to stop all documents in that folder being indexed Its also easier to stop robots indexign a whole folder with robots.txt with the disallow command. You may have a folders of pdf downlods you dont; want Google to index in "downloads" so you can disallow /downloads ro stop all docuemtns in that folder being indxed
    ashbifs likes this.

  6. #5
    Junior Member
    Join Date
    Jun 2017
    Posts
    2
    Member #
    56786
    Meta Robots Tag is not necessary because by default Search Engines automatically crawl and index the sites. However, In case of special conditions for specific urls, we need to add robots.txt file to crawl or index limited amount of webpages.

  7. #6
    WHM
    WHM is offline
    Junior Member WHM's Avatar
    Join Date
    Jun 2017
    Posts
    20
    Member #
    56722
    Liked
    1 times
    Robots.txt or Meta RobotsTag both are important. Meta RobotsTag in given on html page whether to follow or no-follow the content. There are other things to be allowed or disallowed which can be defined in robots.txt file.

  8. #7
    Member
    Join Date
    Feb 2018
    Posts
    80
    Member #
    58388
    Liked
    3 times
    Meta Robots Tag is not necessary because by default Search Engines automatically crawl and index the sites

  9. #8
    Junior Member
    Join Date
    May 2018
    Posts
    13
    Member #
    58950
    If you add meta robots to html code, the robot will visit these pages (and as a result, it will be spend some of its crawling budget).
    If you close these pages in the robots.txt file, the robots won't visit those pages at all.
    Robots.txt, as I think, much better. It can be very useful, because the robot may visit for the one time only pages which you forbided to be indexed, and it's not good for your site.

  10. #9
    Junior Member
    Join Date
    May 2018
    Location
    Nagpur
    Posts
    14
    Member #
    59139
    Quote Originally Posted by akash-varma View Post
    Meta Robot tag needs to be put in HTML code of every webpage. Whereas robots.txt file is uploaded on root path is applicable to complete domain. From my point of view robots.txt file is better, bcoz it gets applied at one stroke.
    Ya,You are right..we can follow the webpage or index the webpage with the the help of meta robot tag .this all are included in one file i.e. robot.txt file.thats why i prefer robot.txt

  11. #10
    Member RH-Calvin's Avatar
    Join Date
    May 2015
    Location
    Forum
    Posts
    39
    Member #
    49557
    Liked
    2 times
    Robots.txt is better compared to meta robots tag. It is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.


Remove Ads

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

Tags for this Thread

All times are GMT -6. The time now is 07:41 AM.
Powered by vBulletin® Version 4.2.3
Copyright © 2019 vBulletin Solutions, Inc. All rights reserved.
vBulletin Skin By: PurevB.com