Register

If this is your first visit, please click the Sign Up now button to begin the process of creating your account so you can begin posting on our forums! The Sign Up process will only take up about a minute of two of your time.

Results 1 to 6 of 6
  1. #1
    Senior Member sarab's Avatar
    Join Date
    Feb 2004
    Location
    UK
    Posts
    525
    Member #
    4818
    Liked
    1 times
    Please don't hit me over the head if this is killingly simple (or if I've chosen the wrong section of the forums again :dead but -

    How does one stop search engine bots from crawling and indexing a page that's just up for test purposes?

    (I have done a search for this and similar questions in the forums and couldn't find any threads that seemed to fit).

    Thanks

    Sara
    ...............................................
    My best pal's site: Algarve Beach Life :ichatcool:

  2.  

  3. #2
    Senior Member DanielOliver's Avatar
    Join Date
    Jun 2003
    Location
    Cornwall, England.
    Posts
    1,202
    Member #
    1879
    I read about it once before. Something to do with "robot.txt" google it and it will tell you how to go about it.
    Basically I think the bot will read that file and in the file you tell the bots what not to index. Don't hold me to this, this is what I think I can remember.

  4. #3
    Unpaid WDF Intern TheGAME1264's Avatar
    Join Date
    Dec 2002
    Location
    Not from USA
    Posts
    14,483
    Member #
    425
    Liked
    2783 times
    http://www.robotstxt.org/wc/exclusion.html

    That will pretty much explain everything you need to know, My Darling English Rose.
    If I've helped you out in any way, please pay it forward. My wife and I are walking for Autism Speaks. Please donate, and thanks.

    If someone helped you out, be sure to "Like" their post and/or help them in kind. The "Like" link is on the bottom right of each post, beside the "Share" link.

    My stuff (well, some of it): My bowling alley site | Canadian Postal Code Info (beta)

  5. #4
    WDF Staff Wired's Avatar
    Join Date
    Apr 2003
    Posts
    7,657
    Member #
    1234
    Liked
    139 times
    Yeah, that's a really good site.
    The Rules
    Was another WDF member's post helpful? Click the like button below the post.

    Admin at houseofhelp.com

  6. #5
    Senior Member sarab's Avatar
    Join Date
    Feb 2004
    Location
    UK
    Posts
    525
    Member #
    4818
    Liked
    1 times
    That will pretty much explain everything you need to know, My Darling English Rose.
    You're a wonderful resource, Game, not to mention my knight in shining polyester... :thumbsup:

    And my thanks also to Daniel O. :classic:

    I shall check that out and learn lots, I'm sure.
    ...............................................
    My best pal's site: Algarve Beach Life :ichatcool:

  7. #6
    Unpaid WDF Intern TheGAME1264's Avatar
    Join Date
    Dec 2002
    Location
    Not from USA
    Posts
    14,483
    Member #
    425
    Liked
    2783 times
    Quote Originally Posted by sarab
    You're a wonderful resource, Game, not to mention my knight in shining polyester... :thumbsup:
    It's polycotton now. My new girlfriend is influencing my sense of style (you know...complete slob. )
    If I've helped you out in any way, please pay it forward. My wife and I are walking for Autism Speaks. Please donate, and thanks.

    If someone helped you out, be sure to "Like" their post and/or help them in kind. The "Like" link is on the bottom right of each post, beside the "Share" link.

    My stuff (well, some of it): My bowling alley site | Canadian Postal Code Info (beta)


Remove Ads

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
All times are GMT -6. The time now is 03:36 PM.
Powered by vBulletin® Version 4.2.3
Copyright © 2019 vBulletin Solutions, Inc. All rights reserved.
vBulletin Skin By: PurevB.com