Register

If this is your first visit, please click the Sign Up now button to begin the process of creating your account so you can begin posting on our forums! The Sign Up process will only take up about a minute of two of your time.

Results 1 to 9 of 9
  1. #1
    Junior Member
    Join Date
    Dec 2013
    Posts
    3
    Member #
    38026

    Is this possible?

    I'd like my website to collect a little information from another website. Basically, there's a number that's always changing each day on website 2. My website would need to be able to read the page from website 2 and grab that number.

    Is this something that's possible? No worry about providing highly technical details on how to accomplish it, I'm really just interested in a simple yes/no type of answer.

    Thanks for your help!

  2.  

  3. #2
    Unpaid WDF Intern TheGAME1264's Avatar
    Join Date
    Dec 2002
    Location
    Not from USA
    Posts
    14,480
    Member #
    425
    Liked
    2784 times
    Since you asked for a yes or no answer only, yes, it's possible. However, make sure that you ask for permission or that you're granted it in some other way first (e.g. a Creative Commons license like the one Wikipedia grants for sharing). Don't just take things from other sites without asking. It's bad from and potential copyright violation.

    I still wasn't technical. I was only legal.
    If I've helped you out in any way, please pay it forward. My wife and I are walking for Autism Speaks. Please donate, and thanks.

    If someone helped you out, be sure to "Like" their post and/or help them in kind. The "Like" link is on the bottom right of each post, beside the "Share" link.

    My stuff (well, some of it): My bowling alley site | Canadian Postal Code Info (beta)

  4. #3
    WDF Staff mlseim's Avatar
    Join Date
    Apr 2004
    Location
    Cottage Grove, Minnesota
    Posts
    7,971
    Member #
    5580
    Liked
    760 times
    Can you show us the website that has the "changing number"? And what number you're trying to scrape?

    "scrape" is the term for getting data from other websites (not your own sites). Yes, it is unethical ... unless ... they offer an RSS feed, API, or some other permission to use the data. If I can see that "changing number", I might be able to determine what to do.

    So the answer to if it's technically possible?, most of the time, is Yes. It can be done. Sometimes the data is so poorly formed (HTML), that it cannot be read.

    And to understand the ethical point ... give me a link to your website and I'll scrape some data from your page(s). I'll scrape that data many times a day, and post it on my site for other people to see and use. I will not link to your site or discuss anything about your site. I just want to use your data for myself.
    .
    .
    .
    I think you might get my point.
    .
    .
    Last edited by mlseim; Dec 23rd, 2013 at 12:00 PM.


  5. #4
    Unpaid WDF Intern TheGAME1264's Avatar
    Join Date
    Dec 2002
    Location
    Not from USA
    Posts
    14,480
    Member #
    425
    Liked
    2784 times
    Depends on the context, Max. I've built several scrapers just in the last six months on a client's behalf and run them all on various sites. I wouldn't say it's unethical to scrape context in and of itself...what's unethical is not acquiring obvious permission first (e.g. API, RSS, permission of the site owners).

    Sometimes there are rules associated with the permission (e.g. don't run the bot during the site owner's business hours, so there are a couple I've had to run on the weekend as a result). So it really is situation-specific.

    Mind you, the smart thing to do is assume you don't have permission until you have it. That way, you're acting ethically regardless. I would suggest that running a bot once a day to acquire a single number probably would be okay on the grounds that 1) it's a bot run once a day and 2) it would presumably only need to hit one page to get that number. But I'm not saying that with certainty.
    If I've helped you out in any way, please pay it forward. My wife and I are walking for Autism Speaks. Please donate, and thanks.

    If someone helped you out, be sure to "Like" their post and/or help them in kind. The "Like" link is on the bottom right of each post, beside the "Share" link.

    My stuff (well, some of it): My bowling alley site | Canadian Postal Code Info (beta)

  6. #5
    Junior Member
    Join Date
    Dec 2013
    Posts
    3
    Member #
    38026
    The number and website in question is your fuel points from the nike+ website. For example: http://nikeplus.nike.com/plus/profile/CraigBerry/
    The number on this profile is currently: 843,679 and could be found using a regular expression search or something similar. There are two places in their html code where it can be found, one with a comma, and one without.

    Scraping that number, I'd just like to use it to show how many new points you have obtained since the last time you updated. It only scrapes from your own personal profile, you wouldn't be able to cause it to scrape from someone elses profile.

  7. #6
    Unpaid WDF Intern TheGAME1264's Avatar
    Join Date
    Dec 2002
    Location
    Not from USA
    Posts
    14,480
    Member #
    425
    Liked
    2784 times
    Then I can all but guarantee you that you won't be able to do what it is that you want without talking to Nike first (and even then, I doubt it).

    1) That site requires a login just to be able to view the profile in question. That's somewhat tricky by itself.
    2) Large companies like Nike tend to notice when things are scraping them and generally frown upon it. Again, you might be able to get away with scraping one number once per day, but if you're planning on building any kind of an application to do what you want and share it with others you're going to put yourself in the NIke crosshairs.

    If all you want to know is how many points a user obtained since (s)he last updated (not sure what that means, by the way), your best bet is simply to ask Nike to build in the feature you want.
    If I've helped you out in any way, please pay it forward. My wife and I are walking for Autism Speaks. Please donate, and thanks.

    If someone helped you out, be sure to "Like" their post and/or help them in kind. The "Like" link is on the bottom right of each post, beside the "Share" link.

    My stuff (well, some of it): My bowling alley site | Canadian Postal Code Info (beta)

  8. #7
    Junior Member
    Join Date
    Dec 2013
    Posts
    3
    Member #
    38026
    Thanks for the information. Looks like I have a lot of things to consider. Considering it would be a web application for people who use their nike+, it should be possible to have them logged into it at the time of browsing, but I agree that is kinda sloppy.

    Asking Nike to make that number available could be the best way to go. Thanks again!

  9. #8
    Unpaid WDF Intern TheGAME1264's Avatar
    Join Date
    Dec 2002
    Location
    Not from USA
    Posts
    14,480
    Member #
    425
    Liked
    2784 times
    If that's the case, you may be able to pull this off without spidering.

    Ask them if you can build an app using the oAuth protocol that will show that information. More and more companies are going the oAuth route so that they can find out who's accessing their stuff from where and what it is that they want. Google does it, Twitter does it, Yahoo! does it, Hotmail does it...there are a bunch of others, but none come to mind right now.

    Alternatively, as Max said if they have an API that would work as well.
    If I've helped you out in any way, please pay it forward. My wife and I are walking for Autism Speaks. Please donate, and thanks.

    If someone helped you out, be sure to "Like" their post and/or help them in kind. The "Like" link is on the bottom right of each post, beside the "Share" link.

    My stuff (well, some of it): My bowling alley site | Canadian Postal Code Info (beta)

  10. #9
    WDF Staff mlseim's Avatar
    Join Date
    Apr 2004
    Location
    Cottage Grove, Minnesota
    Posts
    7,971
    Member #
    5580
    Liked
    760 times
    Does Nike happen to have an app to display your fuel? An app is programmed in a "native" language to the device (iphone, android, etc), but if they offer that with an app, it's likely they can offer that to an "online app" (your website script). They would have to create (or already have) an API with a unique key assigned to each fuel account. A PHP script can access the API using the key to get that fuel number.

    For sure it won't hurt to ask them about it.



Remove Ads

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
All times are GMT -6. The time now is 10:42 AM.
Powered by vBulletin® Version 4.2.3
Copyright © 2021 vBulletin Solutions, Inc. All rights reserved.
vBulletin Skin By: PurevB.com