Register

If this is your first visit, please click the Sign Up now button to begin the process of creating your account so you can begin posting on our forums! The Sign Up process will only take up about a minute of two of your time.

Results 1 to 6 of 6
  1. #1
    Junior Member
    Join Date
    Feb 2009
    Posts
    18
    Member #
    18405
    Hi I have a major problem. I need to get some information off a web site. I have been using cURL to post the info and get the web page returned. However the web page just sends the default page.

    When I do it manually in my browser it works fine. Is there some way to capture the POST data being sent from my browser?

  2.  

  3. #2
    WDF Staff mlseim's Avatar
    Join Date
    Apr 2004
    Location
    Cottage Grove, Minnesota
    Posts
    7,720
    Member #
    5580
    Liked
    718 times
    Can you show us the script?


  4. #3
    Junior Member
    Join Date
    Feb 2009
    Posts
    18
    Member #
    18405
    <?php

    $url = "http://www.wunderground.com/calculators/solar.html";
    $useragent = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1';
    $referer ="http://www.google.com";


    $ch = curl_init();

    curl_setopt($ch, CURLOPT_URL,$url);
    curl_setopt($ch, CURLOPT_FAILONERROR, 1);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
    curl_setopt($ch, CURLOPT_POST, 1);

    curl_setopt($ch, CURLOPT_POSTFIELDS, "solar_lat=41.0877&solar_lon=-73.7768action=submit");
    //"url=index%3Dbooks&field-keywords=PHP+MYSQL"
    curl_setopt($ch,CURLOPT_USERAGENT,$useragent);
    curl_setopt($ch,CURLOPT_REFERER,$referer);

    $result = curl_exec($ch); // run the whole process
    curl_close($ch);

    $myFile = "testFile.html";
    $fh = fopen($myFile, 'w') or die("can't open file");

    fwrite($fh, $result);


    fclose($fh);

    --------

    Here is the code on the page that contains the form. I simplified it somewhat.

    <form name="solar_energy_lookup" action="solar.html" method="POST">
    <input type="hidden" name="action" value="submit" />
    <tr>
    <td class="taR nobr b">Lat:</td>
    <td><input type="text" class="inputText" id="solar_lat" name="solar_lat" value="" /><span class="b">&nbsp;-&nbsp;Required</span></td>
    </tr>

    <tr>
    <td class="taR nobr b">Lon:</td>
    <td><input type="text" class="inputText" id="solar_lon" name="solar_lon" value="" /><span class="b">&nbsp;-&nbsp;Required</span></td>
    </tr>




    <tr><td class="taR nobr b">Give raw insolation data:<br /></td>
    <td class="taL"><input type="checkbox" id="insolation_only" name="insolation_only" onClick="check_insolation_only();" /><br />
    (Check this box if you would like the amount
    of energy per m<sup>2</sup> per day for your
    location to do your own calculations.)</td></tr>


    <a class="buttonO" id="solar_form_submit" onclick="submit_solar_form();">Submit</a></span>
    </td>
    </tr>
    </form>


    ------------------------------------------------------------------
    Can you tell me what I am doing wrong? When I open the saved page it just shows me the default page.

    Something I just thought of is there is more then one form on the page. How do I know I am accessing the correct form? The script from the form I am interested in refers back to the same page.

  5. #4
    WDF Staff mlseim's Avatar
    Join Date
    Apr 2004
    Location
    Cottage Grove, Minnesota
    Posts
    7,720
    Member #
    5580
    Liked
    718 times
    I'm not a CURL expert, but I don't think what you are trying to do is possible.

    You can POST to a 3rd party website, and you can use CURL to read a 3rd
    party HTML, XML, etc. BUT ... I don't think you can POST and then READ the
    results from the action of the POST.

    The only way it can happen is if the 3rd party site serves-back to you ...
    as with an API. But I don't think Weather Underground has an API for that script.

    Maybe someone else can prove me wrong?


  6. #5
    Junior Member
    Join Date
    Feb 2009
    Posts
    18
    Member #
    18405
    Well if the browser can get the info there's no reason a script couldn't get it either. There's theoretically no way for a server to know a browser from a script -- barring some some random test to see if a real live person is at the browser. Which there isn't one. There's a way to do it I just don't know how.

  7. #6
    WDF Staff mlseim's Avatar
    Join Date
    Apr 2004
    Location
    Cottage Grove, Minnesota
    Posts
    7,720
    Member #
    5580
    Liked
    718 times
    Like I said,
    You can POST to a 3rd party.
    You can READ an XML file or HTML file or whatever file from a 3rd party.

    It's POSTING and READING the resulting page that has me hung up.
    Without an API (on the 3rd party's end), I don't know who you can do it.



Remove Ads

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
All times are GMT -6. The time now is 08:43 AM.
Powered by vBulletin® Version 4.2.3
Copyright © 2019 vBulletin Solutions, Inc. All rights reserved.
vBulletin Skin By: PurevB.com