Register

If this is your first visit, please click the Sign Up now button to begin the process of creating your account so you can begin posting on our forums! The Sign Up process will only take up about a minute of two of your time.

Results 1 to 6 of 6
  1. #1
    Member eif media's Avatar
    Join Date
    Apr 2011
    Location
    Glossop, UK
    Posts
    73
    Member #
    27461
    Liked
    4 times
    Good morning everyone,

    I'm thinking of designing a search engine and have succumbed to the lure of earning potential affiliate income rather than retaining any integrity in being impartial.

    I am intrigued by XML datafeeds but have a few questions which hopefully some of you might be able to answer:

    1) What is the approximate size of a single feed. I appreciate that this will depend upon the size of the feed producers catalogue so can anyone provide an approximate size per 100 or 1,000 items in a typical feed (URL, ID, Name/Description, Price, Availability, Link...).

    2) When a merchant updates the information for a product will this change occur in my feed also, or will I need to regularly download and update my own feeds?

    Thank you!

  2.  

  3. #2
    WDF Staff mlseim's Avatar
    Join Date
    Apr 2004
    Location
    Cottage Grove, Minnesota
    Posts
    7,715
    Member #
    5580
    Liked
    717 times
    XML files, or feeds are created by a script that queries the database and build the file
    using that data. In most cases, the XML file is updated several times a day depending
    on how current the information needs to be. Some go as far as generating the XML
    files "on-the-fly", which takes more processing time and bandwidth.

    You won't really know how many people are using your XML file, as your file may be accessed
    every time someone visits a homepage or another site that is using your XML file.

    How or why you should provide an XML file is an important thing to consider. You could
    provide a complete catalog in an XML file for other sites to use (showing your products).
    You may choose to only list the 10 latest items, or the 10 best selling items.

    Other sites may use your XML file to display products in a way to compare your prices
    to theirs. That may not be something you wish to provide. In any case, there are no
    limitations on size or number of items ... it all comes down to the purpose for the
    file, and how you wish the information to be used.


  4. #3
    Member eif media's Avatar
    Join Date
    Apr 2011
    Location
    Glossop, UK
    Posts
    73
    Member #
    27461
    Liked
    4 times
    Quote Originally Posted by mlseim, post: 231736
    You won't really know how many people are using your XML file, as your file may be accessed
    every time someone visits a homepage or another site that is using your XML file.
    Sorry, I wasn't more clear - I will be using XML files generated by other sites, such as Amazon for example, in order to populate my search results. I won't be providing products, merely providing a gateway to products on sale by others.

    Obviously I will have more than one XML feed as I want people to see results from multiple sources, but I think the size is going to have an impact upon who I can host with which is why I was wondering what the approximate size per number of records would be.

    I was also wondering how maintenance heavy using XML feeds would be i.e. would I just need to implement the feed once then when a merchant, say Amazon again, changes their prices or adds items to their catalogue the changes are reflected in the results shown on my site, or am I going to have to regularly overwrite each merchants data feed with a fresh XML feed daily/weekly?

    I'm planning to use pricetapestry to build my search engine and manage the incoming feeds if it makes any difference at all.

  5. #4
    WDF Staff mlseim's Avatar
    Join Date
    Apr 2004
    Location
    Cottage Grove, Minnesota
    Posts
    7,715
    Member #
    5580
    Liked
    717 times
    Each time you query (or look at) their XML file, it will be as current as they choose to make it.

    You will be using a PHP script to get all XML data and display it whenever someone
    views your page search results (dynamically), so you don't have to save (overwrite) anything.

    What you're doing will not be easy. It's possible that you will need to use your own server,
    as shared webhosts will have bandwidth, memory limit, process timeout issues.

    You'll have to decide if the investment in equipment and programming is worth the return.
    Hiring a programmer for all of this might be expensive, not to mention the server installation.


  6. #5
    Member eif media's Avatar
    Join Date
    Apr 2011
    Location
    Glossop, UK
    Posts
    73
    Member #
    27461
    Liked
    4 times
    Quote Originally Posted by mlseim, post: 231740
    You will be using a PHP script to get all XML data and display it whenever someone
    views your page search results (dynamically), so you don't have to save (overwrite) anything.

    What you're doing will not be easy. It's possible that you will need to use your own server,
    as shared webhosts will have bandwidth, memory limit, process timeout issues.

    You'll have to decide if the investment in equipment and programming is worth the return.
    Hiring a programmer for all of this might be expensive, not to mention the server installation.
    That's reasurring re: the updates, but requiring my own server and a programmer will undoubtedly make the project uneconomic.

    I was fairly confident I could build the site myself using pricetapestry as an engine - it looks as though it would be easy to implement and manage all of the feeds through there. I could customise the design of the site myself, but setting up a server isn't something I've ever even remotely looked at.

    It's a shame I can't start small with something like this, but limiting the sources for results is going to put visitors off.

    I was getting really excited about this project too. Ho-hum.

  7. #6
    WDF Staff mlseim's Avatar
    Join Date
    Apr 2004
    Location
    Cottage Grove, Minnesota
    Posts
    7,715
    Member #
    5580
    Liked
    717 times
    I have no experience with pricetapestry, so you may want to experiment and find
    out what limitations will be. I see they also have a forum ... ask someone there
    how large of a search area can you use before running into problems.



Remove Ads

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
All times are GMT -6. The time now is 12:45 AM.
Powered by vBulletin® Version 4.2.3
Copyright © 2019 vBulletin Solutions, Inc. All rights reserved.
vBulletin Skin By: PurevB.com