Find Jobs
Hire Freelancers

Website Scraper and Import Tool

$250-750 USD

Cancelled
Posted over 5 years ago

$250-750 USD

Paid on delivery
Dear Coders, IMPORTANT: Please do not use placeholder bids and respond only with specific questions about my project. If your bid exceeds our limit, then do not bid. We have a csv file containing several thousand tool model numbers. We want you to build a scraper that we can periodically use to scrape the individual URL for each product from 14 retailers and append the csv file with the URL's for each product. If the product is not available on one of the source sites after attempting various combinations for the product model, then simply skip. The 14 sites will be fixed but we'll need to be able to omit one or more of the retailer sites from collecting the data if necessary...a simple checkbox option next to the 14 retailers is all we'll require. That is...by default they are all checked and hence all scraped for the product URL however if one or more is unchecked, they will be excluded from the scraping process and hence results. You'll need to factor in how to prevent the scraper from being blocked. For this, we'll provide a set of proxies - we'll need to be able to either add a list proxies or add a URL for rotating proxies. I would also like you to build some functionality that reports or logs when proxies are failing. Preferably, you should be able to offer advice as to the most suitable proxy provider. Most importantly, do not build for one type of proxy but give options. It is important to note that some of the supplier sites below use CloudFlare making it difficult to scrape. You'll need to find a way around this. We've included a sample csv file with product examples. So far for testing purposes, I have provided a list of about 60 items. In the final csv, there will be many thousands of products to scrape. Your scraper must run fast. The structure of the input csv is as follows: Product Title, Model Number, Brand, Category, Sub-Category, Product Image The 14 retailers are listed here: 1) Total Tools - [login to view URL] 2) Just Tools - [login to view URL] 3) Vek Tools - [login to view URL] 4) Get Tools Direct - [login to view URL] 5) Power Tools Specialist - [login to view URL] 6) Bunnings - [login to view URL] 7) Discount Trader - [login to view URL] 8) Tool Mart - [login to view URL] 9) Melbourne Tools - [login to view URL] 10) Trade Tools - [login to view URL] 11) Australian Industrial Supplies - [login to view URL] 12) C L Tool Centre - [login to view URL] 13) Gasweld - [login to view URL] 14) Mitre 10 - [login to view URL] My suggestion is to use the model Number in the search bar of the retailers below to find a suitable match for the product. If you have a better solution please advise. Please be aware that should the first search for the item fail, you may need to use an alternate product number combination. For example, if you can't find D26500K-XE, then you should try D26500K(XE) or D26500KXE or something similar. It will be your job to determine the best way to find the products taking into consideration the unique way each site represents it's products. Next, you'll need to modify a price comparison plugin we have built to allow it to import this csv data into the database. We will provide access to a copy of the site for you to review the code after acceptance of the project. Time Frame: 2 weeks absolute maximum. Any questions, please advise. Simon
Project ID: 17537050

About the project

30 proposals
Remote project
Active 6 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
30 freelancers are bidding on average $646 USD for this job
User Avatar
Hello Simon, This is Vaishali from "Hire WordPress Experts" and I am here to help you with Website Scraper and Import Tool . We have gone through the information provided by you. and I can assure you that we are expert with HTML, Javascript, PHP, Software Architecture, Web Scraping and I will make your project a reality. This is how we plan to do your task: Surely, We are ready to help you with build a scraper which you will periodically use to scrape the individual URL for each product from 14 retailers and append the csv file with the URL's for each product. Yes, We have understood the structure . Yes, we will modify a price comparison plugin. We will need these things from you: We have understood the requirements. Please provide us the details of site. Yes, we are ready to start the work now and complete within 2 weeks . Let discus little more? Rest be assured you. We will offer you the best. I can promise following 1. You will have exceptional time working 2. We do not hesitate if specifications change for good. 3. The project will be completed in far less than 25 days and you receive free support for up to 25 days. 4. We will be available on sky-pe 5. We do not take any upfront payments Please review our profile and past work here: https://www.freelancer.com/u/OutsourceMan Please select us right away, so that we can get started without wasting a day! Thanks P.S. Be assured you will have a nice time. Vaishali Business Analyst
$750 USD in 10 days
4.9 (591 reviews)
8.9
8.9
User Avatar
Dear sir, This is Lin, i did more than 380+ scraping project, please view my profile page here https://www.freelancer.com/u/mantislin.html then you will know. Can we discuss more details about your project? then i can do example data or script for you, we can continue after you checked example data. Thanks, Lin
$547 USD in 6 days
5.0 (579 reviews)
8.4
8.4
User Avatar
Hello, Dear How are you? I am very interested in your project and am ready for starting your project for now. I have experienced in developing HTML, Javascript, PHP, Software Architecture, Web Scraping. I will work very hard and best for you. Best Regards
$555 USD in 10 days
5.0 (127 reviews)
8.5
8.5
User Avatar
Hi I understand your requirements and we can utilize proxies to avoid the blocking. I bid each site with 125 USD Can work on a demo before you award me. Can finish it in 5-7 days Thanks
$1,750 USD in 7 days
5.0 (230 reviews)
8.0
8.0
User Avatar
Hello, I can help with you in your project Website Scraper and Import Tool I have more than 5 years of experience in HTML, Javascript, PHP, Software Architecture, Web Scraping. We have worked on several similar projects before! We have worked on 380+ Projects. Please check the profile reviews. I can deliver your job with in your deadline. Please ping me for more discussion. I can assure the 100% job satisfaction. Thanks,
$750 USD in 5 days
4.9 (215 reviews)
7.7
7.7
User Avatar
Dear Sir, I'm very much delighted to let you know that i did data scraping with PHP-cURL, PhantomJS, Node.js, Selenium from many sites. I just scraped the data from web site and then wrote the data in mysql database or excel or csv or xml file. I worked on many similar projects, I have big experience in data mining projects. I have written hundreds of web scrapers which scrape millions of pages each day. I'm ready to fulfill your requirement. I can finish this task in short time, with the best quality. I can assure 100% accuracy. Please give me the opportunity to do the work. With Kind Regards, Debdulal Roy Proshanta
$500 USD in 15 days
4.9 (107 reviews)
7.6
7.6
User Avatar
Hello sir, please reply soon , i will start work right now and complete all soon, please share website details, thanks
$250 USD in 0 day
4.8 (453 reviews)
7.4
7.4
User Avatar
Dear scm1971, I do have relative experience achieving something like yours "Website Scraper and Import Tool" thusly I am expecting discuss and progress. Relevant Skills and Experience I have 10+ years of experience in HTML, Javascript, PHP, Software Architecture, Web Scraping I can do this within your budget too $250 - $750 USD if its all the same to you check more from here https://www.freelancer.com/u/mmadi.html?page=portfolio I am set up to work with you,kindly sitting tight for your response. Thankful and Regards, Mmadi
$525 USD in 12 days
5.0 (51 reviews)
7.1
7.1
User Avatar
--^^-- Hello! ***************************** Nice to meet you! ***************************** I am a Web Scrap Professional developer. Scrapping and importing tools. I can meet your requirements fully in a short time. I have the skills for that. C/C++, Android, IPhone, Bootstrap, MySQL, node.js, Python, Vue.js I will do the best for your project. I will succeed your project with no condition. Thanks, regards!
$555 USD in 10 days
4.9 (109 reviews)
7.3
7.3
User Avatar
Hello. My name is Maciej. I am a Python developer and web scraping expert. I can create the scraper matching requirements for the price bidded. Please contact me if you are interested. Kind Regards Maciej J
$1,888 USD in 20 days
4.9 (130 reviews)
7.0
7.0
User Avatar
Hello Sir, I have read your job description regarding Website Scraper and Import Tool. I have 6+ year of experience in HTML, Javascript, PHP, Software Architecture, Web Scraping. I have strong expertise to accomplish this project in decided time frame. Lets discuss in detail so that we never skip basic thing during project implementation. Thanks & Regards
$555 USD in 10 days
4.7 (193 reviews)
6.6
6.6
User Avatar
Hello, I can scrape all the 14 retailer websites and get the date for you CSV file. As you said you will provide a set of proxies, I can rotate the proxies and use it with those websites which blocks us. There are several method we can adopt to scrape so as I said , we can scrape all the websites. Let me know if you are interested. Thanks, Amit
$500 USD in 5 days
5.0 (80 reviews)
6.1
6.1
User Avatar
Hi there, I can do it. I have done many related projects like this. If you are provide this work, it will help my career also. Give me a chance to do this. Waiting for your precious reply. Thank you.
$333 USD in 10 days
4.9 (133 reviews)
6.0
6.0
User Avatar
Hello There, i go through with your all requirement and understand it very well, i make sure i will give you best result, I've experience with scrap websites, i will also build some functionality for reports and logs when proxies failing, i will determine the best way to find the products taking into consideration the unique way each site represents it's products. I've 5+ years of experience in evlopment with PHP, HTML, HTML5, Javascript, Web Scraping, Software Architecture, web design etc. I make sure task will be finished within 2 weeks, i look forward to your response, Thank you. Regards, Arun.
$600 USD in 15 days
4.9 (24 reviews)
5.4
5.4
User Avatar
Hi, Being experienced in working on several data scrapping projects using technologies like .net/selenium/c# based on which I can provide frontend solution which can be used like any software where you can select input csv list and tool will browse all 14 sites you mentioned and get the details you want. I am available to start on the project from now itself and can complete in 10 days of time frame. Waiting for your response for further discussion in detail. thanks.
$277 USD in 10 days
4.7 (50 reviews)
5.4
5.4
User Avatar
am not a random bidder.. Please ping me if you are impressed with my proposal and please let me know any other details.. thank you..
$750 USD in 10 days
5.0 (8 reviews)
4.4
4.4
User Avatar
Website & Mobile App Development Company Vyosys Technologies a leading website design & mobile app development company in India/Singapore, providing complete web designing & mobile app development, SEO and digital marketing, ecommerce website to our global clients. We are highly motivated team and organization with professional skills abides to give “RESULTS” and “CUSTOMER SATISFACTION” rather than “RECOMMENDATION” with adept capabilities.
$500 USD in 10 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hi. Thank you for your time on my profile. I am a professional programmer with about 15 years experience. I am here to work for you, high quality & fast, also on your budget. Server-side Development: - Back-end Development(PHP, Java, Node.js, Express, Koa, Python, Django) - Database Development(MySQL, MongoDB, Redis) Data Mining and Knowledge Discovery: - Data Crawling(Scrapy, Beautiful Soup) - Data Mining - Machine Learning(native, Tensorflow) Software Development: - C/C++ - System Architecture - Embeded System I will be happy to work with you. Thanks!
$500 USD in 10 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Dear Sir, I have 7+ experience in developing my own custom program for each website, I am going to scrap / crawl data from web. I can do any type of Web Scrapping, Data Mining and Web Crawling and I can provide scraped data in any format as required: CSV, Excel, XML, MySql etc. I have scraped data from websites that require parsing JavaScript/AJAX, Secure Login and IP proxy rotation. I use open source technologies like Selenium/JSoup/Phantomjs/HTMLUnit for extraction using Java. I have extracted data from eCommerce, real estate, forums, business listings, sports results data and public directories. My experience matches with your requirement and I am available full time to complete your work soon.
$555 USD in 10 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of AUSTRALIA
Diamond Creek, Australia
5.0
362
Payment method verified
Member since Mar 27, 2005

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759) & Freelancer Online India Private Limited (CIN U93000HR2011FTC043854)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.