Asynchronous Web Scraping and Data Aggregator


The objective of this project is to aggregate data from various websites related to real estate listings and property info. The results should be aggregated into a single CSV which can be easily viewed in Excel.


1. The program should be configurable to limit results based on price, property type, and city including its surrounding area

2. Individual websites should have ability to be enabled/disabled in the configuration settings

3. Datapoints which are scraped must be normalized and/or converted to the same units of measurement so they can be easily and meaningfully sorted in Excel

4. Furthermore, the program must be able to send asynchronous requests to the websites to speed up execution time

5. All code must be thoroughly commented and easy to understand for future modifications and debugging

6. Code must also be robust to continue working even if minor website layout changes occur

7. Program must also produce meaningful logs to debug and troubleshoot should problems arise in the future

8. Upon completion of the project, an instructional video (screen recording with captions and/or audio narration) and/or detailed instruction manual should be provided to demonstrate the key features and functionality of how the program works as well as the code’s structure

Please see attached document for detailed project description including datapoints and sources to scrape.

Please answer the following questions when submitting a proposal:

1.) Do you have any questions about the job description?

2.) Do you have suggestions to make this project run successfully? How do you plan on improving the execution speed when scrapping thousands of listings from several websites?

3.) What past project or job have you had that is most like this one and why?

4.) Which of the required job skills do you feel you are strongest at?

5.) What programming language, libraries, and frameworks do you plan on using for this project and why do you choose them VS available alternatives?

Skills: BeautifulSoup, Python, Scrapy, Selenium, Web Scraping

See more: concurrent web scraping, efficient web scraping, web scraping blog, advanced web scraping python, python web scraping challenges, python async requests, web scraping tips, python async download, asynchronous web scraping python, python web scraping certificate, python asynchronous programming, how to make web scraping faster, data mining web scraping, scraping data web page excel, php web scraping store data, scraping data web match word, scraping data web pagejob, web scraping data collection website, web scraping data facebook, web scraping data extraction

About the Employer:
( 0 reviews ) Edmonton, Canada

Project ID: #25853299

18 freelancers are bidding on average $669 for this job


Hello. As a web scraping and data mining expert by python and node.js, selenium web driver, I am glad to place the bid on your project. I have experienced LinkedIn profile scraping, amazon products scraping and ticket More

$1000 CAD in 5 days
(32 Reviews)

Hi Kushal. I am a senior web and data scraping script developer with 6+ years of experience. I have worked on several similar projects scraping email address and phone number and specific datas from websites with pytho More

$750 CAD in 5 days
(12 Reviews)

Hi sir, I am a Web & Data Scraping Expert who have career for 7 years over. I am very happy to bid on your job. I have rich experience of scraping for collect data & contact & business information such as email, phone More

$700 CAD in 5 days
(6 Reviews)

Hello there, I have gone through your requirement and understood. I have few things to ask but let me answer first what you asked in description.. 1.) Do you have any questions about the job description? Yes.. Are yo More

$850 CAD in 10 days
(7 Reviews)

Hi I am a very experienced statistician, data scientist and developer. I have worked with data from several companies and have done projects involving high level quantitative analysis and data interpretation skills to More

$1500 CAD in 7 days
(20 Reviews)

Hi there I have 10 years experience in Python and web development. I did lots of projects in web world, develop in most web technologies. 8 years developing Django sites and services. I did lots of projects in scrapy a More

$500 CAD in 7 days
(9 Reviews)

1) Not now 2) Yeah, I am gonna use multi-threading and processing. 3) Please check this previous work example. - Realestate Scraping [login to view URL] 4) More

$500 CAD in 7 days
(6 Reviews)

You can also check out my recent review where I was commended for coding standards in particular. My Repositories: [login to view URL] You can take a look at my repos to check code quality ab More

$650 CAD in 3 days
(9 Reviews)

Hello, sir. How are you? I have just read your project description and I can help you as my rich experiences. I have rich experiences in Web Scraping I am a new freelancer, but I can give you the perfect result you ar More

$500 CAD in 7 days
(2 Reviews)

Hi, Your posting is very interesting. I have 8 years of experience in web development field. I have worked on several similar projects scraping email address and phone number and specific data from websites with pytho More

$500 CAD in 7 days
(3 Reviews)

Hi, Hope you are doing good. I have gone through your requirement and I do have skill set you are looking for. I am certified advanced RPA and QA Automation Professional. I have 2+ year of experience with Automation More

$677 CAD in 7 days
(1 Review)

hello how are you? I am very interested in your project i have much experiences in web scraping. I am sure I can do your project I am ready to start work immediately thx

$500 CAD in 7 days
(0 Reviews)

hi i am a fullstack web and application developer. i have about 9 years of experience and most of my activity in developing extensiosn and templates has been for cms.

$300 CAD in 12 days
(0 Reviews)

Hi, I am medical laboratory [login to view URL] experience almost 18 years plus I had 8 years of international gulf [login to view URL] to covid [login to view URL] are in worst stage of our cariers. Relevant Skills and Experience I am fr More

$556 CAD in 10 days
(0 Reviews)

Hi I can answer the question. 1) I have some question. - Do you want to GUI scraping program? - Do you want to scrap all of that websites mentioned on attached file at the same time? - What is the meaning of m More

$750 CAD in 8 days
(0 Reviews)

Dear Prospect Hiring Manager. Thank you for giving me a chance to bid on your project. I am a serious bidder here and I have already worked on a similar project before and can deliver as u have mentioned "I can do th More

$500 CAD in 7 days
(0 Reviews)

Dear Sir I am very motivated in your project. I have many experiences of web scraping and have been working for 5+ years. I already have done many shoppingmall, sports betting bots such as amazon, alliexpress, bet365, More

$555 CAD in 7 days
(0 Reviews)

Why Choose Me ! - I have been working with the Canadian based Company (Elevate-inc) which is positive review generating site. Basically we Scrap Reviews and integrate it with our portal. - I have Multiple Sample Scra More

$750 CAD in 30 days
(0 Reviews)