Python scraper built in Visual Studio code using Scrapy Splash & Lua

In Progress Posted 3 years ago Paid on delivery
In Progress Paid on delivery

NOTE: To be eligible to get this project: share code samples of similar projects you've done.

SOLID REFERENCES & A FAIR PRICE ARE KEY CRITERIA FOR WINNING THIS PROJECT!

I want a Python scraper built in Visual Studio code using Scrapy Splash with Python and Lua. I need these exact technologies because that's what I'm learning myself. The scraper should do the following:

Start URL: https://bit[dot]ly/3jEeN09

Follow links to detail pages of each of the visible results on this page.

From the detail pages the following fields need to be scraped to a single JSON object:

- Full URL of page

- Get (only!) the euro value from h2 element

- [login to view URL] (CSS selector)

- [login to view URL] (CSS selector)

- Full object description text

- Status

- Construction type

- All URLs of the available photos (in "All photos" section)

- From the "Features" section get the "Deposit" field value:

The "Deposit" column might not exist or be located further down on the page.

So, your code should check for the occurrence and position of field "Deposit" and scrape the corresponding value, e.g. "available", "" (non-existing), "rented"

All JSON objects need to be collected in a single array that is then stored as a single file "[login to view URL]" on disk.

Important notes:

- Results are loaded dynamically so you cannot simply get the raw HTML, JavaScript needs to be executed (view the page in the browser to see that the displayed results are different from what is available in the HTML source.

- The search result page loads more results on page scroll, so those results too have to be taken into account when scraping, so Lua custom script needs to be executed

Deliverables:

- File with scraped results

- Full source code in Python via a Visual Studio Code project using Scrapy Splash and Python

- Full source code needs to be documented, explain on each line what that line does

Python Web Scraping JavaScript Scrapy Lua

Project ID: #27943252

About the project

11 proposals Remote project Active 3 years ago

11 freelancers are bidding on average $128 for this job

kingqueen369

Hi, Greetings! ✅checked your project details Python scraper built in Visual Studio code using Scrapy Splash & Lua ✅Completed Time : 24 hours(independent and fast workers only) or your requirement deadline . I would li More

$100 USD in 1 day
(11 Reviews)
4.3
earkania

Hello, there. Here is a web scrapping expert. Your requirements are clear and this task can be started right now. I will deliver a perfect result within a day. Thank you.

$100 USD in 7 days
(6 Reviews)
4.1
ayesha0124

Hi there, I have read the brief details on the job listing. You can check my experience, customer feed backs and my portfolio. I believe its a doable job I have great experience doing projects with JavaScript, Lua, Scr More

$150 USD in 28 days
(1 Review)
3.6
chnsci

Greetings, I think here you are looking for a python scraper built in Visual Studio code using Scrapy Splash & Lua. I understood your project description. I am an expert in Python, Web Scraping, Scrapy, and am inte More

$110 USD in 7 days
(3 Reviews)
3.3
sanjasfrl

I'm a Web Scraping expert, with a lot of experience dealing with python and selenium. I have read your requirement carefully and visited your target site. I've previously worked on the exact same project and I can help More

$150 USD in 2 days
(4 Reviews)
3.0
RabbitFreelancer

Hello How are you? I am the right person for this project. I hope to discuss and learn more details about your project with you. I am waiting for your message. Best Regards.

$250 USD in 7 days
(3 Reviews)
2.4
Arkadyreact

✨✔⭐⭐Dear Employer⭐⭐✔✨python scraping expert(100000 per day) Hi Nice to meet you. how are you today? I sincerely wish you to be safe and always healthy on covid19. I look forward to the analysis of your needs and cooper More

$100 USD in 2 days
(5 Reviews)
2.4
VaLer0chka

Hello, manager! Nice to meet you. I hope you are safe without the effect of covid-19. I have read your requirements carefully. I have a great experience with web scraping. I am using python for web scraping. Please che More

$100 USD in 7 days
(2 Reviews)
2.2