Create a webcrawler using Github project

Closed Posted 12 months ago Paid on delivery
Closed Paid on delivery

I need a web crawler using GitHub project. My first preference would be crawler using go but I'm open to other crawlers too. I looking for something scalable in the future & with many concurrent crawls.

I need the following to be added to the database once a website is added to the crawler through the browser.

Website,

URL,

Page Title,

Linked pages ( URLs),

Linked Page (title)

Here's a list of crawlers, please mention if you have experience with any of them.

[login to view URL]

You would need to deploy the crawler & server & connect to a website. I prefer AWS but I'm open to suggestions.

Web Scraping Python PHP Golang

Project ID: #36257454

About the project

17 proposals Remote project Active 10 mos ago

17 freelancers are bidding on average $231 for this job

chirgeo

Hi, I'm George and with 8+ years of experience building and maintaining web crawlers, I understand your need for a well-structured, efficient crawler. I can help you build & deploy a crawler that meets your requirement More

$500 USD in 3 days
(149 Reviews)
7.9
fhasanbd

Dear Employer, I am excited to submit my proposal for the Create a Webcrawler using Github Project. With my experience in web scraping and proficiency in Python programming language, I am confident that I can deliver More

$90 USD in 7 days
(219 Reviews)
7.3
developersshop

Hi! Thanks for reviewing my cover letter. I am Mohmmad Farhan more than 6 years experienced in PHP / MYSQL / AJAX /JQUERY / JAVASCRIPT / CSS3 / HTML / HTML5 / WordPress / Woo-commerce/ Plugin development. I have re More

$250 USD in 9 days
(151 Reviews)
6.7
simonklemp

Hello Avii, I hope this message finds you well. I will be able to help you with your project. I read through the description and it sounds like I am just the right person for it. I have the right skill More

$200 USD in 3 days
(17 Reviews)
5.5
nisthark

Hi Sir, My name is Nisthar. I am a fullstack developer and an expert in building crawlers. I can build this github crawler for you. I have experience in crawlee. I have more than 5 years of experience in all programmin More

$250 USD in 5 days
(19 Reviews)
5.6
dataspro

Nice to meet you Yogi Network, It is a pleasure to communicate with you. My name is Anthony Muñoz, I am the lead engineer for DSPro IT agency and I would like to offer you my professional services. I have more than 1 More

$245 USD in 7 days
(4 Reviews)
4.9
robartint

Hello I am a professional python developer. My main specializations are automation, web scrapers and bots development. I have already developed over 200 scrapers. From the simplest (for example, a competitor's price More

$30 USD in 7 days
(8 Reviews)
4.0
nitin6312offcie4

? Experience in AWS Security: Created and Managed IAM roles, users, groups, policies, generated credential report for analysis ? Creation of Auto Scaling groups (ASGs) from custom or pre available AMIs, by creating lau More

$140 USD in 7 days
(3 Reviews)
2.1
Talha3344

Hello!, I'm an expert Developer and Designer of having 6+ years of experience in development and Designing. I am readily available to work with you in your project Create a webcrawler using Github project as per your More

$220 USD in 10 days
(0 Reviews)
0.0
Moeen109

Hi sir! I am a dedicated Software Engineer with 3+ years of experience in developing web and mobile applications. My expertise includes front-end development with >>>Html, Css, JavaScript >>>React.js More

$190 USD in 7 days
(0 Reviews)
0.0
andreistalbe

Dear [Recruiter], I am Andrei Stalbe, an experienced web crawler expert. I read the job description for creating a web crawler using GitHub project and review the list of crawlers mentioned. I have relevant experienc More

$500 USD in 3 days
(0 Reviews)
0.0
VishalG2909

Hello, Greetings for the Day!!! I have gone through your requirement and understood every aspect of them. I have 5+ years' experience in PHP, Website Design, Golang, Web scraping, HTML. I have experience with web c More

$250 USD in 30 days
(0 Reviews)
0.0
muneebsuri1

Hello! I can help you with your web crawler project. I have experience with web crawling and data extraction using Python and Scrapy, which is a scalable web crawling framework. I can develop a crawler that meets your More

$100 USD in 5 days
(0 Reviews)
0.0