Web Scraping of links from google automated
$100-200 USD
Paid on delivery
I have this web scarping script for fetching links from google directly. This script is working good but its slow. I want it to be made automated. Like either modify it and make this one automated or add socks option. Because after trying to fetch links from same ip the google shows captcha, then have to change ip and fetch results from same place. It consumes alot of time.
Those who are interested, script is attached. This is how it works!
1- Select "http", select "shop", select "inurl:/catalogsearch/"
2- Select "[login to view URL]"
3- Ignore TLD (Its pretty useless)
4- Select "countryUS"
5- Select the pages till which we want to select results. Normally I select 5 beacause after that google shows captcha and links are not fetched.
6- Click Scrap.
It will fetch all the links and the url will be like.
executing [login to view URL]:/catalogsearch/+inurl:http&cr=countryUS&tbs=ctr:countryUS&num=500&start=0
executing [login to view URL]:/catalogsearch/+inurl:http&cr=countryUS&tbs=ctr:countryUS&num=500&start=1
executing [login to view URL]:/catalogsearch/+inurl:http&cr=countryUS&tbs=ctr:countryUS&num=500&start=2
executing [login to view URL]:/catalogsearch/+inurl:http&cr=countryUS&tbs=ctr:countryUS&num=500&start=3
executing [login to view URL]:/catalogsearch/+inurl:http&cr=countryUS&tbs=ctr:countryUS&num=500&start=4
So these links are fetched for the keyword "/catalogsearch/". Next the script should fetch for HTTPS like this,
[login to view URL]:/catalogsearch/+inurl:https&cr=countryUS&tbs=ctr:countryUS&num=500&start=0
And next
[login to view URL]:/catalog/seo_sitemap/+inurl:http&cr=countryUS&tbs=ctr:countryUS&num=500&start=0
If you can modify it and make it automated good enough. If you can make anyother automated script for fetching links where i just need to add keywords and it will fetch all the links.
If you want to ask any questions. Do let me know. Thanks
Project ID: #6947008
About the project
15 freelancers are bidding on average $180 for this job
Hi sir, I am scraping expert, I have did too many similar projects, please check my feedback then you will know. Can you tell me more details? then I will provide demo data for you. Thanks, Kimi
Hi I work towards providing reliable, relevant and robust IT solutions at most competitive prices to my customers. I ensure 100% customer satisfaction so lets start Thanks
hi there i can create a google search bot for you. kindly initiate the discussions lets discuss more thankyou ----------
Our services are a software consulting company specialized in providing Mobile, E-Commerce and Social media frameworks using cutting edge and emerging technology. Leveraging best-in-class people, processes, and techno More
Hi . Let me do this for you. i have a faster and no captcha show way I'm a programmers and I have experience in the field of Web Scrapping, Web Crawling I look forward to working with you, thank you !
hello, I am expert in data scrapping from the web. I have done may jobs like that. please ping me if you want to have job done 100% efficiently. thanks
Greeting, Sir I just saw your project & willing to provide service for the project. Can we discus about the project. I can assure that you will get 100% satisfaction from our side. Hope you will notice my proposal More
