Convert existing web scraper from PHP to AppEngine (Java) + Cron
$100-500 USD
Cancelled
Posted over 12 years ago
$100-500 USD
Paid on delivery
We have an existing web scraper running on a Linux server in PHP. We need to convert this to run on AppEngine. The web scraper scrapes data from two different web sites, which are android app-stores. Project requirement highlights - Scrape data from 2 different websites - Well-designed scraping framework (modular, well-organized code). If we need to scrape another website, we should be able to add it easily within the framework. - Cron to schedule daily web crawling. - Expert Java developer with very good knowledge of App Engine required. The data will be saved in 3 entities (tables) in datastore: -App Data (saves data in different columns) -Raw Data (saves raw page data for reference) -Scrape Log Please respond and mention your experience with web scraping, Java, and App Engine. for reference: [login to view URL]