Create Working BuBING Web Crawler Docker Image

This job post is about specific open source web crawler: [login to view URL]

You can find overview setup guide: [login to view URL]

This web crawler looks hard to setup for me because I lack Java ecosystem knowlege and documentation on crawler setup is not very detailed.

The Idea would be to setup working docker image of bubing crawler. Requirements for image:

1) It should be parametererized - it must be possible to somehow pass initial seed of URL's

2) Image should do all Bubing configurations listed in [login to view URL]

3) Configurations should be tailored for 16 vCPU core 64 GB RAM 10 Gbit Network VPS machine.

4) One should be able to run container from this image in AWS EC2 Spot Instances.

5) Once started, container should work immediately - crawl must start on container.

6) Container should store list of crawler page files in some folder. The file of crawler page should contain page URL and all HTML page content.

Additional requirements for job:

1) You have to provide Dockerfile that was used to setup docker image

2) You have to provide short Readme description how docker image behave

Skills: Java, Amazon Web Services, Linux, Software Architecture, Web Crawling

See more: perl convert image jpg gif open source, flash image gallery scroll open source, crop image rectangle iphone open source, post list open source, image feature extraction open source tools, image feature extraction open source, working of web crawler, image matching software open source, image recognition software open source, image processing software open source, create your own web crawler, git clone https github com tananaev traccar web git, image annotation tool open source, image recognition api open source, https github com phusion baseimage docker blob master changelog md, gimp 2.8 for photographers image editing with open source software, https github com seleniumhq docker selenium, git clone https github com zoom sample app web git, how to create a simple web crawler in php

About the Employer:
( 3 reviews ) Vilnius, Lithuania

Project ID: #29947265

2 freelancers are bidding on average $570 for this job


Hi, Hope you are doing well. I have full experience about Java/JavaFX so that I have confident to complete your project perfectly. I will be very happy to discuss about your project via chatting. Thank you.

$140 USD in 7 days
(25 Reviews)

hi I am interested in your project and I have 7 years experience in JAVA. Please send me massage in inbox to discussion on project. thanks Inder

$1000 USD in 20 days
(0 Reviews)