Looking for good developer who can make custom search engine php script.. (similar to addsearch [.] com)
1) Back-end: Feature to add different websites into our search index crawling...
a) Add only main URL of websites.. and then the crawling go to that website and search all links ... unlimited pages based on the website no. of post/pages.
b) Add either direct website or RSS to fetch the new URL's or Posts from that specified website.
c) This will crawling the given website url/rss and ... crawling each url page and get Metadata title, description, keywords, featured image url (same image in local), og: social signal data if available.
2) All crawled data insert into mysql database.
3) in the frontend, there is a only search box like GOOGLE search ... so user search box, perform search... after search the keyword matches in the title of words from database, and get the results showing all matched .. like GOOGLE search.
Note: The crawling process always run in background and get new pages crawled from the given websites URL/feed. Cron job keep running for new posts in the given websites... always... run background.
for example see here ... http://resultspage.s3-website-us-east-1.amazonaws.com/filters.html?addsearch=a
Hello, I'm experienced in web scraping (crawling), PHP and MySQL, I'd be happy to work on your search engine, if that suits you, let me know! Regards, Yassine