Spark jobs

We have application running spark jobs, but at a time we are able to submit only 3 to 4 jobs and not more than that

Need some enhancements and imlrovements in mongo db and spark [login to view URL]

Need some enhancements and imlrovements in mongo db and spark mllib

I want a PoC to be done for the below requirement. Source data files(in CSV format and SQL extract format) and a schema files (in JSON format) will be placed in GCS. Based on the schema definition rules given in the json file, data in CSV file should be loaded to Kafka and using Kafka streaming, data needs to be transformed to 3NF form and loaded to GCS/Google Big Query . The main objective is,Whe...

I want a PoC to be done for the below requirement. Source data files(in CSV format and SQL extract format) and a schema files (in JSON format) will be placed in GCS. Based on the schema definition rules given in the json file, data in CSV file should be loaded to Kafka and using Kafka streaming, data needs to be transformed to 3NF form and loaded to GCS . The main objective is,When schema changes ...

Strema xml data from kafka topic and write to hive table

Hi, We need Hadoop, Kafka, Spark consultant that have extensive real world production experience of deploying such system in DC/OS. The job will be advising our engineer to use industry best practice development and deployment of such system. The job include: - Architecture design/consultantion - Use case scenario/best practice - Design/algorithm review - Code review - Pair programming with our...

Help me in persisting data to mysql. Urgent.