PySpark Jobs

We are looking for some one who can help in the below Dockers Package code, deploy applications in portable containers and scale them using AWS ECS service in the AWS cloud. Container Orchestration ( Docker Swarm) to allow manage multiple containers deployed across multiple host machines Kubernetes Use Kubernetes to manage containers Deploy a stateless application Deplo...

building python and pyspark frame work for ETL pipelines, data warehousing, azure data factory

I am looking for a male who is expert in Hadoop, Java, MapReduce, Sqoop, Hive, Hbase and pig with good experience in Design patterns, ETL and Frameworks. Need good communication skills and available to take phone interview US EST time zone during working hours. I will send the job description well in advance. There might be multiple phone interview rounds, and the funds will only be transferred...

1) Peform ETL S3 ( HDFS file) as source and Redshift as target Technical Environment : EC2,EMR, Pyspark & Hive 2) Peform ETL across S3, AWS noSQL database using Glue 3) One sample Python AWS Lambda function to run AWS Redshift SQL scripts 4) One sample on realtime streaming ( preferably Kafka( producer) + Pyspark( Consumer) 5) Executing bash commands ( jobs ) via Airflow ( or AW...

Using Spark ML Library to ReCode "Thinking Process" from 2 Similar Python Panda Scripts. Knowledge of the Codes from [login to view URL] is Required. Entails Features Transformation and Wrangling on json data. Milestone will not be released until project is delivered.

kubernetes is hosted on premise, spark need to be setup over that

i need the Python - Scala expert for multiple jobs. Details will be shared

i need the Python - Scala expert for multiple jobs. Details will be shared