Leads Calculator

Hi, I have reviewed the project details and understand your requirement of a Python, PySpark developer who has expertise in interactive SQL development and PySpark ETL Pipeline. I am capable of completing this task in a timely and cost-effective manner. Let's connect and discussion more on it. Short Into about Me: Currently working in one of the Big4 Companies and solving Big Data Business problem using AWS Cloud and Pyspark Engine. I have expertise on the following Tools/Technologies: - Apache Spark (PySpark) - Python3, unit test script, Numpy, Pandas - AWS Cloud Services (EMR, DynamoDB, S3, Glue Lambda, EBS, Athena, IAM, Step function, EC2, ECS, secret manager (kms), s3, sns, ses, sqssqs, cloudwatch etc) - Creation of cloud formation templates for various AWS Services - CI/CD Pipelines - SQL/MySQL - Git, GitHub, BitBucket - Big Data Pipelines - IBM DB2 - MS Excel, MS Office - PyCharm, VSCode, Databricks, Jupyter Notebook, MS Excel, Putty Please provide more details if it's more than what you have given in the project details. Thank you! Thanks & Regards, Prakash

6 years experience in big data and working currently pyspark In data bricks environment Handling different sources and variety of sources for the ingestion process to consume data till end

I understand what you are looking for in terms of adding history functionality to an existing ETL process in PySpark. With my experience in data manipulation, transformation, and analysis, I am confident that I can provide accurate and high-quality results. Moreover, I have hands-on experience in setting up ADF pipelines and Logic Apps that enable seamless data integration and processing. I am ready to collaborate with you on this project and contribute to data-driven projects that make a tangible impact. I believe that my skillset makes me the best fit for this job requirement. If you give me the chance to work on this project please don't hesitate to reach out with any questions or queries you may have!