▪ Created Hive queries that helped business analysts to spot the customer trends, by comparing fresh data with reference tables and historical metrics.
▪ Automated a generic system using Oozie to handle structured data received from logs, which provide customer related information and performed analysis on the data for different use cases.
▪ Handled importing of customer data from various data sources, performed transformations using Hive and Impala.
▪ Extracted the data from Oracle database into HDFS using Sqoop with incremental load to populate Hive External tables.
▪ Performed data validation on the input data by building a custom model to filter all invalid data and cleansed the data.
▪ Developed a custom File System plug-in, so it can access files on Hybris Data Platform. This plug-in allows Hadoop MapReduce programs, Impala and Hive to access files directly and provides data locality.
▪ Created Workflows in Oozie with Shell, Hive and Email actions and automated the workflow.