Project Title: RTB Role: Senior Data Engineer
Description : RTB project involves in Developing the new integrations and Enhancements to the existing integrations and as well optimizing the integrations in matilion and snowflake
Responsibilities:
• Designed and developed various Matillion jobs like flat file load, database loads, API based integrations, Big Query data load to Snowflake.
• Developed the SnowSQL scripts to find out the duplicates inthetable level.
• Implemented Audit mechanisms for Matillion jobs and created the common frame work jobs for Flat file loads and database loads
• Worked on various credit optimizations tasks on snowflake data warehouse.
• Enhanced the few of the BODS jobs and migrated them from BODS to Matillion.
• Replicated the some of the tables to snowflake using perspectium. Environment: Matillion, Snowflake, Python, SQL ,BODS ,Perspectium
Project Title: HCM (WasteManagement) Role: Snowflake/ Matillion Developer
Description : HCM involves Employee payroll related data for different regions. The main focus of the project is migrating the data from people soft data to HCM cloud
Responsibilities:
• Designed and developed Matillion jobs to load data into various Snowflake tables.
• Developed the SnowSQL scripts to Validate, Cleanse, and Standardize and summarize the data.
• Implemented Error handling for Matillion jobs and worked on different methods of logging mechanism.
• Created a shared jobs for reusability of error framework and HCM loads.
• Implemented the python script for triggering the HDL jobs to oracle cloud.
• Worked with different file formats like fixed width, Flat files, semi structured date.
• Used the various components like truncate, S3Load, S3Unload, SNS,DataTransfer, Detect Changes ,filter ,SQL ,Excel Query ,Python ,Bash Script, API Extract etc. Environment: Matillion, Snowflake, AWS. ………………………………………………………