Data Engineer
Location : Hyderabad, India
Experience : 4 Year
Rate: $14 / Hourly
Availability : Immediate
Work From : Offsite
Category : Information Technology & Services
Satish Data Engineer
Senior data engineer experienced in working across the different stages of the data pipeline, including acquisition integration, ODS and real- time data marts. Adept in working quickly and efficiently in close collaboration analytic, engineering with other stake holders
Senior Data engineer, 05/2021 – present | Bangalore, India Working as a data engineer for health care clients.
Created the Batch data pipeline for the daily load.
Managed a database migration project (SQL Server - Snowflake) to improve and consolidate end-to-end service processes.
A team of 7 to work on different aspects of the project.
Performed in agile methodology, interacted directly with the entire team provided/took feedback on design, Suggested/implemented optimal solutions, and tailored applications to meet business requirements and followed standards.
In-depth understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames. Proven ability in writing complex SQL queries after comprehending Business Transformation Logics Created email alert framework in python.
Created logging framework for storing the status of the notebook.
Experience in creating data lake using spark which is used for downstream applications Designed and Developed Scala workflows for data pulled from cloud-based systems and applying transformations to it. Implemented the incremental approach in the gold layer in the Databricks .
Moved the Databricks gold layer tables into Snowflake and then we have created views on those tables. Using those views, Reports are building for the end users.
Technical expertise on Snowflake Utilities like Snowpipe, Data Cloning and Time Travel and Data Sharing. Expertise in Snowflake Data modelling and implemented complex Stored Procedures.
Having good knowledge on DWH concepts like CDC and SCD type 1 and 2. Design the different views in the SQL server according to the business requirement. Worked on the performance of views in the SQL server
Database migrations from Traditional Data Warehouses to Spark Clusters.
Hands on experience in various Big Data application phases like data ingestion, data analytics and data extraction
Performed thorough testing and data validation to ensure the accuracy of data transformations. Eager to learn new Technologies.
Associate Consultant, Capgemini 11/2018 – 05/2021 | Chennai, india Started working on Teradata and Snowflake for a NBCU and Disney clients.
Technical expertise on Snowflake Cloud data warehouse with basic Hands on in, AWS S3, EC2,python. Technical expertise in ETL methodologies.
Experience in working with snowflake cloud data warehouse.
Extracted Data from Various DB’s like Oracle, Teradata and Flat files and staged into a single DB and applied business logic and load them into the Central Data Warehouse.
Helped the client in migrating from an old system to a new system within a snowflake. Loaded the unstructured JSON data into a snowflake.
Technical Expertise on snowflake Utilities like Snowpipe, Data cloning, Time Travel and Data Sharing.
Expertise in Snowflake Data modelling and implemented complex Stored Procedures.
Having good knowledge on DWH concepts like CDC and SCD type 1 and 2.
Created the ETL pipeline to migrate data from AWS s3 to Snowflake using UC4 Automata. Created email pipeline to notify business after the successful completion/failure of a job. Used snowflake time-travel feature for the data validations purpose.
Developed Teradata BTEQ’s scripts to load the data from source to target in various stage by applying the business logic.
Interacted with Business team on functional requirement, and managing the Delivery of Project.
Education
B.tech Electronics, Gier College 08/2014 – 05/2018 Rajahmundry, India
Skills
SQL
Snowflake Python Control-m
Teradata ETL
Scala UC4
Awards
Quarterly Award, 24/02/2021
Received Quarterly Award for working Extensively on Matillion ETL tool to load s3 data into SNowflake and development of integrating SQL Server Data into Databricks Using Scala Code.
Raising Star, Capgemini
Received Raising star award for excellent contribution in the project.
Delivery Excellence, Capgemini
Received Delivery Excellence award for delivering all the deliverables on time to the customers.
{{contact.cdata.phone}}
{{contact.cdata.email}}
View Resume
