DAPINDER

WORK EXPERIENCE Senior Data Engineer Microsoft 01/2021 - 12/2022, Hyderabad , India Project : Purchase Order & Procurement , Client: Microsoft (Redmond), Technology: Spark , Python , Scala, Azure Data Bricks , Azure Data Factory , Azure Data Lakes, Sql Azure, Java Implemented the Workflow Orchestration for Purchase Order and Procurement system for Data Extraction from EDL Source System Data Lakes to downstream systems using Azure Data Lakes and Azure Data bricks (using Python , Spark SQL , Java and Scala). Implemented Big Data Solutions for Data Extraction from various source streams which involves systems dropping files to data lakes and our workflow extract and transform these multiple source files with billions of data to azure pipelines activities and data bricks clusters and notebooks to be send to multiple destinations like SQL AZURE, SQL DATAWAREHOUSE Worked on Data Quality Framework using spark sql , scala and python in data bricks for Data Quality checks. Worked on handling heavy data transformations and extractions logics from multiple source streams. Implemented CI/CD release pipelines for Automation releases. Implemented Solutions for CI/CD automations using Yaml , PowerShell scripting for Infrastructure setups and Code deployment automations. Design and Implemented Disaster Recovery and Backup Strategy using Azure Blob Storage for Big Data Project Considering RTO and RPO Requirements. Azure Solution Architect Microsoft 05/2020 - 12/2020, Hyderabad, India Project: Supplier Scorecard, Client: Microsoft (Redmond), Technology: ReactJs , Redux, SQL Azure, .NET Core API Implemented Design Solution for Enterprise Level Ux Application setup in Distributed Environments Using techniques such as Load Balancing using Azure Traffic Manager, Offline Data Sync using Service Bus, Geo replicate Application handling for Disaster Recovery, Net Hosting for Public and Private Network. Implemented various screens for the front end using React.js and used various predefined components from NPM (Node Package Manager) and redux library. Suggest Architectural Improvements, Design and Integration Solutions, and formulate methodologies to optimize Object Oriented Design and Development. Implemented stable React components and stand-alone functions to be added to any future pages. Configured Web pack to execute linters ,magnifiers ,pre/post-processors, tests, transpires, etc.basedon build target. Implemented React container and presentational components (as Stateless and Functional components when applicable). Implemented in RESTful web services to integrate between Server side to Client Side Worked on Performance Tuning for Complex SQL Queries for performance bottlenecks areas. Design and Implemented Disaster Recovery plans for front end applications for RTO and RPO Requirments.