Project: Data Mastering Framework Employer: UBS Business Solutions India Duration: April - 2022 - Till date Environment & Tools: Azure Platform, Azure Data bricks, Azure Data Factory Description: Data Mastering Framework enables metadata driven Feed Processing Pipelines. Application consumes metadata of the Feed mapping, Validation rules, Transformation Rules in the form of JSON and produces validation and transformation queries on runtime.
Roles and Responsibilities
• Closely worked with Product Owner for understanding the business specifications and requirements
• Worked on Initial design of different metadata for the framework.
• Developed Scala programs to parse JSON file and generate Spark SQL at runtime.
• Developed Spark Scala UDFs based on business requirement.
• Developed ADF pipeline to run Spark programs.
• Worked on the Spark performance optimization.
• Responsible for UAT, post deployment and production support activities Project: Risk Finder Platform - Counterparty Credit Risk Employer: Deutsche Bank India Duration: February - 2021 - March - 2022 Environment & Tools: HDFS, Spark, Hive, Scala, Impala, Oozie, Description: Risk Finder is the Risk & Capital Analytics Platform for Deutsche Bank. The platform delivers portfolio analytics and scenario analysis across all asset classes and business.
Roles and Responsibilities
• Interacted with different teams for integration testing & achieve deliverables
• Worked with SARC for designing DBAX Margin Model Framework
• Enhanced DINO framework process to read different files format to load into hive tables
• Developed Dispatcher Job with Spark Scala programs to implement the business requirements
• Developed Spark Data Frames and Spark SQLs to process the data