OnBenchMark Logo

Prabhat (RID : 6v6vlqnhb6np)

designation   Data Engineer Python with AWS

location   Location : Noida

experience   Experience : 5 Year

rate   Rate: $16 / Hourly

Availability   Availability : Immediate

Work From   Work From : Offsite

designation   Category : Information Technology & Services

Shortlisted : 1
Total Views : 32
Key Skills
Data Engineer Python AWS Git GitHub Snowflake Kafka Spark
Discription


Projects

Confidential

 

  1. Project            :           Waste ManagementDuration        :                        April2023 –Present

 

The client is the leading provider of comprehensive waste management servicesin North America serving over 20 million customers through collection, landfill,transfer, recycling and other lines of business. The project is to performDiscovery, Design Assessment and implementation of migration from the existinglegacy data warehousing environment built using Netezza, Control M andInformaticatoacloud-basedwarehousing solution.

    • Analysis of existing Data Warehouse Objects and ETL/ELT Pipelines andpreparation of Database Object Inventory of legacy Control M jobs andInformaticaworkflows.
    • Propose new DW Platform design over cloud with required modifications anddevelopment.
    • Development and deployment of the new architecture within specifictimelinetodecommissiontheservers.
    • Creating data validation module to maintain data consistency and qualitywhilemigration tocloudplatform.

Technologies: Snowflake, AWS, DBT, Matillion, Informatica, Git and GitHub,Internaltoolkit.

NagarroPvt.Ltd

 

  1. Project            :           Radiometer SendxDuration           :                        Aug2021–March2023

 

The project is to develop data warehouse for different formats (CSV, JSON, RDBMS,API calls etc.) of data generated by company during manufacturing and testingprocess of healthcare related devices and equipment. The aim was to setup a well-governed Data platform in a secured environment for advanced data analytics andvisualization.

 

  • Stage CDC Pipeline Setup to drive data-warehousing solution via Debezium forPostgresInstances.
  • AutomateddataonboardingusingGitHubWorkflow&Airflowtogetdata

onboardingrequestfulfilledwithoutdependenciesonDataSourceOwner’sTeam.

  • Creatingpipelinesfor incrementaldataload fromvariousdatasourcesforDataLake.
  • Ingesting data from various sources such as APIs, Oracle and MS Server databases,File basedsystem andreal time streamingsources.

 

Technologies:AWS(Glue,Redshift,Athenaetc),Spark,Kafka,Airflowetc

 

TataConsultancyServices

 

  1. Project            :           Commonwealth Bank of Australia – CRA (Customer RiskAnalytics)

Duration         :           Oct2019 –July2021

 

The project was to implement and support investigative application for financialcrime detection, anti- money laundering and customer risk assessment. It provides acomprehensive view of the client by analysing and flagging their demographic andgeographiccharacteristics.

 

  • CreatingJobstofilterout Riskycustomersfromthebank’sBasebasedonvarious

filteringcriteria'ssuchasitsbackground,homecountry,etc.

  • Created File compaction pipeline using custom checkpointing to avoid data skewnessandovercomesmallfileissue.
  • LA(LoadAssurance)upliftmentofexistingdatapipelines.

 

Technologies:Spark,Scala,Kafka,Autosys,YAML,Hive,SBT,TeamCity,XLRelease.

 
Matching Resources
My Project History & Feedbacks
Copyright© Cosette Network Private Limited All Rights Reserved
Submit Query
WhatsApp Icon
Loading…

stuff goes in here!