OnBenchMark Logo

Bhaskar (RID : 15cyvlmyn2szy)

designation   Sr Data Engineer

location   Location : Jaipur, India

experience   Experience : 8 Year

rate   Rate: $22 / Hourly

Availability   Availability : Immediate

Work From   Work From : Offsite

designation   Category : Information Technology & Services

Shortlisted : 0
Total Views : 76
Key Skills
Data Engineer Azure SQL Azure Data Factory Azure
Discription

ProfessionalExperience                                                                                                                           

01/2021–present                 SeniorLead DataEngineer

InvolvedinBusinessrequirementdocumentwalkthroughandclientcallstounderstandfunctionality

DesignedastablearchitectureforProject

Involved in planning and development of ETL pipelines using ADFUnblockingtheteammembersinrequirementsandtechnicalissuesDesignedAzureDataFactoryPipelinestomovedatatoAzureDataLakeImplementedADFPipelinestopulldatafromdifferentsources

Accessed Azure Data Lake files data in Azure Data bricks using mount pointsImplementedBusinesslogicsinSparkSQL/PysparktopopulaterequiredtablesHandledcleansing,aggregationsandtransformationsinSpark

ImplementedAzureFunctionswithPythontopulldatafromAPIsandscheduledtheseAzureFunctionsinAzureDataFactory

ImplementedendtoendloggingframeworkforAzureDatafactorypipelineDesignedofDatawarehouseforstableimplementation.

IntegratedAzureDataFactoryforschedulingJobsandTriggeringotherrelatedJob.ExtensivelyusedAzureKeyVaultstoconfiguretheconnectionsinLinkedservicesCreatedLogicAppstotriggerthemailnotificationfromADFPipelines.

ImplementedStoredProceduresbusinesslogicinAzureSQLDatawarehouse.

CreatedCI/CDpipelinesforADF,AzureSQLDatawarehouseandAzureSQLDatabasedeploymentinADO

DeployedPipelineandcodetomultipleenvironmentsusingCI/CDpipelineValidatedthedataatoverallandeverydimensionlevel.

WrittentestcasestovalidatedatainADO

09/2018–12/2020                SeniorSoftwareEngineer

DesignedofDatawarehouseforstableimplementation.

Designed anddevelopedAzureDatafactorypipelines forDatamigration.IntegratedAzureDataFactoryforschedulingJobsandTriggeringotherrelatedJob.

 

Created LogicApps topull thedata fromSharePoint excelsto ADLS.ImplementedAzureDatabricksnotebooksbusinesslogicatDatawarehouselevel.CreatedamountpointinPythontoaccessthefilesresideinADLS.

CreateduserdefinedfunctionsusingPython.

Createddataframes,viewsandtablesusingPythoninAzureDatabricks.

Used Python loops and aggregation techniques to process data in Azure Databricks.PublishedtheFactandDimensiontablesfromADBtoAzureSQLDatawarehouseandADLSusingPython.

CreatedvalidationframeworkusingPythoninAzureDatabricks.

CreatedTabularmodelonFactandDimensionviewswhichareinAzureSQLDatawarehouse.

UpdatedTabularlevelmeasuresanddimensions,wroteDAXqueriesforcalculatedmeasures.

ValidatedthedataatoverallandeverydimensionlevelinTabular,checkedondataisslicingateverydimensionlevel

WrittentestcasestovalidatedatainatTabularinADODevelopedreportsusingPowerBI.

07/2017–08/2018                SoftwareEngineer2

Designedanddevelopedrelationaldatabases.Guidingteammembers.

DevelopedSSISpackagestoloadthedatafromdifferentsourcestoSQLandcreateaDatawarehouse.

ImplementedStoredProcedurestoprocessbusinesslogicatDatawarehouselevel.CreatedTabularmodelontopofDatawarehouse

ImplementedmainbusinesslogicforfinancialmetricsinTabularusingDAXqueriesValidatedthedataatoverallandeverydimensionlevelinTabular,checkedondataisslicingateverydimensionlevel

DevelopedreportsusingPowerBI.

WroteautomatedBVTSPstocheckvaluesateachlevelonrefreshofdatadevelopedscriptfilesonbigDataplatformtomaketheheavydatatobeprocessed

Project Details
Title : Sr Lead data engineer
Duration : 12 (Month)
role and responsibileties :

Involved in Business requirement document walkthrough and client calls to understand functionality

Designed a stable architecture for Project

Involved in planning and development of ETL pipelines using ADF Unblocking the team members in requirements and technical issues Designed Azure Data Factory Pipelines to move data to Azure Data Lake Implemented ADF Pipelines to pull data from different sources

Accessed Azure Data Lake files data in Azure Data bricks using mount points Implemented Business logics in Spark SQL/Pyspark to populate required tables Handled cleansing, aggregations and transformations in Spark

Implemented Azure Functions with Python to pull data from APIs and scheduled these Azure Functions in Azure Data Factory

Implemented end to end logging framework for Azure Data factory pipeline Designed of Data warehouse for stable implementation.

Integrated Azure Data Factory for scheduling Jobs and Triggering other related Job. Extensively used Azure Key Vaults to configure the connections in Linked services Created Logic Apps to trigger the mail notification from ADF Pipelines.

Implemented Stored Procedures business logic in Azure SQL Data warehouse.

Created CI/CD pipelines for ADF, Azure SQL Datawarehouse and Azure SQL Database deployment in ADO

Deployed Pipeline and code to multiple environments using CI/CD pipeline Validated the data at overall and every dimension level.

Written test cases to validate data in ADO

Description :

.


 
Matching Resources
My Project History & Feedbacks
Copyright© Cosette Network Private Limited All Rights Reserved
Submit Query
WhatsApp Icon
Loading…

stuff goes in here!