OnBenchMark Logo

Shubham

Data Engineer
placeholder
Location 
view-icon
Total Views90
availability
Shortlist0
back in time
Member since30+ Days ago
 back in time
Contact Details
phone call {{contact.cdata.phone}}
phone call {{contact.cdata.email}}
Candidate Information
  • User Experience
    Experience 3 Year
  • Cost
    Hourly Rate$5
  • availability
    AvailabilityImmediate
  • work from
    Work FromOffsite
  • check list
    CategoryInformation Technology & Services
  • back in time
    Last Active OnJuly 13, 2024
Key Skills
SQLPythonapache sparkAWSgcpData WarehousingdatabricksDatabase
Summary

WORK EXPERIENCE Koantek (Former Apisero company), Pune Sept 2022-Present Data Engineer

● Developed and tested the internal framework Ascend for Automated, fast development and monitoring of the data pipelines which reduced deployment time by 90%.

● Designed and developed the Unity Catalog Upgrade Tool to sync the hive-metastore table to the Unity catalog which provided a unified data governance solution and helped manage fine-grained access controls in Databricks lakehouse on any cloud.

● Built, configured, and managed the data infrastructure using Databricks and Cloud Services to provide meaningful insights for the business. Tata Consultancy Services, Pune April 2021-Sept 2022 Data Engineer

● Designed, developed high throughput and reliable data and reporting from the ground up using SQL, Python, and Power BI to provide real-time insights to the logistic company handling 400GB of data every 15 minutes.

● Investigated, Optimized, architected the legacy Data Warehouse which reduced the query runtime from 390 Minutes to 17 Minutes and helped to save cost by 95% associated.

● Implemented ETL, orchestrated the data pipelines using AWS Lambda, and visualized the data using the Power BI dashboard to produce real-time insights into the Business which monitored and increased the efficiency by 70%.

● Analyzed the client requirements and led a team of 4 team members which provided data as a service. 

Copyright© Cosette Network Private Limited All Rights Reserved
Submit Query
WhatsApp Icon