Sai
Experience: 6 Years
Summary:
- A Software engineer with 6+ of experience excited about the data driven society we live in, with keen interest in carrying out work involving data refinement, analysis, to aid businesses make better decisions.
- Familiar with data warehousing concepts.
- Certified GCP Data Engineer
- AWS Solutions Architect Associate Certified
- Excellent interpersonal skills, team coordination and well versed with Software Development life cycle
Technical Knowledge:
Technical Skills
AWS, GCP, python, pyspark, Informatica, OracleSql, Airflow, azure data bricks, Airflow
Languages
SQL,PL/SQL, C, Python
Tools
ODI Studio 11g, OBIEE 12c, Putty, WINSCP, Informatica, AWS, GCP, oracle db, postgres db ,powerbi, looker
Domain
Data Integration and Business Intelligence
Work Experience:
- Currently working as Lead data Engineer in Confidential, since 21-Dec-2021 to till date.
- Previously worked as Senior Data Engineer in Confidential, since 01-JUL-2020 to 20-dec-2021
- Previously worked as Data Engineer in Confidential from 01-Mar-2017 to 26-Jun-2020
Project 1:
Project
TRG
Role
Technical Lead
Position
Technical Lead
About the Project
- This project is implementation of Cost and usage dataanalytics for various clients across multiple cloud providers
Roles and Responsibilities
- Implemented custom logic using python code to load aws capacity data from awsapi and load it into table
- Created lambda function for data process data for cost and spend analytics data.
- Used Aws Crawler to load data to athena.
- Create aws Neptune environment and loaded s3 data to aws Neptune to perform graph analysis
- Implemented custom logic to load right sizing recommendation data from api.
- Used AWS EMR to transform and move large amount of data to other aws services like s3 and dynamo db.
- Worked with Nested json and parquet files using aws glue.
- Analyze cloud infrastructure and recommend improvements for performance gains and cost efficiencies
- Documenting design decisions and ensuring adherences.
- Created power bi reports.
- Adept in troubleshooting and identifying current issue and providing effective solution.
- Worked on various Hadoop file formats json, parquet,csv ,Avro
- Creation of data model and managing data.
- Worked on performance tuning to improvepyspark and sql queries.
- Working on data discrepancy issue and resolving them.
- Optimized existing stored procedure and performance tuned sql queries
- Created etlpiplines to load data into database
- Used Aws glue pyspark to load to load data from big query.
- Using Aws to glue to load parquet data to gcp.
- Created multiple lamba event based triiger function for data processing.
- Created step function integrated with multiple lambda.
- Automated daily load alerts with lamba and ses.
- Implemented anamoly detection on aws cloud data.
- Created machine learing model using arima to predict forecast data
Project 2:
As a Technical lead, I workedon many things like Team handling, Customer Handling, Development, Delivery, Information Gathering, Research and development working on different project from end to end using various technologies like Gcp, Data Prep, Cloud Storage and Data ingestion into pipelines.
Roles and Responsibilities
- Developer looker dashboards to the customers that meet business requirements.
- Using cloud function to load data into bigquery on arrival of csv in gcs bucket
- Customer gets the data from their on-premise systems to Google Cloud Storage bucket periodically.
- Built a Dataflow pipeline that preprocess the data files and then loads the data to BigQuery.
- Used Cloud Functions to trigger the Dataflow pipeline whenever customer loads
DATA ENGINEER
Location : DELHI, India
Experience : 6 Year
Rate: $16 / Hourly
Availability : Immediate
Work From : Offsite
Category : Information Technology & Services
{{contact.cdata.phone}}
{{contact.cdata.email}}
View Resume
