AWS Data Engineer

Position Title: AWS Data Engineer

Required Experience: 5+years

Job Location: Hyderabad / Chennai / Indore

Work Timings/Shift: 2 PM - 11 PM 

Job Description (Job Summary/Roles & Responsibilities):

• Design, develop, and maintain scalable and reliable data pipelines using AWS services, PySpark, and AWS lake formation.
• Implement ETL processes to extract, transform, and load data from various sources into data lakes or data warehouses.
• Collaborate with cross-functional teams to understand data requirements, identify data sources, and define data ingestion strategies.
• Optimize and tune data pipelines to ensure high performance, scalability, and data quality.
• Monitor and troubleshoot data pipelines to identify and resolve issues in a timely manner.
• Collaborate with data scientists and analysts to provide them with clean, transformed, and reliable data for analysis and modelling.
• Develop and maintain data documentation, including data lineage, data dictionaries, and metadata management.
• Stay up-to-date with industry trends and best practices in data engineering, AWS services, PySpark, Athena, and AWS Lake.
• Work in a team environment with the product, QE/QA, and cross-functional teams to deliver a project throughout the whole software development cycle.
• Provide technical support for data-related issues and incidents.
• Collaborate with other teams to resolve issues and ensure data integrity.

Mandatory Skills:

• 5+ years Experience as a Data Engineer on the AWS
• Strong technical expertise in Python and PySpark
• Experience with Big Data Tools such as Hadoop and Apache Spark (Pyspark)
• Solid experience with AWS services such as Cloud Formation, S3, Athena, Glue, Glue Data Brew, SQS, SNS, AWS Lake Formation, RDS, Redshift, Cloud Watch, Cloud Trail, Lambda, Step Functions,
• Solid understanding of data modeling, database design, and architecture.
• Experience with version control systems and collaborative development practices.
• Responsible for building, test, QA & UAT environments using Cloud Formation.
• Build & implement CI/CD pipelines for the EDP Platform using Cloud Formation and Jenkins

Desired Skills:

• AWS Certified Big Data - Specialty certification is a plus.
• Knowledge in Data bricks in preferable.