Data Engineer

Responsibilities

· Creating and maintaining quality ETL pipelines.

· Write high quality, testable code.

· Review code and technical decisions to ensure best practices are being followed.

· Working closely with the product and business to drive the product vision and manage stakeholder expectations

· Investigate and implement new features on an ongoing basis

· Provide guidance and assistance in the selection of tools, technologies, and implementation details

Skills:

· Strong coding skills in Python using pandas, numpy etc.

· Good experience writing complex SQL queries in SQL server or similar data store.

· Experience in developing data pipelines using Azure functions/ AWS Lambda or Standalone python.

· Experience in developing python code to handle the data in data lakes hosted in Azure, AWS or HDFS.

· Experience in working with at least one of the cloud environments (Azure, AWS or GCP)

· Good experience in working with smaller teams and in agile setup.

· Experience with CI/CD and preferably DevOps solutions like Azure DevOps.

· Experience in communications and senior stakeholders’ management.

Desirable:

· Working knowledge with Azure functions.

· Experience in writing powershell scripts

· Experience in distributed data warehousing applications like Synapse, Redshift.

· Exposure to data governance or regulations

Python(Pandas, Numpy), SQL, PL/SQL, Data lakes, Data warehoTechnologiesusing.