Senior Associate (Data Engineer)

Objective: As a Data Engineer, you will be responsible for building effective data solutions that will cater the need of Data Analysts, Data Scientists, and other Data Consumers across the organization. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. They assess a wide range of requirements and apply relevant database techniques to create a robust architecture and will ensure delivering of high quality ETL solutions.

Roles and Responsibilities:

➢ Good knowledge on GCP Data Flow, Big Query, Google cloud storage, cloud function, Cloud composer, etc.

➢ Should be comfortable in building and optimizing performant data pipelines which include data ingestion, data cleansing and curation into a data warehouse, database, or any other data platform using Py Spark.

➢ Experience in distributed computing environment and Spark architecture.

➢ Optimize performance for data access requirements by choosing the appropriate file formats (AVRO, Parquet, ORC etc) and compression codec respectively.

➢ Experience in writing production ready code in Python and test, participate in code reviews to maintain and improve code quality, stability and supportability. ➢ Ability to write modularized reusable code components.

➢ Able to mentor new members for on boarding to the project.

➢ Experience in designing data warehouse/data mart.

➢ Leading the client calls to flag off any delays, blockers, escalations and collate all the requirements

➢ Experience with any RDBMS preferably SQL Server and must be able to write complex SQL queries.

➢ Proficient in identifying data issues and anomalies during analysis.

➢ Strong analytical and logical skills.

➢ Must be able to comfortably tackle new challenges and learn.

➢ Must have strong verbal and written communication skills.