Lead Data Engineer

 

Key Responsibilities

In this role, you’ll partner with our Fortune 100 clients and work within our Data Science team as a Data Engineer. You will clean, transform, and analyze vast amounts of raw data from various systems to provide ready-to-use data to our Data Scientists. 

 

This involves both ad-hoc requests as well as data pipelines that are embedded in our production environment.

 

Additionally, you will: 

• Gain experience using different big data and Cloud based Data technologies. 
• Experience operating distributed systems of data extraction and processing of large data sets. 
• Identify, analyze, and interpret trends or patterns in complex data sets.  
• Design data processing pipelines. 
• Write good documentation with all code. 
• Develop strong communication skills with a proven success communicating with users, other tech teams.

 

Educational Qualifications

Bachelor/Master’s degree in Computer Science, Computer Engineering, quantitative studies, such as Statistics, Math, Operation Research, Economics and Advanced Analytics 

 

Requirements

• 4 - 8 years of hand on experience of developing python-based solutions/applications.
• Strong experience in Python, preferably with a good low level designing and programming concepts.
• Strong experience in any cloud platform preferably AWS or GCP. 
• Strong experience of working with API’s and integrating multiple applications together.
• Experience with SQL and relational databases.
• Solid architectural understanding of NOSQL databases.
• Good understanding of cloud security best practices.
• Excellent understanding of OOP concepts with experience applying it in enterprise programming.
• Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)
• Good Git version control practices
• Proficient with Microsoft office suite (Excel, Word, PowerPoint)
• Experience of working with Data Warehousing Technologies like BigQuery, Redshift, or snowflake is a plus.
• Experience in using Business Intelligence tools such as: Tableau, Power BI for business applications is a plus.
• Self-starter and have strong interpersonal skills
• Strong analytical, problem solving, and troubleshooting abilities.
• Flexibility to adapt to a variety of engagement types, working hours and work environments and locations.