Jayamaalathy

EXPERIENCE Welkinz (Remote, Netherlands) 05/2023 – 10/2023 Data Engineer

 Spearheaded design and implementation of a robust data pipeline, with a focus on efficiency and scalability.

 ETL Operations: Executed essential Extract, Transform, Load (ETL) operations to ensure the seamless flow and transformation of data within the pipeline.

 Agile Methodology: Successfully operated within an Agile framework, contributing to iterative development cycles and quick responses to changing project needs.

 Data Warehousing: Acquired hands-on experience with data warehousing concepts, thereby enhancing my understanding of effective data storage and retrieval.

 Big Data Technologies: Demonstrated proficiency in a range of big data tools and technologies, including Apache Spark, PySpark, and AWS services such as S3, EMR, EC2, Lambda, Step Function, Glue Jobs, Athena, and DynamoDB.

 Performed the integration of APIs to enhance data flow between internal systems, resulting in improvement.

 Developed Python scripts and applications for data analysis, automating repetitive tasks and improving data processing workflows.

 Orchestration and Scheduling: Utilized big data orchestration and scheduling tools like Airflow to enhance the coordination and automation of data workflows.

 Containerization with Docker: Implemented and managed containerization using Docker for efficient deployment and runtime environments, ensuring consistency across different stages of development.

 CI/CD Integration: Integrated Docker into the Continuous Integration/Continuous Deployment (CI/CD) pipeline, contributing to reliable and streamlined application delivery.