Data Engineer
Remote, India
Posted on 18 Oct, 2021
406 views
4 applications
ContractRemote/OnsiteVerified
Job Requirements
Experience
2 - 5 years
No. of Positions
4
Duration
3-6 months
Skills
pysparkPythonsparkhadoophiveHDFS
Job Description
Skills Required: PySpark, Python, SQL, Spark SQL, HIVE, HDFS, Spark
Roles and Responsibilities:
- Responsible for developing and maintaining applications with PySpark
- Contribute to the overall design and architecture of the application developed and deployed.
- Performance Tuning wrt to executor sizing and other environmental parameters, code optimization, partitions tuning, etc
- Interact with business users to understand requirements and troubleshoot issues.
- Implement Projects based on functional specifications.
Must-Have Skills:
- Good experience in Pyspark - Including Dataframe core functions and Spark SQL
- Good experience in SQL DBs - Be able to write queries including fair complexity.
- Should have excellent experience in Big Data programming for data transformation and aggregations
- Good at ETL architecture. Business rules processing and data extraction from Data Lake into data streams for business consumption.
- Having exposure to SCALA and Apache Airflow is desirable
- Good customer communication.
- Good Analytical skills
Interested in this job?
Job Overview
Location
Remote, India
Job Type
Contract
Rate
$ 14 / Hourly
Work From
Remote/Onsite
Start Date
24 Oct, 2021