Jayaprakash

PROFESSIONAL SUMMARY

• Strong programming experience on the Data Engineering side of Java and Python.

• Strong hands-on experience in GCP with data analysis, creating data pipelines and orchestration on GCP tools like Compute Engine, Dataflow, Cloud Dataproc, Pub/Sub, Composer, Storage, Cloud SQL, Big Query and Cloud functions.

• Experience building or maintaining ETL processes from a wide variety of data sources using SQL, JAVA, Python and Shell Scripting based on the requirement.

• Building and optimizing ‘big data’ data pipelines, architectures, and data sets.

• Proficient in creating tables, views, stored procedures, datasets in Big Query. Strong hands-on experience on SQL and PL-SQL.

• Expertise in creating pipelines in google dataflow to handle batch and streaming data.

• Created DataMart’s using Big Query job config and Airflow big query operators as per the reporting need.

• Created Airflow dags to schedule batch jobs, trigger Big Query operator to refresh the DataMart’s. Utilized Airflow variable to control the Airflow job tasks and the logic.

• Expert in working with cloud Dataflow, Pub/Sub to replicate data real-time from source system to GCP Big Query.

• Good knowledge on GCP service accounts, billing projects, authorized views, datasets, GCS buckets and gsutil commands.

• Created partition tables for the daily and monthly data snapshots

• Experience on build tools Gradle, Maven, Ant and Jenkins

• Experience on version control tools CVS, SVN, GIT and release management tools JIRA.

• Working experience developing and deploying software on platforms Windows, Linux using tools Putty, Win SCP.

• Experienced in Agile, Scrum based development and good at working on sprints.

• Experience in data visualization tools in integrating Bigquery with Data Studio and Grafana.

• Identifying the data quality issues and has worked with creating a framework to resolve data quality issues.

• Build processes supporting data transformation, data structures, metadata, dependency, and workload management access database. • Developed and implemented JUnit Classes to perform Unit Testing and PMD