Swapnil, Data Engineer | Bench Resource on Contract | September 2024

Swapnil (RID : a49xldv8vi0f)

Data Engineer

Location : Jaipur

Experience : 7 Year

Rate: $26 / Hourly

Availability : Immediate

Work From : Offsite

Category : Information Technology & Services

Shortlisted : 1

Total Views : 196

Key Skills

Hadoop Azure Data Engineer Big Data Ecosystems Python Informatica Power BI

Discription

SWAPNIL

DATA ENGINEER

TECHINCAL EXPERTISE:

Professional with 7+ years of experience in Big Data Ecosystems like (HDFS, YARN, Hive, Impala, Spark) and Cloud technologies like (Azure Databricks, Azure Data factory, Azure Data Lake Storage, Azure Synapse) , Python, Informatica, SQL, PowerBI.

Installation, Configuration, and Administration of Hadoop distributions like Cloudera (CDH)
Experience in cluster deploying, performance tuning, and administering and monitoring the Hadoop ecosystem
Knowledge of Hadoop Ecosystem - HDFS, Yarn, MapReduce, Hive, Hue, Sentry, Impala, Zookeeper, Spark.
Hands on Experience in Cloud Technologies like Azure Databricks, Azure Data Factory, Azure Synapse Analytics, Azure Blob Storage, Azure Data Lake Storage.
In-depth understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames.
Hands on experience in various Big Data application phases like data ingestion, data analytics.
Expertise in using Spark-SQL with various data sources like JSON, Parquet, and Hive.
Experience in using Hadoop distribution like Cloudera, Azure Databricks.
Experience in transferring data from RDBMS to HDFS and Hive table using Azure Data Factory.
Experience in creating tables, partitioning, bucketing, loading, and aggregating data using Hive/Impala.
Uploaded and processed terabytes of data from various structured and semi structured sources into HDFS.
Worked on Informatica Designer Components, created Tasks, Sessions and Workflow using Workflow manager and Workflow Monitor to monitor the Workflows.
Extensive experience in extraction, transformation and loading of data directly from different heterogeneous source systems like flat files, Oracle, Netezza.
Good knowledge of different schemas (Star and Snowflake) to fit reporting query and business analysis requirements.
Developed and supported Informatica mappings with transformation Filter, Router, Expression, Joiner, Aggregator, Lookup, Union, Sequence generator etc.
Experience in Slowly Changing Dimensions like SCD1 and SCD2.
Involved in Technical and Business meetings with internal teams and high-level management.
Hands on experience in Analysis, Design, Coding and Testing phases of Software Development Life Cycle (SDLC).
Having good Knowledge of SQL.
Worked extensively in team-oriented environment with strong, analytical, interpersonal and communication skills.

MAR, 2020 – PRESENT

Project: Barclays Collect Roles and Responsibilities:

Worked on requirement gathering, analysis and translating business requirements into technical design with Hadoop Ecosystem.
Extensively used Hive, Spark optimization techniques like Partitioning, Bucketing, Map Join, parallel execution, Broadcast join and Repartitioning.
Created partitioned, bucketed Hive tables, loaded data into respective partitions at runtime, for quick downstream access.
Experienced in handling large datasets using Partitions, Spark in Memory capabilities, Broadcasts in Spark, Effective & efficient Joins, Transformations and other during ingestion process itself.
Analyzed asset matrix (mapping document) and used to develop PySpark projects in PyCharm.
Python modules are built in PyCharm where the entire PySpark logic is developed to achieve business requirements.
Used Tivoli to schedule Spark jobs. Also used to trigger spark jobs in both client and cluster mode in lower environments.

Project: SOLAR

Roles and Responsibilities:

Worked on multiple Hadoop clusters with 150 nodes on Cloudera distribution 5.x, 6.x
Currently working as Hadoop Administrator and responsible for taking care of everything related to the clusters total of 100+ nodes ranges from Non-PROD to PROD clusters.
Worked on Kerberized Hadoop cluster.
Involved in upgrading CM and CDH.
Worked on the cluster, commissioning & decommissioning of Data Nodes.
Involved in communication with Cloudera team for cluster tuning.
Monitoring Hadoop cluster using Cloudera Manager and make sure that all the services are up and running.
Involved in resolving day to day Incidents and implementing change related activities making sure to follow proper ITIL process.
Setup data authorization roles for Hive and Impala using Sentry.
User management, involving user creation, granting permission for the user to various tables and database, giving group permissions.
Managed and reviewed Hadoop log files, fil

Matching Resources

Python , Data Analysis , MySQL , Excel

Java , Big Data , Hadoop , Hbase , Hive ...

Java , Big Data , Hadoop Admin , Hadoop , Hive ...

Java , Big Data , Hadoop Admin , Hadoop , Hbase ...

PROJECT MANAGEMENT SKILLS , PHP7.3 Python3.7, Angular JS

PHP YII2 Framework, Python, M , Core PHP MYSQL, Zend framewor , jQuery JavaScript, CSS, HTML,

ASP.NET MVC , LINQ , Jquery , Web API , MS SQL Server ...

Python , OpenCv , Keras , Tensorflow , R Programming ...

Microsoft 365 , SharePoint , Ms Power Automation , Power BI , power App ...

Python , django , Web API

Python , django , HTML , CSS , Pycharm ...

django , Python , machine learning , AWS , REST API ...

Python , django , Java Script , SQL , PostGre SQL ...

django , Python , flask , HTML , CSS ...

Python , django , MS-SQL , MySQL , Flask ...

Python , HTMLXML. , python 3.8Django 2.2 , MySQL , JavaScript

Python Django and Rest-Api T , HTMLCSS,JavaScript, JQuery , , MongoDb , MySQL , Sqlite3 ...

Python , django , html , css , sql ...

Python , django , MongoDB , PostGre SQL , MySQL ...

python , django , html , css , JavaScript ...

python , django , html , CSS , Java ...

Python , django , html , css , Bootstrap ...

Python , django , html , css , Bootstrap ...

python , django , htmls , css , Bootstrap ...

Python , django , MySQL , HTML , css ...

Hadoop 2.7.0 , MapReduce , Flume , Pig , Sqoop ...

Sql , NoSql , PHP , Python , Core PHP ...

Java , HTML , Python

Java , Oracle , Python , JavaScript

Hadoop Admin , AWS or Azure

C++ , Core Java , Python , MySQL , SQL ...

Sr. Java Full Stack

Core Java , Advanced Java , Spring MVC , MySQL , Hive ...

Azure , .AWS , AWS Cognito , S3 , API Gateway ...

Codepipline , Azure , Kubernetes , Jenkins , Terraform ...

Reactjs , React-redux , Javascript , Jquery , Nodejs ...

Python , R , SQL , Java , MySQL ...

Python , Database

NodeJs , Python , Laravel , JavaScript , REST API ...

Node.JS , Angular JS , JavaScript , Ajax , PHP ...

Docker , Kubernetes , AWS or Azure , Jenkins , Python ...

azure , azure data factory , adf , Power BI , mssql ...

Node.JS , Python , Express .JS , PHP , PHP GraphQL ...

Core Java , Python , Junit , JavaScript , TypeScript ...

Python , R Programming , Business Objects , SAS , MATLAB ...

Python , django , Angular 2 , Android , HTML ...

PostGre SQL , AWS , Docker , Kubernetes , Jenkins ...

React JS , Node.JS , Native , Manual QA , Automation Testing ...

Node.JS , Java , Python , SQL , Maven ...

React JS , Node.JS , JavaScript , MongoDB , express ...

java , python , nodejs

Java , JavaScript , Python , Spring Boot , Angular JS ...

Python , django , MySQL , Git or SVN , reactjs

Python , machine learning , deep learning , MySQL , SQL

Python , Automation Testing , DevOps

Python , Angular JS , django

Python , MySQL , SQL , django

Python , django , Node.JS , MySQL , SQL

hadoop , spark , pyspark , Python , scala ...

Python , flask , django

Python , Machine learning , Deep Learning , SQL , MySQL

Power BI , SQL , SAP , big data , .Net ...

Python , Pytorch , Data Science , Machine Learning Algorithms , Deep Learning ...

Python , Computer Vision , NLP , Deep learning , Data Science ...

Java script , Angular 10 , Node JS , Python , React JS ...

Power BI , Database Design , Data Model

Power BI , Power BI Map Visuals , Dimensional Modeling , Gateway migration

Python , Django , AWS , Flask , HTML ...

Power BI , .Net Core , M Query , Developed Websites , .Net MVC ...

SharePoint Online , SharePoint , Sharepoint 2016 , SharPoint 2013 , PHP ...

Java , C , C++ , Python , JSP ...

SharePoint Online , Power Automates , Power Bi , MS Forms , Teams ...

Bootstrap , Sharepoint , SPFx , React , Node.JS ...

Java , Python , JavaScript , Selenium WebDriver , Automation Testing ...

Python , Ansible , Jenkins , Git , AWS Cloud ...

React JS , React Native , Redux , Firebase , SQL ...

Python , Numpy , SVM , Clustering -K-Means , Regression ...

Python , SQL , Keras , Pandas , Linear ...

Python , MS SQL Server , Django , REST API , DJjango ...

Blockchain , Scrum-Scaled Agile Framework , Golang , Node.JS , Python ...

Node.JS , Python , Java , Struts1.0 , Hibernate ...

Machine Learning , Data Analysis , Python , Java , SQL

Python , django , JavaScript , XML , JSON

ERPNext , Python , CRM , frappe , ERP ...

Python , django , Java

SQL Server Integration Services , .Net , AWS or Azure , ADW , Azure ...

Python , JavaScript , SQL , Numpy , Pandas ...

React Native , Python , Django , DRF

React Native , Redux , JavaScript , AWS , Python

SDLC , STLC , Black box testing , Selenium Web driver. , Python ...

Microsoft Power BI , Microsoft SQL Server. , SQL T- SQL. , SQL Server , Power BI ...

Python , DevOps , Machine Learning , Flask , Django

React Native , Python Scripting , Git , AWS , Apache ...

Python , JavaScript , XML , JSON , TwiML ...

Python , Machine Learning , Bash , Pytorch , Image Processing ...

NodeJS , PHP , Laravel , Cakephp , Pythonwith Django ...

NodeJS , PHP , Laravel , Cakephp , Agile practices ...

Angular , Node.JS , Vue.JS , Python , HTML ...

JavaScript , Typescript , Express JS , Nest JS , PostgreSQL ...

Drupal , Python , Acquia , Mongo DB , MYSQL ...

My Project History & Feedbacks

Please Update your Package

Company contact view balance is zero.

Custom Plan

Contact View	Amount	Days	Action
{{list.companycontact}}	{{list.amount}}	{{list.expiry}}	Pay Now

Monthly Plan

Name	Amount	Days	Action
Premium	15000	30	Pay Now
Enterprise	40000	30	Pay Now

Copyright© Cosette Network Private Limited All Rights Reserved

Loading…

stuff goes in here!