Sign In
Looking for talent?
Check out our hiring section
Login to your account
Remember me?
Login
Forgot password?
Not a user yet?
Click here to register.
LOADING
Select Login
Uploaded File
Sai
saikirandata2@gmail.com
651-235-6664
Minneapolis, MN 55443
Sr. Data Engineer
8 years experience
W2
0
Recommendations
Average rating
179
Profile views
Summary
5+ years of working experience in Data Engineering, Data Pipeline Design, Development, and Implementation.
Strong experience in writing scripts using Python API, PySpark API and Spark API for analysing the data.
Experience in building data pipelines using Azure Data Factory, Azure Databricks, and loading data to Azure Data Lake, Azure SQL Database, Azure SQL Data Warehouse to control and grant database access.
Experience with both AWS, AWS Glue & Azure (Data Factory, Databricks & Data Lake) cloud.
Good experience with Azure services like HDInsight, Stream Analytics, Active Directory, Blob Storage, Cosmos DB, Storage Explorer.
Developed Spark Applications that can handle data from various RDBMS (MySQL, Oracle Database) and Streaming sources.
Implemented different Python libraries Pandas, NumPy, matplotlib, python-twitter, Panda's data frame in various tasks.
Hands On experience on Spark Core, Spark SQL, Spark Streaming and creating the Data Frames handle in SPARK with snowflake.
Proven proficiency in leveraging Google Cloud Platform's suite of data services, including BigQuery, Cloud Storage, and Dataflow, to design and implement robust and scalable data solutions.
Experienced working on Hadoop/ Bigdata Framework and its ecosystem like HDFS, MapReduce, Yarn, Spark, Hive, impala, Sqoop and Oozie.
Hands-on use of Spark and Scala API's to compare the performance of Spark with Hive and SQL, and Spark SQL to manipulate Data Frames in Scala.
Experienced in scripting with Python (PySpark), Scala and Spark-SQL for development, aggregation from various file formats such as XML, JSON, CSV, Parquet.
Experience in developing Data pipelines using AWS services including EC2, S3, Redshift, Glue, Lambda functions, Step functions, CloudWatch, SNS.
Expertise in migrate an existing on-premises application to AWS. Used AWS services like EC2 and S3 for small data sets processing and storage
Proficient with Spark Core, Spark SQL for processing and transforming complex data using in-memory computing capabilities written in Python. Spark to improve efficiency of existing algorithms using Spark Context, Spark SQL, Data Frame, Pair RDD's and Spark YARN. databases, SQL queries, establishing data relationship and performance tuning for data extract.
Proficient in DevOps tools such as Jenkins and Selenium, with expertise in Apache Spark, SparkSQL, Spark DataFrames, and Spark RDDs, responsible for efficiently managing data from multiple sources.
Expertise in use of Sqoop to migrate data between RDBMS, NoSQL databases and HDFS.
Expertise in creating Kubernetes cluster with cloud formation templates and PowerShell scripting to automate deployment in a cloud environment.
Hands-on experience on Ad-hoc queries, Indexing, Replication, Load balancing, Aggregation in MongoDB
Experience in various methodologies like Waterfall and Agile.
Experience
Edit Skills
Non-cloudteam Skill
Education
Master's in Information Technology Management
Concordia university
Record has not been verified.
Skills
Data Engineering
2024
6
ETL
2024
6
Hadoop
2024
6
PostgreSQL
2024
6
PySpark
2024
6
Python
2024
6
Spark
2024
6
SQL
2024
6
API Development
2024
5
Big Data
2024
5
Data Integration
2024
5
Data Science
2024
5
Flume
2024
5
Hive
2024
5
Metadata
2024
5
Performance Tuning
2024
5
Pipeline
2024
5
Snowflake
2024
5
Sqoop
2024
5
Talend Studio
2024
5
Azure Data Factory
2024
4
Data Warehousing
2024
4
Git
2024
4
MS Azure
2024
4
Apache
2024
3
Azure Storage
2024
3
Data Lakes
2024
3
Data Validation
2024
3
HDFS
2024
3
Informatica
2024
3
Machine Learning
2024
3
MapReduce
2024
3
MS Power BI
2024
3
MySQL
2021
3
Pig
2024
3
T-SQL
2024
3
AWS
2021
2
AWS Redshift
2021
2
AWS S3
2021
2
Data Integrity
2021
2
Data Management
2021
2
Data Mapping
2021
2
DBA
2021
2
JDBC
2021
2
JSON
2021
2
Oracle
2021
2
Oracle Cloud
2021
2
SQL Server
2021
2
Stored Procedure
2021
2
Tableau
2021
2
Triggers
2021
2
XML
2021
2
Azure Data Bricks
2019
1
Business Analysis
2019
1
CSS
2019
1
Data Analysis
2019
1
ETL Developer
2019
1
HTML
2019
1
React.js
2019
1
REST
2019
1
Agile Methodology
0
1
AWS CloudFormation
0
1
Cloudwatch
0
1
Data Replication
0
1
impala
0
1
Jenkins
0
1
Load Balancing
0
1
MongoDB
0
1
Oozie
0
1
PowerShell
0
1
Scripting
0
1
Selenium
0
1
