SHIKHA SHARMA

Data Engineer

New Delhi, IN.

About

Highly skilled Data Engineer with 3+ years of experience in designing, developing, and optimizing Azure-based ETL pipelines using ADF, Databricks (PySpark), and SQL. Proven ability to drive significant performance improvements, evidenced by reducing pipeline runtime by 50% and optimizing PySpark jobs by 40%, while delivering near real-time analytics. Expertise spans end-to-end pipeline ownership, including incremental loads, robust data quality checks, and proactive monitoring, ensuring scalable and reliable data workflows.

Work

CheqIT Pvt Ltd

|

Data Engineer

Pune, Maharashtra, India

→

Summary

Led the migration of FMCG Sales Analytics workflows from Alteryx to Azure Databricks, significantly enhancing data ingestion, processing, and real-time insights for 5+ regional sources.

Highlights

Led migration of 15+ Alteryx workflows to Azure Data Factory (ADF) and Databricks pipelines, automating ingestion of ~500GB/day of sales and inventory data from 5+ regional sources.

Replaced multiple Alteryx macros with parameterized ADF datasets, reducing maintenance effort by 80% and enabling a single reusable pipeline.

Optimized Databricks workflows via `repartition()` and `filter pushdown`, converting Alteryx transformations into PySpark jobs for data cleaning, which improved transformation performance by 40%.

Implemented incremental loads with watermarking, decreasing data ingestion runtime by 50% (from 6 hours to 3 hours), and established robust pipeline monitoring & alerting with Azure Monitor + MS Teams, reducing failure recovery time from 2 hours to 20 minutes.

Achieved significant operational improvements, reducing manual interventions by 80%, data failures by 30%, and reporting delays by 85% for near real-time FMCG sales insights.

IBM Pvt Ltd

|

Application Developer

Bengaluru, Karnataka, India

→

Summary

Designed and deployed robust SQL Server ETL workflows using SSIS, efficiently handling millions of daily transactions and optimizing database performance.

Highlights

Designed and deployed SQL Server ETL workflows utilizing SSIS to handle millions of daily transactions efficiently.

Optimized SQL queries and stored procedures, enhancing data retrieval speed by 30% and supporting faster reporting.

Accenture Pvt Ltd

|

Associate Software Engineer

Bengaluru, Karnataka, India

→

Summary

Contributed to enhancing application reliability and code quality by implementing JUnit test cases and resolving SonarQube issues within agile development cycles.

Highlights

Improved application reliability by implementing JUnit test cases, effectively reducing bugs by 30%.

Resolved SonarQube issues across agile sprints, significantly increasing overall code quality.

Education

Truba Institute of Engineering and Information Technology

Bhopal, Madhya Pradesh, India
→
Bachelor of Engineering Information Technology

Languages

English

Certificates

Azure Fundamentals (AZ-900)

Microsoft Certified

Storytelling Using Power BI

Analytics Vidhya

Storytelling Using Tableau

Analytics Vidhya

Microsoft Excel

Microsoft

SQL for Data Science

Online Course

Skills

Software Development & Testing

JUnit, SonarQube, Application Reliability, Agile Methodologies.

Cloud & Big Data

Azure Data Factory, Azure Databricks, ADLS Gen2, Data Migration, Workflow Automation, Pipeline Development.

Programming & Scripting

Python, PySpark, SQL.

Databases

SQL Server, Azure SQL Database.

ETL & Data Integration

ADF Pipelines, Parameterization, Incremental Loads, Schema Drift Handling, SSIS.

Big Data Processing & Optimization

PySpark Transformations, Partitioning, Caching, Optimization, Filter Pushdown, Performance Tuning, Real-time Analytics.

DevOps & Monitoring

Git, Azure DevOps, Azure Monitor, Log Analytics, MS Teams Alerts, Pipeline Monitoring.

Data Quality & Governance

Data Cleaning, Data Quality Checks, Error Handling, Data Consistency, Watermarking.

Reporting & Visualization

Power BI, Tableau, Microsoft Excel.