Azure data Engineer

Overview

Azure Data Engineer

Experience: 7-12 Years
Location: Bangalore, Hyderabad, Pune

Job Description:

We are seeking a highly skilled Azure Data Engineer with 7-12 years of experience to design, build, and optimize data engineering solutions on Azure. The ideal candidate must have strong hands-on experience in Azure Databricks, PySpark, and Data Engineering with expertise in Databricks Jobs, Scheduling, Orchestration, and REST API Integration.

Key Responsibilities:

• Design, develop, and manage scalable data pipelines using Azure Databricks and PySpark.
• Implement Databricks Jobs, Scheduling, Orchestration, and Workflow Automation.
• Develop efficient and optimized data processing frameworks using Delta Lake Architecture.
• Integrate Databricks with Azure Data Services using REST APIs.
• Work with Azure Data Factory (ADF), Azure Data Lake (ADLS), and other Azure components for ETL/ELT processes.
• Ensure data quality, security, and performance optimization across all solutions.
• Collaborate with business stakeholders and cross-functional teams to implement data-driven solutions.
• Optimize Spark jobs for better performance and cost efficiency.

Key Skills & Experience:

✔ experience in Data Engineering.
✔ Strong hands-on expertise in Azure Databricks and PySpark.
✔ Proficiency in Databricks Job Scheduling, Orchestration, and Workflow Management.
✔ Experience with Databricks REST API integration.
✔ Expertise in Delta Lake Architecture for data management and optimization.
✔ Strong knowledge of Azure Data Factory (ADF), Azure Data Lake (ADLS).
✔ Good understanding of ETL processes, data modeling, and data warehousing concepts.
✔ Strong proficiency in SQL for data processing and transformation.
✔ Experience in performance tuning and optimization of big data solutions.

Preferred Qualifications:

✅ Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.
✅ Azure certification (DP-203: Azure Data Engineer Associate) is a plus.
✅ Experience with CI/CD for Data Pipelines using Azure DevOps is preferred.
✅ Knowledge of MLflow, Feature Store, or Databricks Unity Catalog is an advantage.

Job Types: Full-time, Permanent

Pay: ₹1,000,000.00 - ₹1,800,000.00 per year

Benefits:

• Health insurance
• Provident Fund

Schedule:

• Day shift

Experience:

• Azure Databricks: 3 years (Required)
• Azure data lake: 3 years (Required)
• Azure data factory: 3 years (Required)
• Pyspark: 4 years (Required)

Work Location: In person