Senior Databricks Engineer (Azure Data Engineer)

Overview

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines using Azure Databricks.
• Build and optimize ETL/ELT processes for structured and unstructured data.
• Collaborate with data scientists, analysts, and business stakeholders.
• Integrate Databricks with Azure Data Lake, Synapse, Data Factory, and Blob Storage.
• Develop real-time data streaming pipelines and manage data models/data warehouses.
• Optimize performance, manage resources, and ensure cost efficiency.
• Implement best practices for data governance, security, and quality.
• Troubleshoot and improve existing data workflows.
• Contribute to architecture and technology strategy.
• Mentor junior team members and maintain documentation.

Required Skills and Qualifications:

• Bachelor’s/Master’s in Computer Science, IT, or related field.
• 5+ years of Data Engineering experience (minimum 2+ years with Databricks).
• Strong expertise in Azure cloud services (Data Lake, Synapse, Data Factory).
• Proficiency in Spark (PySpark/Scala) and big data processing.
• Experience with Delta Lake, Structured Streaming, and real-time pipelines.
• Strong SQL skills and understanding of data modeling and warehousing.
• Familiarity with DevOps tools: CI/CD, Git, Terraform, Azure DevOps.
• Excellent problem-solving and communication skills.

Preferred Qualifications:

• Databricks Certified (Associate/Professional) preferred.
• Experience with machine learning workflows on Databricks.
• Knowledge of data governance tools like Purview.
• Experience with REST APIs, Kafka, Event Hubs, etc.
• Cloud performance tuning and cost optimization experience.

Job Type: Full-time

Pay: ₹1,200,000.00 - ₹1,500,000.00 per year

Benefits:

• Flexible schedule

Schedule:

• Monday to Friday

Supplemental Pay:

• Overtime pay

Education:

• Bachelor's (Required)

Experience:

• Azure: 5 years (Required)
• Big data: 5 years (Required)
• Scala: 5 years (Required)

Work Location: In person