Lead Data Engineer

Job Description:

We are seeking a highly skilled Lead Data Engineer to design, develop, and optimize data pipelines and architectures. The ideal candidate should have strong expertise in Azure, Python, and Data Lake technologies and a good understanding of MS Fabric. This role requires hands-on experience in data engineering, ETL processes, and data pipeline development, working with structured and unstructured data.

Key Responsibilities:

• Design, develop, and implement scalable and efficient data pipelines using Azure services.
  
• Work with Azure Data Lake, Azure Synapse, and other Azure data services for data processing and storage.
  
• Develop and maintain ETL processes using Python and other data transformation tools.
  
• Optimize data architectures to ensure high performance, reliability, and security.
  
• Collaborate with data scientists, analysts, and business teams to understand data requirements.
  
• Implement best practices in data governance, security, and compliance.
  
• Work with large-scale datasets and ensure efficient data processing and transformation.
  
• Stay updated with emerging data engineering technologies and Azure advancements.
  

Required Skills & Qualifications:

• 8-10 years of experience in Data Engineering.
  
• Strong proficiency in Azure Data Services (Azure Data Lake, Azure Synapse, Azure Data Factory, etc.).
  
• Expertise in Python for data processing and scripting.
  
• Experience working with big data technologies and modern data architectures.
  
• Proficiency in designing and implementing ETL/ELT workflows.
  
• Good to have experience with Microsoft Fabric.
  
• Hands-on experience in SQL, performance tuning, and data modeling.
  
• Strong problem-solving skills and ability to work in an agile environment.