Data Engineer

Overview

Hyderabad / Remote (with quarterly visits)
Notice Period: 30 Days
Experience: 5+ Years

Responsibilities:
Design and develop business-critical backend systems using high-quality data pipelines.
Assemble large, complex datasets that meet functional and non-functional business requirements.
Build infrastructure for optimal data extraction, transformation, and loading (ETL) from various data sources using Azure Databricks and Azure Data Factory (ADF).
Develop a cloud-native, real-time stream processing and data lake platform that scales to massive data volumes.
Construct data pipelines that deliver actionable insights into customer acquisition, operational efficiency, and key business performance metrics.
Automate data pipelines using Logic Apps and DevOps practices.
Conduct root cause analysis on external and internal data processes to identify improvement opportunities.
Embrace continuous improvement, always seeking to enhance existing systems with better solutions.
Collaborate with stakeholders to design and code large-scale batch and real-time data pipelines on Azure.
Perform code reviews, emphasizing readability, extensibility, and testability.

Requirements:
Bachelor’s degree in Computer Science or a related field.
5+ years of experience as a Data Engineer.
Proficiency in Azure big data tools, including Azure Databricks, Azure Synapse, ADLS, and Azure Data Factory (ADF).
Strong knowledge of relational SQL and Azure Synapse.
Excellent problem-solving and analytical skills, especially in handling structured and unstructured datasets using Azure big data tools.
Hands-on experience with data pipeline and workflow management tools such as ADF and Logic Apps.
Familiarity with Azure cloud services, including Azure SQL DB, Azure Synapse, and Azure Databricks.
Proficiency in Python for scripting and automation.