Azure Data Engineer

Overview

Responsibilities

• Design and develop business-critical backend systems using stream processors and high-quality data pipelines.
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Azure ‘big data’ technologies.
• Build a cloud-native, real-time stream processing & data lake platform that scales into the Zettabytes and beyond.
• Build a data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
• Ability to automate data pipelines using DevOps tools.
• Perform root cause analysis on external and internal processes and data to identify opportunities for improvement.
• Build processes that support data transformation, workload management, data structures, dependency, and metadata.
• Adopt for change, always open to replacing what you built yesterday with something better today.
• Work with all stakeholders to design and code large-scale batch and real-time data pipelines on Azure.
• Perform code reviews, and lead by example on code refactoring for readability, extensibility, and testability.Requirements
• A bachelor's degree in computer science or equivalent
• 5+ years of experience as a Data Engineer,
• Experience with Azure big data tools: Azure Databricks, Azure Synapse, HDInsight, ADLS
• Experience with relational SQL and NoSQL databases.
• Excellent problem solving and analytic skills associated with working on structured and unstructured datasets using Azure bigdata tools.
• Experience with data pipeline and workflow management tools: ADF and Logic Apps
• Experience with Azure cloud services: VM, Azure Databricks, Azure SQL DB, Azure Synapse
• Experience with stream-processing systems: Azure Streams
• Experience with scripting languages Python. and have strong knowledge in developing Python notebooks.

Job Type: Full-time

Pay: ₹1,400,000.00 - ₹1,500,000.00 per year

Schedule:

• Day shift

Ability to commute/relocate:

• Pune, Maharashtra: Reliably commute or planning to relocate before starting work (Required)

Education:

• Bachelor's (Required)

Experience:

• azure: 5 years (Preferred)
• ETL: 5 years (Preferred)
• data bricks: 5 years (Preferred)

Work Location: In person