Senior Data Engineer - Kerala candidates only

Overview

About the Role

We are seeking a highly skilled Senior Data Engineer to join our team and drive the development of our data platform. This role will focus on building scalable data pipelines, optimizing data storage, and ensuring data security and governance. You will collaborate with product teams and other engineering teams to build efficient, high-performance data solutions.

Key Responsibilities

• Design, develop, and optimize scalable ETL/ELT pipelines using Databricks and Apache Spark for both batch and streaming data processing.
• Develop and maintain data models that support analytical and operational workloads.
• Collaborate with product teams and other engineering teams to understand data requirements and design robust solutions.
• Ensure data security, access control, and governance frameworks are implemented and adhered to.
• Monitor data quality and performance, identifying areas for optimization and improvement.
• Ensure code quality and implement test cases on data pipeline codes.
• Follow the latest trends/updates in the data side across the industry and validates its applications on improving our data platform.

Required Skills

• 5+ years of experience in data engineering with a focus on large-scale data processing.
• Expertise in Databricks, Apache Spark, and SQL.
• Strong experience in batch and streaming data processing.
• Strong experience in Python and PySpark for data processing and automation.
• Strong experience in data modeling for both Operational and Analytical data.
• Strong problem-solving skills and ability to optimize performance in distributed data environments.
• Hands-on experience with data security, access control, and governance frameworks.
• Experience with Apache Druid/ MSSQL/ MySQL/ MongoDB/ Oracle/ PostgreSQL.
• Good to have experience with Graph Databases to manage complex relationships within data
• Good to have experience in working with Open-Source community.
• mail to - thasleema@qcentro.com / hr@qcentro.com

Job Types: Full-time, Permanent

Schedule:

• Monday to Friday

Experience:

• Data engineering: 5 years (Required)

Location:

• Kochi, Kerala (Required)

Work Location: In person