Senior Data Engineer - Python/PySpark

About The Company

Loyalytics is a fast-growing Analytics consulting and product organization based out of Bangalore.

We work with large retail clients across the globe helping them monetize their data assets through our consulting assignments and product accelerators.

We are a young dynamic team of 100+ analytics practitioners working on some of the most cutting-edge tools and technologies.

Who We Are

• Technical team : A team full of data scientists, data engineers and business analysts who work with 1M+ data points every day.
• Market Size : Massive multi-billion $ global market opportunity.
• Leadership : Combined experience of 40+ years of experience in the industry.
• Customers : Word-of-mouth and referral driven marketing to acquire customers like big retail brands in GCC regions like Lulu, GMG, among others (Strong product-market fit).
• What makes us stand apart: 8 years old bootstrapped and 100+ people company that is still hiring.
  
  

Role Overview

As a Data Engineer, you will play a key role in leading the end-to-end data engineering process, including requirements gathering, design, development, code optimization, production deployment, and ongoing support.

You will also be responsible for mentoring and guiding junior resources.

This position requires a blend of technical expertise in cloud platforms, big data technologies, and a solid understanding of finance to ensure that our data solutions meet the business's objectives and expectations.

Responsibilities

• Lead the development process end-to-end, including requirements gathering, design, development, testing, and deployment.
• Optimize and maintain code to ensure that it is performant, scalable, and easily maintainable.
• Deploy and monitor solutions in production environments, ensuring systems are reliable, secure, and meet business requirements.
• Mentor and guide junior resources, helping them grow technically and professionally.
• Collaborate with cross-functional teams, ensuring data solutions align with the broader business strategy and requirements.
• Work closely with finance teams, leveraging your understanding of financial processes to ensure that the data systems are in line with business needs.
• Ensure high-quality data management, including data governance, data quality, and performance tuning.
• Troubleshoot and provide ongoing support to address issues and ensure smooth functioning of the data systems.
  
  

Required Skills & Experience

• 4-5 years of hands-on experience in data engineering, working with Azure Databricks, PySpark, Python, Azure Data Factory, and ADLS.
• Solid experience with ETL/ELT pipeline development, ensuring seamless data extraction, transformation, and loading.
• In-depth knowledge of PySpark and Databricks for data processing and optimization.
• Hands-on experience in Azure Data Lake Storage (ADLS) and other cloud storage solutions.
• Strong problem-solving skills and the ability to think critically to address technical challenges.
• Good understanding of financial data, enabling efficient integration with business teams and their specific requirements.
• Ability to mentor junior team members, ensuring growth in skills and productivity.
• Strong communication skills to collaborate with stakeholders and present data-driven insights.
  
  

Preferred Skills

• Understanding and hands-on experience with Azure Cloud Services and SAP.
• Proficiency in BI tools such as Power BI
  
  

(ref:hirist.tech)