Senior Data Engineer

Key Responsibilities:

• Architecture Design: Design and implement scalable and robust big data architectures to support data processing, storage, and analysis.
• Data Management: Develop and maintain data lake and data warehousing solutions for efficient data storage and retrieval.
• ETL Processes: Design, develop, and optimize ETL processes to ensure seamless data integration from various sources.
• Database Management: Manage and optimize Amazon Redshift clusters for high performance and cost efficiency.
• Big Data Technologies: Leverage big data technologies such as Hadoop, Spark, and other relevant tools to handle large-scale data processing.
• Scripting: Write and maintain efficient Python scripts for data processing and automation tasks.
• Pipeline Orchestration: Utilize Airflow or similar orchestration tools to automate and manage complex data pipelines.
• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver suitable solutions.
• Quality Assurance: Ensure data quality, integrity, and security across all data platforms.
• Monitoring & Troubleshooting: Monitor and troubleshoot data pipelines and architectures to ensure high availability and performance.
• Continuous Learning: Stay updated with the latest industry trends and best practices in big data technologies and architectures.
• Nice to have: Data Platform Engineering

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
• 5+ years of relevant experience in big data architecture, data engineering, or a related field.
• Strong understanding of data lake and data warehousing concepts and architectures.
• Proven experience with ETL technologies and tools.
• Hands-on experience with Amazon Redshift and managing large-scale data warehouses.
• Proficiency in big data technologies, including Hadoop, Spark, and related ecosystems.
• Strong Python scripting skills for data processing and automation.
• Experience with Airflow or similar orchestration tools.
• Experience in handling large databases and data sets, ensuring data quality and performance.
• Excellent problem-solving skills and ability to troubleshoot complex data issues.
• Strong communication skills and ability to work collaboratively with cross-functional teams.
• Knowledge of data governance, security, and compliance best practices.

Preferred Skills

• Experience with cloud platforms such as AWS, Azure, or Google Cloud.
• Familiarity with additional programming languages like SQL, Java, or Scala.
• Experience with data visualization tools and techniques.

Benefits Offered

• Flexible Work Hours.
• Work on impactful, industry-changing projects.
• Continuous learning and career growth opportunities..
• Collaborative, innovative, and inclusive environment.
• Insurance – Group Medical Coverage, Employee Personal Accident, Maternity and Paternity Leaves.
• Performance Feedback Sessions.
• Competitive Compensation.
• Rewards and Recognition Program, Employee Referral Program, Fun Events.