Senior AWS Data Engineer

Overview

Job Description: Senior AWS Data Engineer
Position Overview:
We are seeking a highly skilled Senior AWS Data Pipeline Engineer to design, develop, and optimize scalable data pipelines on AWS. The ideal candidate will have expertise in ETL/ELT processes, big data technologies, and cloud-native data solutions, ensuring high-performance data ingestion, transformation, and storage for analytics and machine learning applications.
________________________________________
Key Responsibilities:
1. Data Pipeline Design & Development

• Design and develop scalable, resilient, and secure ETL/ELT data pipelines using AWS services.
• Build and optimize data workflows leveraging AWS Glue, EMR, Lambda, and Step Functions.
• Implement batch and real-time data ingestion using Kafka, Kinesis, or AWS Data Streams.
• Ensure efficient data movement across S3, Redshift, DynamoDB, RDS, and Snowflake.
• 2. Cloud Data Engineering & Storage
• Architect and manage data lakes and data warehouses using Amazon S3, Redshift, and Athena.
• Optimize data storage and retrieval using Parquet, ORC, Avro, and columnar storage formats.
• Implement data partitioning, indexing, and query performance tuning.
• Work with NoSQL databases (DynamoDB, MongoDB) and relational databases (PostgreSQL, MySQL, Aurora).

3. Infrastructure as Code (IaC) & Automation4. Performance Optimization & Monitoring5. Security, Compliance & Governance6. Collaboration & Stakeholder Engagement

• Deploy and manage AWS data infrastructure using Terraform, AWS CloudFormation, or AWS CDK.
• Implement CI/CD pipelines for automated data pipeline deployments using GitHub Actions, Jenkins, or AWS CodePipeline.
• Automate data workflows and job orchestration using Apache Airflow, AWS Step Functions, or MWAA.
• Optimize Spark, Hive, and Presto queries for performance and cost efficiency.
• Implement auto-scaling strategies for AWS EMR clusters.
• Set up monitoring, logging, and alerting with AWS CloudWatch, CloudTrail, and Prometheus/Grafana.
• Implement IAM policies, encryption (AWS KMS), and role-based access controls.
• Ensure compliance with GDPR, HIPAA, and industry data governance standards.
• Monitor data pipelines for security vulnerabilities and unauthorized access.
• Work closely with data analysts, data scientists, and business teams to understand data needs.
• Document data pipeline designs, architecture decisions, and best practices.
• Mentor and guide junior data engineers on AWS best practices and optimization techniques.

Job Types: Full-time, Permanent

Pay: ₹558,389.34 - ₹1,938,721.07 per year

Benefits:

• Health insurance
• Provident Fund

Schedule:

• Day shift

Supplemental Pay:

• Yearly bonus

Work Location: In person