Lead AWS Data Engineer

Band: C1

Position - Lead AWS Data Engineer

Location - Pune/ Gurgaon/Hyderabad Hybrid

Experience - 9+ Years

About Clairvoyant (an EXL Company):

Clairvoyant (an EXL Company) is a global technology consulting and services organization founded in 2012, headquartered in Chandler, US, and has delivery centers across the globe. We help organizations maximize the value of data by providing data engineering, analytics, machine learning, and user experience consulting and development projects to multiple Fortune 500 clients. Clairvoyant clients rely on its deep vertical knowledge and best-in-class services to drive revenue growth, boost operational efficiencies, and manage risk and compliance. Our team of experts with direct industry experience in data engineering, analytics, machine learning, and user experience has your back every step of the way.

Technical Proficiency:

• 9+ years of experience in Data Engineering on the AWS Stack, Databricks, PySpark and DBT

• Programming experience with advanced Python, SQL

• Experience with Kubernetes and Docker

• Hands on experience with AWS services such as Cloud Formation, S3, Lambda, Step Functions, IAM, KMS, Athena, Glue, Glue Data Brew, EMR/Spark, Data Sync, Event Bridge, EC2, SQS, SNS, Lake Formation, Cloud Watch, Cloud Trail
• Good experience building Real-Time streaming data pipelines with Kafka, Kinesis, etc.

• Proven competency in building solutions on data lake/data warehouse

Roles & Responsibilities:

• Participate in requirements and design discussions

• Design and implement automated data pipelines to handle high-velocity and volumes of streaming data

• Generate quality data for supporting real time decisioning systems and AI needs

• Understand existing data pipelines and identify improvement areas.

• Optimize storage, retrieval, monitoring and scaling mechanisms

• Adopt CI/CD pipelines and IaaS

• Ensure best practices and modularity of code

• Mandate peer review for code quality and conduct end to end testing

• Work in a team environment but take ownership

• Ability to engage with client stakeholders effectively

• Excellent communication skills and ability to articulate project requirements, decisions, design principles and outcomes

Good to have

• Domains: Sports, Betting
• AWS DE Certification, Databricks Certified DE Professional

Key Skills: AWS, Databricks, PySpark, Kafka, Kubernetes, Docker, SQL

Education: BE/BTech or Graduate / Postgraduate in Computer Science / Information Technology.