Data Engineer

Key Responsibilities:

• A day in the life of an Infoscion
• As part of the Infosys consulting team your primary role would be to actively aid the consulting team in different phases of the project including problem definition effort estimation diagnosis solution generation a nd design and deployment
• You will explore the alternatives to the recommended solutions based on research that includes literature surveys information available in public domains vendor evaluation information etc
• and build POCs
• You will create requirement specifications from the business needs define the to be processes and detailed functional designs based on requirements
• You will support configuring solution requirements on the products understand if any issues diagnose the root cause of such issues seek clarifications and then identify and shortlist solution alternatives
• You will also contribute to unit level and organizational initiatives with an objective of providing high quality value adding solutions to customers
• If you think you fit right in to help our clients navigate their next in their digital transformation journey this is the place for you

Technical Requirements:

• Must have 5 years of experience in data engineering
• Proficiency with Databricks and Apache Spark
• Strong SQL skills and experience with relational databases
• Experience with big data technologies e
• g
• Hadoop Kafka
• Knowledge of data warehousing concepts and ETL processes
• Experience with CI CD tools particularly Jenkins
• Excellent problem solving and analytical skills
• Solid understanding of big data fundamentals and experience with Apache Spark
• Familiarity with cloud platforms e
• g
• AWS Azure
• Experience with version control systems e
• g
• BitBucket
• Understanding of DevOps principles and tools e
• g
• CI CD Jenkins
• Databricks certification is a plus

Additional Responsibilities:

• Design develop and maintain scalable data pipelines on Databricks using PySpark
• Collaborate with data analysts and scientists to understand data requirements and deliver solutions
• Optimize and troubleshoot existing data pipelines for performance and reliability
• Ensure data quality and integrity across various data sources
• Implement data security and compliance best practices
• Monitor data pipeline performance and conduct necessary maintenance and updates
• Document data pipeline processes and technical specifications
• Location of posting Infosys Ltd
• is committed to ensuring you have the best experience throughout your journey with us
• We currently have open positions in a number of locations across India Bangalore Pune Hyderabad Mysore Kolkata Chennai Chandigarh Trivandrum Indore Nagpur Mangalore Noida Bhubaneswar Coimbatore Mumbai Jaipur Hubli Vizag
• While we work in accordance with business requirements we shall strive to offer you the location of your choice where possible

Preferred Skills: