Data Engineer

Job Title : Data Engineer

Years of Exp : 3-7

Location : Bangalore

Job Description

We are seeking a skilled Data Engineer to join our growing team in Bangalore. The ideal candidate will have a strong background in data engineering, working with large-scale data systems, and driving data pipelines that facilitate high-quality data processing and analysis. You will work closely with data scientists, analysts, and other cross-functional teams to ensure the efficient management, transformation, and storage of data.

Key Responsibilities

Data Pipeline Development :

• Design, develop, and maintain robust data pipelines to process large volumes of structured and unstructured data from various sources.
• Ensure data accuracy, reliability, and scalability.
  
  

Data Integration & ETL

• Develop and implement ETL processes to extract, transform, and load data into centralized data warehouses and data lakes.
• Ensure seamless integration with external APIs and databases.
  
  

Data Quality & Validation

• Monitor data quality, perform regular data cleansing, and ensure that datasets are accurate, consistent, and up to date.
• Implement automated checks and validation :
• Work closely with data scientists and analysts to understand data needs and help optimize data flows for analysis and machine learning model development.
  
  

Database Management

• Manage and optimize relational and NoSQL databases, ensuring efficient data storage and fast query performance.
• Experience with technologies such as SQL, Hive, and Spark is essential.
  
  

Performance Optimization

• Continuously monitor and optimize data processes and storage systems for performance improvements, ensuring efficient resource usage and :
• Maintain clear and comprehensive documentation for all data engineering processes, pipelines, and infrastructure to ensure transparency and ease of :
• 3-7 years of experience in data engineering, with hands-on experience in building data pipelines and working with big data technologies.
• Strong experience with SQL, ETL processes, and data warehousing.
• Proficiency in data engineering tools such as Apache Hadoop, Spark, Kafka, or similar technologies.
• Experience with cloud platforms like AWS, Google Cloud, or Azure.
• Strong problem-solving skills and attention to detail.
• Excellent communication and collaboration skills to work with cross-functional teams.
  
  

If you are passionate about working with data at scale and eager to contribute to building innovative data solutions, we encourage you to apply and be part of our dynamic team in Bangalore!

(ref:hirist.tech)