Data Engineer

Overview

Role Overview: As a Data Engineer with a minimum of 6 years of experience, you will design and manage data systems to power the development of intelligent AI agents. Your role will focus on creating efficient pipelines for collecting, preparing, and delivering data for AI agent training, optimization, and analytics, ensuring a seamless flow of high-quality data to enhance agent capabilities.

Key Responsibilities:

• AI Agent Data Pipelines: Build and maintain scalable ETL pipelines to ingest and process data specifically for AI agent training and continuous learning. Manage datasets used for building AI agents and optimizing their performance across diverse tasks.
• Data Quality and Security: Ensure data quality, consistency, and security, enabling reliable agent training and evaluation. Implement robust data validation workflows to maintain dataset integrity.
• Collaboration for AI Agent Development: Collaborate closely with ML engineers to prepare datasets tailored for LLM and task-specific AI agent models. Enable real-time data feeds for adaptive AI agents to learn from live interactions.
• Analytics and Visualization: Develop dashboards for end-users, stakeholders, and landowners to visualize AI agent performance and platform metrics. Provide insights into data trends that inform agent behavior and decision-making optimization.
• Decentralized Data Systems: Integrate decentralized storage solutions (e.g., IPFS) to manage distributed data pipelines for AI agents.

Qualifications:

• At of experience in Python, SQL, and data pipeline tools (e.g., Apache Airflow).
• Strong experience with cloud platforms (AWS, GCP, or Azure).
• Familiarity with decentralized storage systems (e.g., IPFS).
• Strong knowledge and experience in RAG framework and LLM.
• Knowledge of data visualization.
• Experience in supporting ML models with clean, labeled datasets.

Experience : 6 to 12 yrs.

Job Type: Full-time

Pay: ₹800,000.00 - ₹1,500,000.00 per year

Benefits:

• Health insurance

Schedule:

• Night shift

Experience:

• Data Engineer: 6 years (Required)

Work Location: Remote