Bangalore, Karnataka, India
Information Technology
Full-Time
FiftyFive Technologies
Overview
Role
We are looking for a Data Engineer with a minimum of 1 year of experience working with LLMs and a strong background in data pipelines, storage optimization, and AI-driven data processing. The ideal candidate will play a key role in managing, optimizing, and scaling data architectures to support LLM applications and AI :
We are looking for a Data Engineer with a minimum of 1 year of experience working with LLMs and a strong background in data pipelines, storage optimization, and AI-driven data processing. The ideal candidate will play a key role in managing, optimizing, and scaling data architectures to support LLM applications and AI :
- Design and develop scalable, high-performance data pipelines for AI and LLM-powered applications.
- Work with structured and unstructured data, ensuring efficient preprocessing, transformation, and storage.
- Implement and optimize data retrieval and indexing strategies for LLM fine-tuning and inference.
- Manage vector databases (FAISS, Chroma, Pinecone, Weaviate, Astra DB. ) for retrieval-augmented generation (RAG) workflows.
- Build and maintain ETL/ELT workflows using tools like Airflow, Prefect, or Dagster.
- Ensure data quality, governance, and lineage to support AI-driven insights.
- Collaborate with ML engineers and researchers to improve LLM data pipelines and infrastructure.
- Work with cloud-based data storage solutions (AWS S3 GCP BigQuery, Azure Data Lake, etc. ).
- Automate data monitoring, validation, and debugging to ensure seamless pipeline :
- 4+ years of overall experience in data engineering, AI infrastructure, or related fields.
- At least 1 year of hands-on experience working with LLMs and AI data workflows
- Strong expertise in Python, SQL, and distributed data processing frameworks (Spark, Dask, Ray, or similar).
- Experience with vector databases and retrieval systems for AI-driven applications.
- Knowledge of data modeling, indexing, and storage optimization
- Familiarity with ETL/ELT frameworks like Apache Airflow, Prefect, or Dagster.
- Experience handling large-scale datasets and optimizing data ingestion pipelines.
- Understanding of cloud-based data architectures (AWS, GCP, Azure).
- Basic knowledge of MLOps principles and integrating data workflows with AI models.
- Exposure to LLM fine-tuning, embeddings, and retrieval-augmented generation (RAG).
- Experience with LangChain, Hugging Face Transformers, or OpenAI APIs.
- Familiarity with feature stores (Feast, Tecton) and streaming platforms (Kafka).
Similar Jobs
View All
10 Hours ago
MTS II - Software Engineer
Information Technology
- 4 - 7 Yrs
- Pune
MAJOR RESPONSIBILITIES
• Design, implement, integrate, and verify software applications and tools using JavaScript, NodeJS, and C++.
• Enhance, optimize, and improve the efficiency and robustness of current software, with a particular focus on OSS ...
1 Day ago
Business Advisory Analyst
Information Technology
- Bangalore, Karnataka, India
Skill required: Banking Services - Core BankingDesignation: Business Advisory AnalystQualifications:BBA/BCom/Master of Business AdministrationYears of Experience:3 to 5 yearsAbout AccentureAccenture is a global professional services company with lea...
1 Day ago
Front End Developer
Information Technology
- Bangalore, Karnataka, India
Position Title: Front End DeveloperCompany: Johnson Controls (JCI)Location: BangaloreJob Summary: We are seeking a talented Front End Developer with 4-7 years of experience to join our dynamic team. The ideal candidate will have a strong background ...
1 Day ago
Database Engineer III (Big Data)
Information Technology
- Bangalore, Karnataka, India
LivePerson (NASDAQ: LPSN) is the global leader in enterprise conversations. Hundreds of the world’s leading brands — including HSBC, Chipotle, and Virgin Media — use our award-winning Conversational Cloud platform to connect with millions of consume...
1 Day ago
Data Scientist Manager
Information Technology
- Bangalore, Karnataka, India
Job DescriptionLeads a team of people who design, develop and program methods, processes, and systems to consolidate and analyze unstructured, diverse “big data” sources to generate actionable insights and solutions for client services and product e...
1 Day ago
Data Scientist Manager
Information Technology
- Bangalore, Karnataka, India
Job DescriptionLeads a team of people who design, develop and program methods, processes, and systems to consolidate and analyze unstructured, diverse “big data” sources to generate actionable insights and solutions for client services and product e...
1 Day ago
Sr. QA Engineer
Information Technology
- Bangalore, Karnataka, India
Role Summary:Picarro is seeking an exceptional Sr. QA Engineer for functional testing of Picarro Analyzers. This role expects you to analyze requirements, create and execute test-plan, and record results in test-repo. This person is also expected to...
1 Day ago
C++ Graphics and Windowing System Software Engineer - Mir
Information Technology
- Bangalore, Karnataka, India
We build a high-performance, high-efficiency stack for window managers and display subsystems in C++, called Mir. We're growing the team and looking for new colleagues who share our passion for precision, performance and user experience.Our goal is ...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in