Data Engineer (Full time)

Overview

Job Title: Data Engineer (Full time)

Location: Hyderabad (5 days work from office)

General Information:

Description: The data engineer is responsible for designing, developing, and maintaining the infrastructure and systems required for data storage, processing, and analysis. They play a crucial role in building and managing the data pipelines that enable efficient and reliable data integration, transformation, and delivery for all data users across the enterprise.

Responsibilities:

· Designs and develops data pipelines that extract data from various sources, transform it into the desired format, and load it into the appropriate data storage systems.

· Integrates data from different sources, including databases, data warehouses, APIs, and external systems.

· Ensures data consistency and integrity during the integration process, performing data validation and cleaning as needed.

· Transforms raw data into a usable format by applying data cleansing, aggregation, filtering, and enrichment techniques.

· Optimizes data pipelines and data processing workflows for performance, scalability, and efficiency.

· Monitors and tunes data systems, identifies and resolves performance bottlenecks, and implements caching and indexing strategies to enhance query performance.

· Implements data quality checks and validations within data pipelines to ensure the accuracy, consistency, and completeness of data.

· Establishes the governance of data and algorithms used for analysis, analytical applications, and automated decision making.

Job Requirements:

Education

A bachelor’s degree in computer science, data science, software engineering, information systems, or related quantitative field; master’s degree preferred.

Experience:

At least 5-8 years of work experience in data management disciplines, including data integration, modeling, optimization and data quality, or other areas directly relevant to data engineering responsibilities and tasks.

Proven project experience developing and maintaining data warehouses in big data solutions using modern data platforms like Snowflake.

Skills:

· The candidate must have Data & Analytics background with a good understanding of Data Engineering, Data Governance, Data Warehousing, Business Intelligence & Analytics.

· The candidate must have strong technical skills using AWS well architected frameworks and driving/delivering many successful Data & Analytics projects.

· Hands-on experience with designing and developing data lakes, data warehouses and data marts on AWS along with Snowflake.

· Hands-on experience with AWS serverless architecture components/ecosystem providing optimal & efficient solutions.

· Hands-on experience with source/target systems integration using AWS services. Manage and configure AWS service integration i.e AWS Glue, Lambda, DynamoDB, S3, Kinesis etc.

· Strong hands-on experience using Python/PySpark for data engineering (ETL) activities.

· Hands-on experience using RDBMS SQL & NoSQL databases like Oracle, SQL Server, Snowflake, MongoDB, DynamoDB etc.

· Hands-on experience with handling structured and semi-structured data.

· Should be able to prepare/maintain documentation around best practices & standards covering data architecture, data security, data catalogs, performance & scalability etc.

· Strong Analytical and Communication skills working with Business and technical teams.

· AWS Solution Architect or AWS Developer Associate certification will be a plus.

· Good understanding on using Power BI Analytical tool.

Job Type: Full-time

Pay: Up to ?2,000,000.00 per year

Schedule:

• Monday to Friday

Experience:

• total work: 1 year (Preferred)

Work Location: In person