Free cookie consent management tool by TermsFeed Senior Data Engineer - ETL/Python | Antal Tech Jobs
Back to Jobs
3 Weeks ago

Senior Data Engineer - ETL/Python

decor
Bangalore, Karnataka, India
Information Technology
Full-Time
Fulcrum Digital Inc

Overview

Advanced SQL Development And Optimization

  • Design, develop, and optimize complex SQL queries and stored procedures in Oracle and SQL Server environments to efficiently extract, transform, and load (ETL) data for various analytical and operational needs.
  • Implement advanced SQL techniques including window functions, common table expressions (CTEs), and subqueries to solve complex data manipulation challenges.
  • Troubleshoot and resolve intricate SQL query performance issues, identifying root causes and implementing effective solutions.

Performance Tuning And Optimization

  • Proactively identify and address performance bottlenecks in SQL queries and data processing pipelines.
  • Implement and manage indexing strategies (B-tree, bitmap, etc.) to optimize query performance for large datasets.
  • Utilize database partitioning techniques (range, list, hash) to improve data manageability and query speed.
  • Design and implement materialized views to pre-compute and store frequently accessed data for faster retrieval.
  • Apply query rewriting techniques to improve the efficiency of SQL execution plans.
  • Conduct thorough analysis of query execution plans using database-specific tools (e.g., Oracle Explain Plan, SQL Server Execution Plan) to identify areas for optimization and reduce resource consumption.

Data Modeling And Design

  • Design and implement efficient and scalable data models (both relational and dimensional) to support analytical reporting, data warehousing, and operational systems.
  • Apply best practices for database normalization and denormalization based on specific use case requirements and performance considerations.
  • Collaborate with data architects to ensure data models align with enterprise data architecture standards and principles.

Collaboration And Communication

Work closely with data architects, business analysts, and reporting teams to understand data requirements, translate them into technical specifications, and deliver effective data solutions.

Communicate complex technical concepts clearly and concisely to both technical and non-technical stakeholders.

Actively participate in team meetings and contribute to the planning and execution of data engineering projects.

Technical Leadership And Mentorship

  • Provide technical guidance and mentorship to junior data engineers on SQL best practices, indexing strategies, performance tuning techniques, and data modeling principles.
  • Contribute to the development and maintenance of team standards and best practices for data engineering processes.
  • Conduct knowledge transfer sessions and workshops to upskill team members in SQL optimization, data warehousing concepts, and DSS best practices.

Data Security And Governance

  • Implement and enforce data security measures, including role-based access control (RBAC) to ensure appropriate data access and prevent unauthorized modifications.
  • Work with security teams to implement data encryption techniques (at rest and in transit) to protect sensitive information.
  • Adhere to data governance policies and procedures to ensure data quality, integrity, and compliance.

Continuous Improvement

  • Stay up-to-date with the latest trends and technologies in data engineering, database systems, and performance optimization.
  • Proactively identify opportunities for process improvements and automation within the data engineering workflow.
  • Contribute to the evaluation and adoption of new tools and technologies to enhance the team's capabilities.

Qualifications

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
  • Minimum of 5-7 years of hands-on experience in Data Engineering, with a strong focus on SQL development and optimization.
  • Expert-level proficiency in writing and optimizing complex SQL queries for Oracle and SQL Server databases.
  • Deep understanding of database performance tuning techniques, including indexing, partitioning, materialized views, and query rewriting.
  • Proven experience in analyzing query execution plans and identifying performance bottlenecks.
  • Solid understanding of data modeling concepts (relational, dimensional) and best practices for normalization and denormalization.
  • Experience working with large datasets and data warehousing concepts.
  • Strong analytical and problem-solving skills with a keen attention to detail.
  • Excellent communication, collaboration, and interpersonal skills.
  • Ability to work independently and as part of a team in a fast-paced environment.

Preferred Qualifications

  • Experience with other database technologies (e.g., PostgreSQL, MySQL, NoSQL databases).
  • Familiarity with ETL/ELT tools and frameworks (e.g., Apache NiFi, Informatica, Talend).
  • Experience with cloud platforms (e.g., AWS, Azure, GCP) and cloud-based data warehousing solutions

(e.g., Snowflake, Amazon Redshift, Google BigQuery).

  • Knowledge of scripting languages like Python or Shell scripting for data manipulation and automation.
  • Experience with data governance and data quality frameworks.
  • Prior experience in a leadership or mentorship role.

(ref:hirist.tech)
Share job
Similar Jobs
View All
1 Day ago
bluCursor Infotech - Technical Lead - Software Development
Information Technology
  • Hyderabad, Telangana, India
Job Description Lead and manage the development team to ensure timely and high-quality delivery of software projects. Provide technical direction, mentorship, and performance feedback to team members. Design, develop, and maintain scalable and ef...
decor
1 Day ago
Medvarsity - Business Analyst - LMS Apps
Information Technology
  • Hyderabad, Telangana, India
Job SummaryWe are seeking a proactive and technically skilled Moodle LMS Administrator & Client-Facing Project Manager to lead the end-to-end management, customization, and client coordination for our Moodle-based learning platform.This role bridges...
decor
1 Day ago
Python Artificial Intelligence/Machine Learning Engineer
Information Technology
  • Hyderabad, Telangana, India
Responsibilities Design, develop, and implement AI/ML models and algorithms to address specific business needs and opportunities. Utilize your deep understanding of Python programming to build robust and scalable AI/ML solutions. Apply your exper...
decor
1 Day ago
Software Engineer - Network Software Development
Information Technology
  • Hyderabad, Telangana, India
One of our client is looking for a Software Engineer- Network Software DevelopmentJob SummaryYou will be working on the design, development, and integration of critical elements for the network stack of our wireless communication systems, including ...
decor
1 Day ago
Interesting Job Opportunity: i2k2 - Network Engineer - Routing & Switching
Information Technology
  • Hyderabad, Telangana, India
Job Description BS preferred or equivalent work experience. Generally, 3 to 4 years work experience in a related discipline. CCNP Certification preferred. Advanced knowledge of routing protocols such as BGP and OSPF 2. Advanced knowledge of laye...
decor
1 Day ago
bluCursor Infotech - Python/Machine Learning Engineer
Information Technology
  • Hyderabad, Telangana, India
Responsibilities Develop Python applications and software solutions based on business requirements and technical specifications, with a focus on integrating AI/ML capabilities. Write well-designed, efficient, and reusable code adhering to best pra...
decor
1 Day ago
Salesforce Technical Lead - Apex/Lightning Component Framework
Information Technology
  • Hyderabad, Telangana, India
Job DescriptionSalesforce Technical Lead the Technical Development Lead - CRM will play a technical leadership role. In this role, he/she is responsible for working with business analysts, developers and stakeholders to implement, integrate and main...
decor
1 Day ago
Oracle & PL SQL developer
Information Technology
  • Hyderabad, Telangana, India
Mandatory SkillsStrong skill in Oracle PL SQLDesired SkillsExposure to Informatica power centreRoles & Responsibilities Must have a capability to gather requirements from Client Capable enough to own and manage the delivery from Start to end. Deve...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media