Lead Data Engineer

* Salário: R$ 18.000 a R$ 30.000 por mês (estimado)

* O valor exibido é uma estimativa calculada com base em dados públicos e referências do mercado. Não garantimos que este seja o salário oferecido para esta vaga específica.

Área: Tecnologia da Informação

Nível: Gerente

WHO WE ARE LOOKING FOR:

As our Lead Data Engineer, you will not only build and optimize end-to-end data pipelines to drive our clients' global operations, but also lead technical excellence and innovation within our data platform. You will serve as a technical authority, acting as a strategic pivot to transform complex business requirements into scalable and robust data architectures and solutions, as well as mentor and inspire our junior and senior engineers. Your ability to balance technical detail with strategic vision will be fundamental to the success of our projects and the evolution of our team.

RESPONSIBILITIES:

• Lead the conception, architecture, development, and implementation of complex, high-performance data pipelines, leveraging cutting-edge technologies like Databricks, Delta Lake, and PySpark.
• Act as a technical mentor and guide for agile data engineering teams, promoting best engineering practices, code design, scalability, and development methodologies.
• Serve as the strategic bridge between technical teams and business stakeholders, translating business requirements into clear technical specifications and ensuring alignment of the data strategy with company objectives.
• Define and implement data quality, governance, security, and compliance standards across the entire data platform, ensuring the integrity and reliability of data assets.
• Drive continuous optimization of pipeline and data infrastructure performance, scalability, and cost.
• Actively contribute to defining the technical vision and strategy for our data platform, researching and evaluating new technologies and tools to enhance our capabilities.
• Actively participate in code reviews, providing constructive feedback and ensuring code quality and consistency.

BASIC REQUIREMENTS:

• Solid and in-depth experience in architecting and implementing data solutions with Databricks, including Delta Lake and PySpark, acting as a technical reference on the platform.
• Proven ability to architect and lead the development of end-to-end data pipelines, with a focus on scalability, resilience, and high availability.
• Proficiency in programming languages PySpark, SQL, and Python (for automation, development, and support scripts).
• Experience leading or being a key contributor in agile data engineering teams, fostering collaboration and continuous delivery.
• Fluent English (written and spoken), capable of interacting with global teams and clients, often serving as a technical focal point in strategic discussions.
• Exceptional communication and influence skills, able to interact and negotiate with business and technical stakeholders at all levels, translating complex data concepts into clear, actionable terms.
• Robust understanding of data architecture design principles and software engineering best practices.

DIFFERENTIATORS (NICE-TO-HAVES):

• Experience in multi-cloud architecture (Azure, AWS, or GCP) and expertise in relevant data services (e.g., Azure Data Factory, AWS Glue, Google Dataflow).
• Deep knowledge and experience in orchestrating complex pipelines (e.g., Apache Airflow, Azure Data Factory, Prefect).
• Solid understanding and practical experience in Data Governance, Data Catalog, and metadata management.
• Expertise in designing and implementing scalable Data Warehouses, Data Lakes, and Lakehouses.
• Proficiency with code versioning tools (e.g., Git) and CI/CD practices for data pipelines.
• Prior experience mentoring or technically leading other engineers.
• Knowledge of monitoring and observability tools for data environments.

BENEFITS OF WORKING WITH DTI:

• Private Pension Plan: For every dollar contributed by the employee (up to 6% of salary), DTI matches the same amount.
• Dental Plan.
• Meal Voucher and/or Food Voucher (R$890/month).
• Health Plan: No cost to the employee or dependents.
• Home Office Allowance of R$120.00.
• TotalPass: Discounts for gyms, studios, online classes, and more.
• Partnership with SESC: Discounts on accommodations and leisure activities.
• Certification Reimbursement.
• Discounts on English Courses: With Open English, CNA, and EHeadset.
• Partnership with USP/ESALQ: Discounts on postgraduate and specialization courses.
• Partnership with PUC-RS: Discounts on undergraduate, postgraduate, specialization, language courses, and certifications.
• Partnership with FIAP: Discounts on undergraduate and postgraduate courses.
• Partnership with FIA: Discounts on undergraduate and postgraduate courses.
• Partnership with ESPM: Discounts on postgraduate and short-term courses.
• Pet Health Plan.
• Pharmacy Discounts: Drogasil and Droga Raia.
• Relaxed, Flexible Environment with Continuous Feedback.
• Routine of Continuous Learning: With Tech Shots, Mentorships, Discussion Circles ("Rodas de Conversa"), and Knowledge Transfers.

Do you identify with our culture? Then apply now!

If your resume is selected for the first phase of our selection process, we will contact you, so please keep an eye on your email (including your spam folder) and phone. If your resume is not selected, we will not send individual feedback, but it will be kept in our talent pool, and you may apply again at another time.