* Salário: R$ 11.000 a R$ 20.000 por mês (estimado)
* O valor exibido é uma estimativa calculada com base em dados públicos e referências do mercado. Não garantimos que este seja o salário oferecido para esta vaga específica.
Área: Tecnologia da Informação
Nível: Senior
Application
Following the acquisition of Tooploox, a premier AI and product development company, Confidencial (Apenas para Cadastrados) now offers true end-to-end delivery—from strategic advisory and solution design to custom AI development and enterprise-scale implementation. Our capability centers combine deep technical expertise, proven delivery methodologies, and sector-specific knowledge to address complex business challenges quickly and effectively.
We are looking for a Staff Data Platform Engineer (Python, Spark, Databricks) to join our team.
Project description
The project consists of building a scalable backend platform that runs data-driven analytical models in a controlled, automated, and auditable way. The backend must orchestrate model execution, enforce data validation, manage metadata and artefacts, and ensure full traceability of every processing step. The architecture is modular and cloud-oriented, with a strong focus on automation, versioning, and operational reliability.
What you'll do
- Build, maintain, and optimize data ingestion pipelines using Python, pandas, Spark, and Databricks
- Parse and transform complex Excel-based financial models into structured Delta Lake tables
- Implement data processing workflows and calculation logic in Python
- Develop and manage pipelines using Databricks Jobs, clusters, and SQL Warehouse
- Improve performance, scalability, and reliability of existing data pipelines
- Write clean, maintainable, and production-quality Python code
- Use SQL for data validation, transformation, and pipeline integration
What you bring
- Senior-level engineer with 5–8+ years of experience
- Strong Python skills, with proven experience building production data pipelines
- Hands-on experience with pandas for data transformation and ingestion
- Experience working with Spark (PySpark or Scala) and Databricks
- Solid SQL skills and experience writing complex queries
- Experience designing, implementing, and maintaining data processing pipelines
- Experience working with medium to large datasets
- Strong ownership and ability to work hands-on in a data platform environment
- English fluency for daily communication with actuarial and engineering teams.
Nice to have
- Experience optimizing Python or Spark workloads for performance and scalability
- Experience working with Delta Lake
- Experience ingesting and transforming large Excel-based datasets
- Experience working with financial or structured modeling data
When you join Confidencial (Apenas para Cadastrados), you'll…
- Shape real-world AI-driven projects across key industries, working with clients from startup innovation to enterprise transformation.
- Be part of a global team with equal opportunities for collaboration across continents and cultures.
- Thrive in an inclusive environment that prioritizes continuous learning, innovation, and ethical AI standards.
Ready to make an impact?
If you're excited to build things that matter, champion responsible AI, and grow with some of the industry’s sharpest minds. Apply today and let’s innovate together.