Senior Data Scientist - WFN

Confidencial (Apenas para Cadastrados) is hiring a Data Analytics Engineer

We're seeking a Data Analytics Engineer to build and maintain scalable data pipelines for payroll analytics and LLM feature engineering.

You'll work on event-driven data ingestion, transformation pipelines, and feature store development to support AI/ML initiatives and business analytics.

Key Responsibilities

• Data Pipeline Engineering
• Design and implement event-driven data ingestion pipelines using SQS/SNS and Kafka
• Build Java-based integration services to consume and process events
• Develop and maintain Databricks-based ETL/ELT workflows

Feature Engineering for LLM & Analytics

• Create feature datasets from ingested data to support LLM training and fine-tuning
• Design and implement feature stores using Python and SQL on Databricks
• Build data quality validation frameworks to ensure feature accuracy and consistency
• Develop data preprocessing pipelines for model training and inference

Data Platform Management

• Maintain Databricks environments across multiple regions and deployment stages
• Optimize Spark jobs for performance and cost efficiency
• Implement data validation and monitoring solutions
• Manage canonical data models and transformation logic using SQL and PySpark

Analytics & Reporting

• Build analytics-ready datasets for business intelligence and reporting
• Create data documentation and lineage tracking
• Support compliance and audit requirements through data validation pipelines

Must-Have Qualifications

• Strong experience with Databricks platform and Apache Spark (PySpark)
• Proficiency in SQL for complex data transformations and analytics
• Hands-on experience with event-driven architectures using AWS SQS/SNS and/or Kafka
• Java development experience for building integration services and event consumers
• Python programming skills for data processing, feature engineering, and automation
• Experience building ETL/ELT pipelines for large-scale data processing
• Understanding of data modeling, schema design, and data warehouse concepts
• Familiarity with CI/CD practices and version control (Git)
• Strong problem-solving skills for data quality and pipeline optimization

Nice-to-Have

• Experience preparing datasets for machine learning model training (especially LLMs)
• Knowledge of feature store architectures and MLOps practices
• Experience with multi-region data processing and compliance requirements
• Background in data governance and data quality frameworks
• Experience with Databricks Delta Lake and Unity Catalog
• Knowledge of AWS services
• Understanding of data security and privacy best practices
• Experience with test-driven development for data pipelines

Technical Environment

**Languages**: Python, SQL (Spark SQL), Java

**Platforms**: Databricks, AWS (SQS/SNS, S3)

**Frameworks**: Apache Spark, Kafka, PySpark

**Tools**: Git, pytest, Databricks CLI

What are you waiting for?Apply today!

Find out why people come to Confidencial (Apenas para Cadastrados) and why they stay: https://youtu.be/ODb8lxBrxrY

(ADA version: https://youtu.be/IQjUCA8SOoA )

• O modelo de trabalho adotado pela Confidencial (Apenas para Cadastrados) é office based/presencial, com a possibilidade de trabalho em home-office por até duas vezes na semana.

• Considerando que as atividades desempenhadas pelos ocupantes deste cargo envolvem acesso a informações altamente confidenciais e sensíveis de clientes da Confidencial (Apenas para Cadastrados) e de seus respectivos empregados, a Confidencial (Apenas para Cadastrados) reserva-se o direito de conduzir checagem de histórico, de tempos em tempos, conforme autoriza o Incidente de Recurso de Revista Repetitivo nº 01 do Tribunal Superior do Trabalho, mediante consentimento do candidato/trabalhador.

• O modelo de trabalho adotado pela Confidencial (Apenas para Cadastrados) é office based/presencial, com a possibilidade de trabalho em home-office por até duas vezes na semana.

• Considerando que as atividades desempenhadas pelos ocupantes deste cargo envolvem acesso a informações altamente confidenciais e sensíveis de clientes da Confidencial (Apenas para Cadastrados) e de seus respectivos empregados, a Confidencial (Apenas para Cadastrados) reserva-se o direito de conduzir checagem de histórico, de tempos em tempos, conforme autoriza o Incidente de Recurso de Revista Repetitivo nº 01 do Tribunal Superior do Trabalho, mediante consentimento do candidato/trabalhador.

#LI-TD1
#LI-Híbrido

A little about Confidencial (Apenas para Cadastrados): We are a comprehensive global provider of cloud-based human capital management (HCM) solutions that unite HR, payroll, talent, time, tax and benefits administration and a leader in business outsourcing services, analytics, and compliance expertise. We believe our people make all the difference in cultivating a down-to-earth culture that embraces our core values, welcomes ideas, encourages innovation, and values belonging. We've received recognition for our work by many esteemed organizations, learn more at Confidencial (Apenas para Cadastrados) Awards and Recognition.

Diversity, Equity, Inclusion & Equal Employment Opportunity at Confidencial (Apenas para Cadastrados):Confidencial (Apenas para Cadastrados) is committed to an inclusive, diverse and equitable workplace, and is further committed to providing equal employment opportunities regardless of any protected characteristic including: race, color, genetic information, creed, national origin, religion, sex, affectional or sexual orientation, gender identity or expression, lawful alien status, ancestry, age, marital status, protected veteran status or disability. Hiring decisions are based upon Confidencial (Apenas para Cadastrados)’s operating needs, and applicant merit including, but not limited to, qualifications, experience, ability, availability, cooperation, and job performance.

Ethics at Confidencial (Apenas para Cadastrados):Confidencial (Apenas para Cadastrados) has a long, proud history of conducting business with the highest ethical standards and full compliance with all applicable laws. We also expect our people to uphold our values with the highest level of integrity and behave in a manner that fosters an honest and respectful workplace. Click https://jobs.Confidencial (Apenas para Cadastrados).com/life-at-Confidencial (Apenas para Cadastrados)/ to learn more about Confidencial (Apenas para Cadastrados)’s culture and our full set of values.