Pleased to meet you, we are Confidencial (Apenas para Cadastrados).
We are much more than a technology company; we are a diverse, plural, and talented community. Our purpose is to lead digital transformation with excellence and agility, promoting mutual and genuine growth in ethical and long-lasting relationships. Flexibility is one of our trademarks; we operate in different team models and formats, maintaining our light, collaborative, and integrated culture, providing equitable opportunities in a space where everyone feels safe and heard.
What you will do:
- Design, build, and scale data ingestion pipelines within a cloud-based data platform.
- Own the ingestion layer, ensuring reliable and secure capture of raw data from multiple source systems.
- Implement real-time Change Data Capture (CDC) streams and batch ingestion workflows using cloud-native services.
- Integrate data from heterogeneous sources such as relational databases, REST APIs, cloud platforms, and SaaS applications.
- Define and apply data serialization standards using formats like AVRO, Parquet, JSON, and ORC, ensuring schema evolution and backward compatibility.
- Collaborate with infrastructure teams to design and maintain secure cross-cloud connectivity (e.g., VPN-based integrations).
- Monitor, troubleshoot, and optimize ingestion pipelines for performance, reliability, and cost efficiency.
- Lead ingestion initiatives end to end, from design and implementation to deployment, monitoring, and maintenance.
- Implement data quality checks, validation, monitoring, and alerting to ensure data integrity across pipelines.
- Handle schema changes, data drift, backfills, and historical data loads for high-volume datasets.
- Develop production-grade data workflows using Python, SQL, and Infrastructure-as-Code principles.
- Collaborate closely with data architects and infrastructure engineers on architectural patterns and technical decisions.
- Share best practices and technical knowledge with the broader data and analytics engineering teams.
What we are looking for:
- Advanced/fluent English for communication with a global team.
- Strong background in data engineering, backend engineering, or data platform development.
- Proven experience building scalable and reliable data ingestion pipelines (CDC (Change Data Capture) and/or batch ingestion) in production environments.
- Advanced proficiency in Python and SQL for complex data processing.
- Hands-on experience with cloud-based data platforms, preferably Google Cloud Platform.
- Experience working with modern data stacks, including data warehouses, object storage, and orchestration tools.
- Solid understanding of data formats, schema management, and schema evolution strategies.
- Knowledge of cloud networking concepts and security best practices for data systems.
- Experience with CI/CD pipelines and containerized deployments.
- Strong sense of ownership, autonomy, and problem-solving mindset.
- Ability to communicate technical concepts clearly to both technical and non-technical stakeholders.
It will be a plus if you have:
- Experience with Dataflow (Apache Beam), Cloud Datastream, or Debezium.
- Experience with Infrastructure as Code (IaC) tools such as Terraform.
- Familiarity with data observability and monitoring tools (Datadog, Prometheus, or similar).
- Experience with data quality frameworks (Great Expectations, dbt tests, Soda).
- Experience integrating APIs and SaaS platforms (CRM, ERP, marketing tools, etc.).
- Previous experience in hybrid or multi-cloud environments (AWS + GCP, Azure + GCP).
- Experience integrating third-party SaaS platforms through REST APIs.
- EInterest in or experience with AI-assisted development tools (e.g., GitHub Copilot).
- Experience contributing tointernal frameworks, playbooks, or technical standards.
Where you will work:
- This is a 100% remote position.
Employment type:
- Cooperado;
- PJ.
Our benefits include:
💪 TotalPass: Access to gyms and sports activities to support your physical health.
🧘 Zenklub: Take care of your mental health with two free consultations per month and affordable rates for additional sessions.
💼 Referral Program: Refer talented individuals to join Confidencial (Apenas para Cadastrados) and receive bonuses ranging from R$2,500 to R$5,000!
🛡 Life Insurance: Financial security for you and your family.
🎁 Baby Confidencial (Apenas para Cadastrados): Special celebration for the arrival of new Confidencial (Apenas para Cadastrados) babies!
🌍 Volunteer Communities: We encourage participation in three internal communities: Diversity, Equity & Inclusion, Sports & Movement, and Technology.
🎓 Discounts at Educational Institutions: Access benefits for undergraduate, graduate, short courses, and language studies through partnerships.
✈️ Experiences and Development: Opportunities to participate in trips, workshops, events, and team-building activities.
👥 BP Support: Receive personalized support and guidance from our Business Partners team.
Whether in your personal or professional life, Confidencial (Apenas para Cadastrados) cares about your growth and well-being! Join us on this journey! 🚀
Requisition Code:771d
