Software Engineer (Delta Lake) - Remote Work

At Confidencial (Apenas para Cadastrados)®, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley.

Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works remotely on roles that drive significant impact worldwide.

When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success.

As a Software Engineer specializing in Delta Lake, you will work with advanced table formats to ensure ACID transactions and reliability on data lakes. You will play a key role in implementing robust data storage solutions that support time travel, schema evolution, and high-performance analytics.

What You'll Do:

• Design and implement scalable data lake architectures using Delta Lake and Apache Spark.
• Enable reliable data processing by managing ACID transactions and ensuring data consistency across the lakehouse.
• Optimize storage performance through indexing, partitioning, and vacuuming strategies within Databricks environments.
• Develop and maintain complex ETL/ELT pipelines that leverage schema evolution and time travel for historical data analysis.
• Collaborate with data teams to define storage standards and metadata management using Parquet-based formats.
• Ensure the scalability and efficiency of large-scale data systems through continuous performance tuning and monitoring.

What we are looking for:

• 4+ years of experience in Software Engineering, Data Engineering, or Data Lakehouse technologies.
• Proven expertise in working with Delta Lake table format to ensure ACID transactions.
• Hands-on experience with Spark and Databricks for large-scale data processing and optimization.
• Deep understanding of Parquet file formats, time travel features, and schema evolution.
• Advanced proficiency in English.

How we do make your work (and your life) easier:

• 100% remote work (from anywhere).
• Excellent compensation in USD or your local currency if preferred
• Hardware and software setup for you to work from home.
• Flexible hours: create your own schedule.
• Paid parental leaves, vacations, and national holidays.
• Innovative and multicultural work environment: collaborate and learn from the global Top 1% of talent.
• Supportive environment with mentorship, promotions, skill development, and diverse growth opportunities.

Join a global team where your unique talents can truly thrive and make a significant impact!

Apply now!