GN - Data Engineer (Databricks) -111

This a Full Remote job, the offer is available from: Brazil

You will play a key role in building and maintaining our data infrastructure to support our business needs.

Responsibilities:

• Design, develop, and maintain scalable and reliable data pipelines using Databricks.
• Write and deploy Python code for data processing and ETL (Extract, Transform, Load) processes.
• Package and deploy Python code effectively for production environments.
• Utilize AWS foundation technologies such as EC2, S3, and SQS to build robust data solutions.
• Implement and adhere to DevOps processes and utilize relevant tools for continuous integration and continuous deployment (CI/CD).
• Write and execute SQL queries for data analysis and manipulation.
• Collaborate with data scientists and analysts to understand their data requirements and provide necessary data infrastructure.
• Monitor and troubleshoot data pipelines and system performance.
• Stay up-to-date with the latest technologies and best practices in data engineering and cloud computing.

• Experience using Python with Apache Spark within the Databricks environment.
• Experience working with Databricks hosted on AWS
• Minimum of 2 years of hands-on experience working with Databricks.
• Strong proficiency in Python programming language.
• Experience with packaging and deployment of Python code.
• Solid understanding of AWS foundation technologies (EC2, S3, SQS, etc.).
• Experience with DevOps processes and tools.
• Proficiency in SQL language.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills.

This offer from "Thaloz" has been enriched by Jobgether.com and got a 72% flex score.