Senior Data Engineer - Hadoop to Databricks Migration

This role focuses on migrating existing data infrastructure from Hadoop to Databricks, ensuring a smooth transition and optimized performance. The Senior Data Engineer will be responsible for designing, developing, and implementing data pipelines and solutions within the Databricks environment, contributing to the overall efficiency and scalability of the data platform.

Key Responsibilities:

Design and implement data pipelines for migrating data from Hadoop to Databricks.
Optimize existing data processes and workflows within Databricks.
Develop and maintain data quality checks and monitoring systems.
Collaborate with cross-functional teams to understand data requirements and deliver solutions.
Troubleshoot and resolve data-related issues.
Contribute to the development of data engineering best practices.

Requirements/Qualifications:

Significant experience in data engineering and data warehousing.
Strong proficiency in Hadoop ecosystem technologies (e.g., HDFS, MapReduce, Hive, Spark).
Experience with Databricks and its associated tools and services.
Proficiency in programming languages such as Python or Scala.
Experience with data modeling and database design.
Excellent problem-solving and analytical skills.
Strong communication and collaboration skills.

About the Company:

Atyeti is a technology solutions provider focused on delivering innovative services to businesses across various industries. They specialize in helping organizations leverage technology to achieve their strategic goals and improve operational efficiency.