PySpark Databricks Engineer

This role focuses on designing, developing, and maintaining robust data pipelines and solutions using PySpark and Databricks. The ideal candidate will be responsible for building scalable and efficient data processing workflows to support business intelligence and data-driven decision-making.

Key Responsibilities:

Develop and maintain PySpark jobs and Databricks notebooks for data transformation and analysis.
Design and implement data pipelines for ingestion, processing, and storage.
Optimize Spark jobs for performance and scalability.
Collaborate with data scientists and engineers to integrate data solutions into existing systems.
Troubleshoot and resolve data pipeline issues.
Ensure data quality and integrity.

Requirements/Qualifications:

Strong experience with PySpark and the Databricks platform.
Proficiency in data warehousing concepts and technologies.
Experience with ETL processes and data pipeline design.
Solid understanding of distributed computing principles.
Experience with cloud platforms (e.g., AWS, Azure, GCP) is a plus.
Excellent problem-solving and communication skills.

About the Company: Infosys is a global leader in providing technology consulting and digital transformation services. The company helps businesses across various industries navigate their digital journeys and achieve their strategic goals.