PySpark Developer

This role focuses on developing and maintaining robust data pipelines and applications using PySpark. The ideal candidate will be responsible for designing, implementing, and optimizing data processing solutions to meet business requirements, ensuring data quality and efficiency.

Key Responsibilities:

Develop and maintain PySpark jobs for data extraction, transformation, and loading (ETL).
Design and implement scalable data processing pipelines.
Optimize PySpark code for performance and efficiency.
Collaborate with data engineers and data scientists to understand data requirements.
Ensure data quality and integrity through rigorous testing and validation.
Troubleshoot and resolve data processing issues.

Requirements/Qualifications:

Strong experience with PySpark and related technologies.
Proficiency in Python programming.
Experience with data warehousing concepts and technologies.
Understanding of distributed computing principles.
Experience with data quality and data governance practices.
Excellent problem-solving and analytical skills.
Ability to work independently and as part of a team.

About the Company: SightSpectrum is a technology company specializing in data-driven solutions. They help organizations leverage data to gain insights and improve decision-making.