Data Engineer

This role focuses on designing, building, and maintaining robust data pipelines and infrastructure to support data-driven decision-making. The Data Engineer will work closely with data scientists, analysts, and other stakeholders to ensure data is accessible, reliable, and optimized for various analytical and operational needs.

Key Responsibilities:

Develop and maintain scalable data pipelines using various technologies.
Design and implement data models and databases.
Ensure data quality and integrity through monitoring and validation.
Collaborate with cross-functional teams to understand data requirements.
Optimize data infrastructure for performance and efficiency.
Implement data security and governance policies.

Requirements/Qualifications:

Experience in data engineering principles and practices.
Proficiency in web crawling techniques.
Strong programming skills (e.g., Python, Java).
Experience with data warehousing and ETL tools.
Familiarity with cloud platforms (e.g., AWS, Azure, GCP).
Excellent problem-solving and analytical skills.
Ability to work independently and as part of a team.