Overview

5+ years of experience required. Remote position.

Key Responsibilities

• Develop and maintain robust data pipelines using Python and ADF.
• Design and optimize ELT/ETL processes to load data into Snowflake from various sources.
• Implement performance tuning and monitoring on Snowflake queries and tables.
• Collaborate with data architects and analysts to understand data requirements and provide scalable solutions.
• Automate recurring data ingestion and transformation tasks using scripting in Linux environments.
• Manage data security, access controls, and data quality standards in Snowflake.
• Schedule and monitor ADF pipelines for production loads and troubleshooting failures.
• Integrate external APIs or databases as needed for data enrichment and ingestion.
• Perform root cause analysis and provide timely resolutions to data issues.

Required Qualifications

• 5+ years of experience in data engineering or backend data development.
• Strong proficiency in Python for scripting, data manipulation, and API interaction.
• Hands-on experience with Snowflake including schema design, SQL development, and performance tuning.
• Practical experience with Azure Data Factory: pipelines, triggers, datasets, and linked services.
• Proficiency with Linux/Unix command line, shell scripting, and cron jobs.
• Experience working with large-scale datasets and cloud data platforms.
• Strong understanding of data modeling concepts (star/snowflake schema).
• Familiarity with version control systems such as GitLab.

Preferred Qualifications

• Experience with Azure services.
• Knowledge of CI/CD pipelines in a data engineering context.
• Background in data warehousing or big data solutions.