Job Description

Your Key Responsibilities:
- Design and build data pipelines using Spark-SQL and PySpark in Azure Databricks.
- Build and maintain a Lakehouse architecture in ADLS / Databricks.
- Perform data preparation tasks including data cleaning, normalization, deduplication, type conversion etc.
- Work with DevOps team to deploy solutions in production environments.
- Control data processes and take corrective action when errors are identified. Corrective action may include executing a work around process and then identifying the cause and solution for data errors.
- Participate as a full member of the global Analytics team, providing solutions for and insights into data related items.
- Collaborate with your Data Science and Business Intelligence colleagues across the world to share key learnings, leverage ideas and solutions and to propagate best practices. You will lead projects that include other team members and participate in projects led by other team members.
- Apply change management tools including training, communication, and documentation to manage upgrades, changes, and data migrations.

To be qualified for this role, you must possess the following:
- Bachelor’s degree or higher in STEM or related field.
- 2-5 years of relevant IT experience with good exposure to Cloud technologies.
- Experience using Databricks and proficiency in Microsoft Azure platform (incl. ADF and ADLS).
- Experience querying databases and strong programming skills PySpark, Python, SQL, etc.
- Strong knowledge of data concepts and tools.
- Ability to be an effective team member with remote colleagues spread across the globe.
- Strong communication skills in English language (both written and spoken).
- Preference will be given to these candidates
- Immediate joiners
- Knowledge of distributed systems (e.g., Apache Spark or Databricks)
- Knowledge of Delta Lake and Lakehouse architecture
- Experience with multiple data programming languages (Spark-SQL, PySpark, Pandas)
- Experience with On-premises databases such as SQL server, Oracle etc.
- Experience with version control (e.g., Git), DevOps, CI/CD