Data Engineer at Anglo American

Job Description

Ongoing Job Vacancy at Anglo American for the position of Data Engineer for immediate employment.

Duties & Responsibilities

The successful applicant will be responsible for but not limited to the following job functions:

What you’ll do

• Develop ETL pipelines. The data transformations will be developed in Azure Databricks using Python and on Azure SQL using T-SQL and deployed using ARM templates
• Combine and curate data in a central data lake
• Serve data for application and analytics through a variety of technologies such as SQL, Server Synapse, Cosmos DB and TSI
• Build transformation pipelines into dimensions and facts and therefore a strong knowledge of standard BI concepts is mandatory
• Build stream pipelines leverage IoT Hub, Event Hub, Databricks streaming and other Azure stream technologies.
• Work in a fluid environment with changing requirements whilst maintaining absolute attention to detail.

 Required Knowledge and Experience

We pride ourselves with having the best people, which are our most important assets. Our company has been recognized for having the highest ethics and strives for excellence through distinctly higher standards than the norm.

We therefore urge only candidates with these unique requirements and experience to apply for this stimulating position.

Qualifications:

• IM Professional Qualification (Degree/Diploma) or Relevant Qualification

Technical Knowledge/Experience

• 5 years relevant experience in data engineering
• Strong experience of building large scale file shipping and pipelines, ideally using Azure services such as AzCopy and Azure Data Lake. Experience of managing unstructured file meta-data, conversions, standardisation and related workflows.
• Experience of building analysis jobs that scale on technologies such as Databricks or Azure Batch.
• Data Lake
• Data Modelling
• Data Architecture
• Azure Data Environment

How to apply: Interested and qualified? Visit  www.debeersgroup.com South Africa

Job Alert: Join Us on Telegram Now !!