Note: The job is a remote job and is open to candidates in USA. Nexere Consulting is seeking a Data Engineer to play a key role in the development of their cloud-based data platform. The role involves designing and delivering scalable data pipelines and lakehouse solutions using Azure technologies to ensure high-quality data for analytics and reporting.
Responsibilities
- Design, build and support scalable data platforms, pipelines and lakehouse solutions using Azure Databricks, Azure Data Factory and Python/PySpark
- Partner with business stakeholders, analysts and technical teams to understand data requirements and deliver robust data solutions that support reporting, analytics and operational needs
- Develop and maintain data ingestion, transformation and orchestration processes, integrating data from a variety of internal and external sources
- Ensure data solutions are performant, reliable and maintainable, applying best practices in data engineering, automation and distributed processing
- Monitor, troubleshoot and optimise data pipelines, proactively addressing issues relating to data quality, availability and performances
Skills
- Commercial experience in a Data Engineering role, with strong expertise in Python/PySpark and the Azure data ecosystem
- Hands-on experience with Azure Databricks, Azure Data Factory, Azure SQL, Azure Data Lake and related cloud technologies
- Strong understanding of data modelling, ETL/ELT development and data integration principles
- Knowledge of modern cloud-based data architectures, distributed computing concepts and large-scale data processing
- Experience using Infrastructure as Code tools such as Terraform or Bicep, alongside CI/CD practices and Azure Devops
- Excellent communication and stakeholder management skills, with the ability to engage effectively with both technical and business audiences
Benefits
- Benefits
Company Overview