Data Engineer | Financial Services | London
My client is a Large Global Consulting firm based in London and they are on the market looking for a Data Engineer to work on a large scale data project for a Financial services firm.
- Experience creating data pipelines, to provision data from a variety of sources using tools like Spark, Azure Data Factory and Kafka.
- Identify appropriate data processes, structures, and models to meet customer objectives and have experience performing data transformations that meet customer and organisational needs.
- Will be proficient at coding in languages such as Python, PySpark, Scala, and writing SQL
- Databases, modelling, and data flows: Common relational, non-relational and spatial databases. Workflow and ETL/ELT tools and methods.
- Cloud: Azure, AWS or GCP. Docker, Kubernetes, helm, data lake storage
- Web application development: Python frameworks (Flask, FastAPI), basic HTML/CSS/JS, frontend frameworks (like Angular)
- Programming: Linux terminal (e.g. ssh, bash, cron), APIs, developer collaboration (e.g git, jira, confluence)
- Data Science/AI principles: Principles of machine learning. Model serving.
- Agile and DevOps: Kanban, scrum, CI/CD, test automation, monitoring, and alerting