Data Engineer – AWS

Data Engineer - AWS

Data Engineer - AWS

The Data Engineer will have a key role in helping to define and develop a new Data Lake to drive a new data-driven business strategy. The Data Engineer will help propose and implement solutions primarily based on AWS, including:

  • designing and implementing ETL tasks.
  • identifying key data relationships.
  • cleansing data.
  • building a framework for BI developers to generate reporting solutions.
  • orchestrating the flow of data.
  • Some experience in Map Reduce and Machine Learning is desirable.

The role will involve working with colleagues, both IT and business, to help define the longer-term Data Lake strategy., with a focus on agile working and continuous delivery. The initial scope is to focus on retail data, followed by financial data. Currently, AWS tools are being used, however, open-source solutions may be considered in the near future.

Key Skills:

  • Cloud environments.
  • ETL, data migration, and cleansing.
  • Hands-on Apache Spark using Python or Scala, creating data frames and applying transformations/actions.
  • Data Lakes, data analytics, and working with large data sets.
  • Relational databases (particularly Oracle and PostgreSQL)
  • Working independently.
  • Agile working.
  • Customer-first approach to development.


  • Hands-on experience with AWS.
  • Hands-on experience with AWS Glue, Lambda, and Step functions.
  • Hands-on experience with SQL, Python, and Scala
  • Familiarity with AWS data lake-related components such as S3, Athena, Presto, QuickSight, EMR, Redshift, Kinesis, etc.
  • Familiarity with structured RDBMS.
  • Familiarity with large data techniques (Hadoop, MapReduce, Spark, etc.)
  • Familiarity with providing data via a microservice API.
  • Experience in analyzing and cleansing data using a variety of tools and techniques.
  • Experience in working with a variety of data formats, JSON, XML, CSV, parquet, etc.
  • Experience with building and maintaining data dictionaries / meta-data.
  • Experience in Linux and cloud environments.
  • Desirable, but not essential, to have retail experience.
Job Category: Data Engineering
Job Type: Contract
Job Location: London

Apply for this position

Allowed Type(s): .pdf, .doc, .docx