Data Engineer

Data Engineer

Target Rate: £310 a day

IR35 Determination: Inside IR35

Location: Remote with some meetings in Edinburgh

Duration: 6 Months initially

Data Engineer: You will join a blue chip consultancy, working in an innovative team tasked with creating and managing Big Data infrastructure and tools, typically designing and building big data pipelines and platforms.

You will have ability with a combination of some of the following technologies:

• Extensive cloud expertise using one or two of the Azure/AWS technology stacks:
• Azure: ADLS, Databricks, Stream Analytics, SQL DW, COSMOS DB, Analysis Services, Azure Functions, Serverless Architecture
• AWS: S3, EMR, Glue, Kinesis, Redshift, Dynamo DB, RDS, AWS Lambda/Serverless
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• Experience with object-oriented/object function scripting languages: Python, SQL, Scala, Spark-SQL etc.
• Experience in Data Engineering projects and relevant experience on Azure/AWS technology services Experience working in Agile development team Desired traits
• Experience with big data tools: Hadoop, Spark, Kafka, etc and relational SQL and NoSQL databases, including Postgres and Cassandra
• Graduate degree in Computer Science, Statistics, Information Systems or another quantitative field or equivalent experience.

Datalake Related Skills

GCP Services

AWS

Dynamo, Lambda, S3, IAM, Secrets manager, DMS, EC2, EMR, DMS, SQS, Cloudwatch

Cloud Storage, Composer, Cloud Functions, GCP Logging

Terraform / Cloudformation / Polumi / Ansible

Jenkins config & pipelining

Python

Git

Postgres / Cassandra / SQL Server / SQL expertise & Schema mgmt experience

K8s, helm

Kafka

Airflow

Snowflake BI / Parquet

Spark / Databricks

Datadog

Apply now and I will call to discuss this role and your situation in more depth.