£300 - £310 per day
about 2 months ago
Target Rate: £310 a day
IR35 Determination: Inside IR35
Location: Remote with some meetings in Edinburgh
Duration: 6 Months initially
Data Engineer: You will join a blue chip consultancy, working in an innovative team tasked with creating and managing Big Data infrastructure and tools, typically designing and building big data pipelines and platforms.
You will have ability with a combination of some of the following technologies:
- Extensive cloud expertise using one or two of the Azure/AWS technology stacks:
- Azure: ADLS, Databricks, Stream Analytics, SQL DW, COSMOS DB, Analysis Services, Azure Functions, Serverless Architecture
- AWS: S3, EMR, Glue, Kinesis, Redshift, Dynamo DB, RDS, AWS Lambda/Serverless
- Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
- Experience with object-oriented/object function scripting languages: Python, SQL, Scala, Spark-SQL etc.
- Experience in Data Engineering projects and relevant experience on Azure/AWS technology services Experience working in Agile development team Desired traits
- Experience with big data tools: Hadoop, Spark, Kafka, etc and relational SQL and NoSQL databases, including Postgres and Cassandra
- Graduate degree in Computer Science, Statistics, Information Systems or another quantitative field or equivalent experience.
Datalake Related Skills
Dynamo, Lambda, S3, IAM, Secrets manager, DMS, EC2, EMR, DMS, SQS, Cloudwatch
Cloud Storage, Composer, Cloud Functions, GCP Logging
Terraform / Cloudformation / Polumi / Ansible
Jenkins config & pipelining
Postgres / Cassandra / SQL Server / SQL expertise & Schema mgmt experience
Snowflake BI / Parquet
Spark / Databricks
Apply now and I will call to discuss this role and your situation in more depth.