Smarkets logo for Data Engineer job on DevsUnite

Data Engineer

Smarkets

Actively HiringFull-time
Apply
Location

London Area, United Kingdom

Experience

3+ years

Role

Data Engineer

Salary

Competitive

Share this job

Know someone who'd be a great fit?

Job Description

Role Overview

Smarkets operates a high-volume prediction market platform handling £29B+ in volume. The Data Team builds ETL pipelines, ML models, and analytics to drive business insights. This role focuses on data engineering, API development, and ML model deployment within a modern stack (Linux, Kafka, Kubernetes, Python, Rust, React).

Responsibilities

  • Develop and maintain real-time ETL pipelines using Python frameworks (Luigi, Airflow)
  • Build and deploy Flask/FastAPI services with PostgreSQL integration
  • Maintain data warehouse health (AWS Redshift, Google BigQuery) and data lake integrity
  • Automate reporting pipelines for stakeholders and third parties
  • Train and deploy ML models for user segmentation, anomaly detection, and recommender systems

Requirements

  • 3+ years of experience
  • ETL pipeline development with Python (Luigi or Airflow)
  • Python REST API development and PostgreSQL integration
  • Proficiency with Pandas, NumPy, scikit-learn
  • Experience training and deploying ML models

Required Skills

PythonFlaskPostgreSQLAWS RedshiftApache Kafka

Ready to apply for Data Engineer?

See how you stack up at Smarkets before you apply, or go straight to the application.

Apply Anyway