Role Overview
Smarkets operates a high-volume prediction market platform handling £29B+ in volume. The Data Team builds ETL pipelines, ML models, and analytics to drive business insights. This role focuses on data engineering, API development, and ML model deployment within a modern stack (Linux, Kafka, Kubernetes, Python, Rust, React).
Responsibilities
- Develop and maintain real-time ETL pipelines using Python frameworks (Luigi, Airflow)
- Build and deploy Flask/FastAPI services with PostgreSQL integration
- Maintain data warehouse health (AWS Redshift, Google BigQuery) and data lake integrity
- Automate reporting pipelines for stakeholders and third parties
- Train and deploy ML models for user segmentation, anomaly detection, and recommender systems
Requirements
- 3+ years of experience
- ETL pipeline development with Python (Luigi or Airflow)
- Python REST API development and PostgreSQL integration
- Proficiency with Pandas, NumPy, scikit-learn
- Experience training and deploying ML models