Download App
Data Engineer with Scala and Spark/ Pyspark
Capco
Actively HiringFull-time
Location
MH, IN
Experience
4+ years
Role
Data Engineer with Scala and Spark/ Pyspark
Salary
Competitive
Your Application Toolkit
Stand out. Get hired.
80% of applicants fail the initial ATS screen. Check your compatibility before you apply and stand out from the crowd.
Share this job
Know someone who'd be a great fit?
Job Description
Role Overview
Capco, a Wipro company, is a global technology and management consulting firm supporting over 100 clients across banking, financial, and energy sectors. We are looking for a Data Engineer with 4+ years of experience to join our team in a hybrid work mode. You will design and optimize large-scale data processing applications to transform the financial services industry.
Responsibilities
- Design, develop, and optimize large-scale data processing applications using Scala, Apache Spark, and Java.
- Build and maintain high-performance data pipelines capable of processing 80–90 million records daily.
- Develop and integrate Native APIs and data services to support business-critical applications.
- Optimize Spark jobs, data workflows, and distributed computing processes for high throughput and low latency.
- Implement best practices for data quality, monitoring, governance, and operational excellence.
- Troubleshoot and resolve performance bottlenecks across data processing and ingestion pipelines.
- Participate in code reviews and contribute to engineering standards and architecture decisions.
Requirements
- Strong hands-on experience in Scala, Apache Spark, and Java.
- Proven experience building and supporting large-scale distributed data processing systems.
- Expertise in developing and consuming Native APIs and microservices.
- Strong understanding of Spark architecture, performance tuning, partitioning, and caching.
- Experience with data modeling, ETL/ELT processes, and large-scale batch and streaming pipelines.
- Solid understanding of distributed systems, concurrency, and high-volume data processing.
- Experience with SQL and relational/non-relational databases.
Nice to Have
- Experience in Financial Services, Payments, or FinTech domains.
- Exposure to cloud platforms such as AWS, Azure, or GCP.
- Familiarity with Kafka, Airflow, or the Hadoop ecosystem.
- Experience with CI/CD pipelines, DevOps, and Agile methodologies.
Skills
- Scala
- Apache Spark
- Java
- Native APIs
- SQL
Required Skills
ScalaApache SparkJavaNative APIsSQL
Ready to make your move?
Take the next step in your career. Apply directly on the company's platform.
Apply for this role