What you’ll do
● Work together with other teams to understand their current processes and missing opportunities around data engineering and machine learning development.
● Work on solutions to cover those needs, which might involve either exploring the state of the art of existing solutions, developing new ones or something in between.
● Work closely with those teams to help them onboard our platform and services, track their progress and learn from their feedback.
● Build highly scalable solutions for Big Data and Machine Learning on top of a wide spectrum of technologies (like:
○ Cloud providers like AWS, GCP or Azure
○ Containerized applications with Docker and Kubernetes
○ Orchestration tools like Kubeflow Pipelines, Argo Workflows
○ Computation frameworks like Hadoop, Apache Spark
○ Programming languages like Python, Go or Rust
○ Machine Learning frameworks like TensorFlow, PyTorch, Spark ML, Torch, Keras
○ Continuous integration and continuous development tools (CI/CD) like Jenkins, Travis, Concourse, Tekton, Flux or Argo CD)
● Provide monitors and automation of the operations and production of large-scale distributed systems
● Do it together with interdisciplinary teams composed of devops/software/data/ML engineers, data scientists, analysts and product specialists.
Who you are
● A BSc (or equivalent) degree in computer science
● Experience with cloud providers (AWS, GCP, DigitalOcean)
● Strong analytical / problem solving skills
● Strong UNIX background
● The ability to write scripts to diagnose problems (Python or similar)
● Proven ability and experience developing computer programs (Golang, Java, Python, Ruby or similar)
● Experience in building systems that scale
● Experience with modern deployment tools for CI/CD