Staff Data Engineer

Melbourne, AU

Job Description / Skills Required

Swift Navigation provides precise positioning solutions for automotive, autonomous vehicle, mobile and mass-market applications. What began as the GNSS industry’s first low-cost, high-accuracy, real-time kinematic (RTK) receiver has evolved into a Swift Navigation ecosystem of positioning solutions for autonomous applications. From the nationwide GNSS corrections delivered from the cloud by the Skylark™ precise positioning service, the hardware-independent, integrated software solution that is the Starling® positioning engine to the centimeter-level accurate Piksi® Multi and ruggedized Duro® and Duro Inertial RTK receivers, Swift Navigation is enabling a future of autonomous vehicles to navigate and understand the world. Learn more online at swiftnav.com, or follow Swift on Twitter @Swiftnav

Meet The Team

The data platform product team at Swift Navigation is building a toolkit for location-enabled applications across the last-mile delivery, machine automation, and fleet management industries. Our product team consists of software engineers, product managers, and works closely with the CTO. We are designing systems to ingest real-time location data from tens of thousands of customer devices and transform that data into business intelligence to provide solutions to a wide range of customers

What You’ll Do: 

  • Selecting products / solutions for setting up data infrastructure that meets the need of the data product platform
  • Setup / establish cloud infrastructure for the selected data solutions in cloud providers such as AWS and GCP
  • Design and implement scalable and reliable systems for ingestion, processing, and real-time analysis of large, disparate data sets from diverse sources, deployed on AWS using tools such as MSK, Kinesis, AWS EMR, AWS Athena, AWS Glue, Lambdas, API Gateway etc, or champion the adoption of GCP equivalents
  • Develop tools and applications to proactively measure, monitor, and improve data quality and consistency during loading and analysis processes
  • Analyze and improve efficiency, reliability, and scalability of data infrastructure and processes
  • Work with data team to define and promote best practices for data management and analysis, and to build and improve systems to implement and support these practices
  • Build appropriate logging of metrics and diagnostics in code to enable effective monitoring and data observability using tools like Splunk, DataDog, Grafana or Prometheus
  • Design, architect, build and scale a modern and cloud-native data platform with requisite tooling and APIs necessary to help democratize access to data
  • Manage the orchestration of batch jobs with solutions like Apache Airflow
  • Support building modular set of data services for data transformation and business logic processing using container microservices like Apache Spark on AWS EMR, ETL technologies like AWS Glue, dbt and SQL

What You'll Need To Succeed

  • Experience in building highly scalable data architectures from scratch on AWS or GCP
  • Experience with native cloud development on the AWS ecosystem and event-driven architectures and pub-subs like Kafka, Kinesis, AWS Event Bridge, Flink, web sockets and SSE is a must
  • Experience retrieving, parsing, cleaning, and transforming data from multiple formats and delivery mechanisms
  • Experience with a modern data engineering platform on AWS using MSK, Kinesis, AWS EMR, AWS Athena, AWS Glue, Lambdas, API Gateway etc.
  • Experience with batch data processing using tools like Apache Spark
  • Experience with structured Spark streaming is highly desired
  • Familiarity and experience with data pipeline frameworks, such as Luigi
  • Experience with gRPC and Protocol buffers is highly desired
  • Familiarity with machine learning and data science ecosystems such as AWS Sagemaker and Databricks