Lead critical efforts designing and building our next generation Streaming Data Platform, leveraging PubSub, Finagle, Scala, Dataflow, and Google Cloud Platform.
Build scalable Dataflow data pipelines leveraging Airflow scheduler/executor framework
Design our data models for optimal storage and retrieval and to meet machine learning modeling needs.
Contribute to shared Data Engineering tooling & standards to improve the productivity and quality of output for Data Engineers across the company
Take an active role in the technical growth of other team members and the processes that support overall team growth
Lead or participate in code and technical design reviews
What we are looking for:
5+ years of significant professional experience in server-side development written in Scala, Java, Go, or Python
5+ years of professional experience developing large-scale, high-volume services using Finagle, Akka, or gRPC to handle petabytes of data
5+ years of experience managing data pipelines and infra using Dataflow, Spark, Hadoop, Flink, and Airflow.
Professional experience across the entire software development process, from development environment, to CI & CD including CircleCi, Jenkins, or TravisCi, to production deployment, product and application performance measurement
Confident articulating complex technical content to peers and partners across the organization via Technical Design Documents.
Strong ability to communicate
Proven ability in mentorship of junior engineers.
What we would like to see:
DevOps experience with Google Cloud Platform, AWS, or Azure
Data quality experience especially using & improving internal tools to automatically detect issues
Experience with Data Quality tools including Deequ or Apache Griffin
Experience building Scala based services in a production environment
Apply for the Job
Unleash the next generation of your enterprise with our customised technology solutions. Contact Us
Get In Touch
USA OFFICE 405 State Highway 121 Byp, Suite A250, Lewisville, Texas, 75067