We are looking for a Platform Engineer that will focus on enhancing a data driven business NinthDecimal. We process terabytes of data and that presents the engineering team with great opportunities and challenges. The responsibilities for this position include these activities:
- Design and implementation of the data driven initiatives at NinthDecimal such as Big Data analytics application development and architecture using Kafka, Spark, Hadoop, HBase, etc.
- Optimize data pipeline on Hadoop using Kafka or Spark streaming, and optimal use of Spark jobs
- Optimize devops setup around platform systems
- Collaborate with other engineers to formulate innovative solutions to experiment and implement advanced data algorithms
- Communicate complex concepts and the results of the analyses in a clear and effective manner to Product Owners
- Expert in Scala. Python, Java a plus
- Expert hands-on experience with Hadoop based technologies, such as Spark, Hive, HBASE
- Previous experience in dealing with hundreds of Petabytes of data
- Expert in complex SQL optimization
- Experience with building and optimizing high performance systems (caching, queueing, GC tuning, etc.)
- Be an excellent communicator and a leader able to articulate complex decisions to the cross functional team
- Experience in dealing with the terabytes per day data growth
- Advance degree in CS, Math or related subject is preferred
- Familiarity with the digital media / advertising industry is a big plus
- Advanced ping-pong skills are a plus J.