As part of LendingClub’s Big Data Platform team, you will directly support the Company's critical data architecture, and core development of batch and real time ingestion and analytics platform. The team is actively involved in both modernizing and migrating the platform to AWS.
What You'll Do
- You will leverage your Agile development knowledge and technical skills to design, develop, and maintain LendingClub’s BigData Ingestion and Analytic Platform
- You will design, maintain, monitor and automate cloud-based services in AWS for Big Data
- You will implement and improve Continuous Integration/Continuous Delivery environments
- You have demonstrated skills at troubleshooting and diagnosing problems in the data pipeline
- You will build security, monitoring, alerting and reporting infrastructure for the platform
- Plan and execute performance tuning & testing
- Participate in a 24/7 on-call rotation to ensure system stability and efficiencies
- You are committed to teamwork and have a passion for staying current with industry news, advancements and open source initiatives
- BS/MS in Computer Science or similar
- 3-5 years’ experience in Java Development
- 2+ years’ experience in Big Data Space – Hadoop (HDFS, YARN), Hive, Spark, Presto
- Experience with cloud data platform solutions like AWS EMR, AWS S3, Qubole, Databricks etc.
- Exposure to different file formats like AVRO, Parquet, ORC etc
- Experience in building and maintaining Realtime Data Ingestion system with Kafka/Kinesis, Spark Streaming
- Experience in scripting in languages like Python and shell scripting
- Experience in any Scheduler – Oozie/Airflow
- Scala experience desired
- Experience delivering solutions on AWS or other large scale cloud platforms desired
- You have exposure to Cassandra or any NoSQL database
- You have experience in building AI and ML platform like TensorFlow, Keras, Scikit-learn, H2O, Spark ML Lib, R etc
LendingClub is an equal opportunity employer and dedicated to diversity and inclusion in the workplace. We do not discriminate on the basis of race, religion, color, national origin, sex, gender, gender identity, sexual orientation, age, marital status, pregnancy status, veteran status, or disability status. We believe that a variety of perspectives will make our teams and business stronger as we work together to transform the traditional banking system.