Sr. Data Engineer
Are you interested in working on a platform to empower creators to tell their stories and connect with millions of viewers? We are currently working with one of the largest video streaming providers who is in 150 countries, and is one of the world's largest open video platforms.
They build analytical tools to help creators know where their viewers are from, where they click, and even track when a viewer stops watching.
They are looking to bring on Data Engineers to their Data team located in NYC. As a Data Engineer, you will build the tools and infrastructure that enables access to event streams. You will work with their business teams as well as other engineering teams to help define data access and discoverability requirements and implement systems to satisfy these requirements.
What You'll Do:
- Work with various teams across all parts of the business to define new events streamed through our pipeline
- Build ETL pipelines for events as needed
- Help construct a system for data discover ability from start to finish
- Review metrics and KPI's for monitoring and performance management
Skills & knowledge you should possess:
- +5 years of engineering experience
- Experience working with Airflow or any other Open Source Data Pipeline
- Experience with Java, Scala, or other JVM language experience
- Experience working with Apache Kafka
- Working knowledge of Hadoop, Spark, Flink, or similar data processing framework experience
- Relational databases and SQL experience
Additional useful skills; not required:
- Python or other scripting language experience a plus
- Go experience
- NoSQL database experience a plus
- Cloud Provider experience (AWS tools, Google Cloud tools, Azure tools)