Requirements
● Bachelor’s in Computer Science or another related fields.
● Over 3 years of experience in a data engineer role.
● The person should also have experience using the following software, tools.
● Big data tools: Hadoop (YARN, HDFS), Spark, Kafka.
● Relational SQL and NoSQL databases, including Postgres and Cassandra.
● Data pipeline and workflow management tools: Airflow, Luigi.
● Stream-processing systems: Storm, Spark-Streaming.
● Object-oriented/object function scripting languages: Python.
● Experience building and optimizing data ingestion pipelines.
● Being comfortable with working on Linux operated machines.
● A successful history of manipulating, processing and extracting value from large disconnected datasets.
● Strong project management and organizational skills.
● Experience supporting and working with cross-functional teams in a dynamic environment.