Data Engineer

Job Description

________________________________________
Job Opportunity: Data Engineer – Recommender Systems & Big Data
Location: Tehran, Iran
Company: Ofogh Kourosh Chain Stores
Employment Type: Full-time
________________________________________
About Us
Ofogh Kourosh is the largest hard-discount retail chain in Iran with over 4,700 stores nationwide. We are transforming retail with cutting-edge analytics and AI. Our flagship initiative is building a large-scale recommender system powered by big data pipelines (Spark/Hadoop) to personalize shopping experiences for millions of customers.
________________________________________
Your Role
As a Data Engineer, you will be the backbone of our Big Data & AI team. You will design, implement, and maintain the data infrastructure and pipelines that fuel our recommender system—from raw transaction logs in HDFS to feature stores, Spark-based models (ALS, K-Means), and GPU-powered deep learning systems. You will work closely with data scientists and ML engineers to ensure scalable, reliable, and high-quality data flows.
________________________________________
Key Responsibilities
. Design, build, and maintain scalable ETL pipelines for billions of retail transactions and customer data.
. Manage and optimize a Hadoop/Spark cluster for large-scale data processing.
. Implement data ingestion workflows (e.g., Airflow + PySpark + SQL Server) to bring daily transactional data into the data lake (HDFS/Parquet).
. Build and maintain customer feature stores (RFM scores, frequency, recency, segmentation).
. Develop baseline models (ALS, K-Means) in Spark MLlib and prepare training datasets for deep learning models (PyTorch/TensorFlow on GPU).
. Collaborate with ML engineers to integrate deep learning models into the data pipeline.
. Ensure data quality, lineage, and governance across all stages.
. Automate workflows with Apache Airflow and monitor system health/performance.
. Support deployment of results into databases (SQL Server) and API layers (FastAPI).
. Experience with Docker/Kubernetes and Linux system administration.

________________________________________
Requirements
Must Have:
. Bachelor’s/Master’s degree in Computer Science, Software Engineering, or related fields.
. Strong hands-on experience with Apache Spark (PySpark, DataFrame API) and distributed data processing.
. Advanced SQL (SQL Server) – schema design, query optimization, large joins.
. Solid Python programming skills (pandas, pyarrow, ETL scripting).
. Familiarity with big data storage formats (Parquet, Delta Lake) and HDFS.
. Experience with workflow orchestration (Apache Airflow).
. Understanding of ETL, data modeling, and data warehouses.
Nice to Have (big plus):
. Exposure to recommender systems, personalization, or customer analytics.
. Familiarity with machine learning pipelines (MLlib, scikit-learn, or PyTorch/TensorFlow).
. Knowledge of GPU environments (CUDA, NVIDIA drivers) for deep learning.
. Previous work with retail or e-commerce data at scale.
________________________________________
What We Offer
. Be part of one of the largest recommender system projects in Iran’s retail industry.
. Work with cutting-edge big data & AI technologies (Spark, Hadoop, PyTorch, TensorFlow, Airflow).
. Access to powerful hardware infrastructure (multi-node CPU clusters + GPU servers).
. A dynamic, fast-growing team at the intersection of AI and retail.
. Competitive salary, benefits, and career growth opportunities.