This project predicts item ratings in a retail environment where product shipments and a comprehensive master dataset come together. Integrating inventory details, dispatch records and item attributes gives a unified view of each product's lifecycle; once the data is prepared, machine learning models classify the final rating of new items, which helps decide inventory distribution, promotions and product line management.

The workflow is split into three scripts:

  1. data_preparation.py — gathers, cleans and merges raw data from multiple sources.
  2. data_processing.py — feature selection, train/test split and preprocessing pipelines.
  3. model.py — trains several algorithms (neural network, decision tree, KNN) and generates predictions.

1. Data preparation

A small SQL helper queries shipment data from the warehouse, process_table normalizes text and cleans prices, and load_data reads the item master. main() merges everything with historical ratings and returns a single DataFrame the other scripts can use. 

# data_preparation.py
from sqlalchemy import create_engine
import pandas as pd

class SQL:
    def __init__(self, cnn_str):
        self.engine = create_engine(cnn_str)

    def shipments(self):
        query = "SELECT * FROM shipments_table WHERE ...;"
        return pd.read_sql(query, self.engine)

def process_table(df):
    df['precio'] = (df['precio'].str.replace('$', '', regex=True)
                                .str.replace(',', '', regex=True)
                                .astype(float))
    # unify text, categories, colors, silhouettes...
    return df

def load_data(mes_exh=False, columns_to_remove=None):
    items_df = process_table(pd.read_csv("items_master.csv", sep='\t'))
    if mes_exh:
        items_df['fecha_exhibicion'] = mes_exh
    for col in columns_to_remove or []:
        items_df.drop(col, axis=1, inplace=True, errors='ignore')
    return items_df

def main():
    shipments_df = SQL('postgresql://user:password@host:port/db').shipments()
    items_df = load_data()
    merged = items_df.merge(shipments_df, on='item', how='inner')

    rating_df = pd.read_excel("item_rating.xlsx").iloc[:, :2]
    rating_df.columns = ['item', 'calificacion']
    final = merged.merge(rating_df, on='item', how='inner')

    final['fecha_exhibicion'] = pd.to_datetime(final['fecha_exhibicion'])
    final = final[final['fecha_exhibicion'] > '2020-01-01']
    final['fecha_exhibicion'] = final['fecha_exhibicion'].dt.month.astype(object)
    return final.set_index('item')

2. Data processing

DataProcessor.data_final fetches the prepared data, optionally filters by gender and category, and splits it into training and test sets. create_pipeline chains categorical encoding, numerical scaling, feature selection with chi2 and the classifier itself, so every model is trained under exactly the same preprocessing. 

# data_processing.py
from sklearn.model_selection import train_test_split
from sklearn.compose import make_column_selector, ColumnTransformer
from sklearn.feature_selection import SelectKBest, chi2
from sklearn.pipeline import Pipeline
from data_preparation import main

class DataProcessor:
    @staticmethod
    def data_final(gender=None, category=None, rs=10):
        df = main()
        if gender:
            df = df[df['genero'] == gender]
        if category:
            df = df[df['categoria'] == category]
        X = df.drop(['calificacion', 'genero', 'categoria'], axis=1)
        y = df['calificacion']
        X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=rs)
        return X_train, X_test, y_train, y_test, X, y

    @staticmethod
    def create_pipeline(cat_encoder, model, num_encoder):
        return Pipeline([
            ('preprocessing', ColumnTransformer([
                ('cat', cat_encoder, make_column_selector(dtype_include='object')),
                ('num', num_encoder, make_column_selector(dtype_include='number')),
            ], remainder='passthrough')),
            ('feature_selection', SelectKBest(score_func=chi2)),
            ('classification', model),
        ])

3. Model training

ModelTrainer manages three classifiers — a neural network (MLP), a decision tree and K-Nearest Neighbors — trains each through the shared pipeline and can predict on a specific list of items, saving the results for review. 

# model.py
from sklearn.neural_network import MLPClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.preprocessing import OneHotEncoder, MinMaxScaler
from data_processing import DataProcessor

class ModelTrainer:
    def __init__(self, gender, exhibition_month):
        self.classifiers = [
            MLPClassifier(max_iter=1000, random_state=5),
            DecisionTreeClassifier(random_state=5),
            KNeighborsClassifier(n_neighbors=40),
        ]
        self.gender = gender
        self.exhibition_month = exhibition_month
        self.predictions = {}

    def train_models(self, X, y, item_list=None, test_mode=False):
        for clf in self.classifiers:
            pipeline = DataProcessor.create_pipeline(OneHotEncoder(), clf, MinMaxScaler())
            pipeline.fit(X, y)
            if item_list:
                X_items = self.load_items_for_prediction(item_list, test_mode)
                self.predictions[clf.__class__.__name__] = pipeline.predict(X_items)
        return self.predictions

if __name__ == "__main__":
    X_train, X_test, y_train, y_test, X, y = DataProcessor.data_final(gender='hombre', category='camisas')
    trainer = ModelTrainer(gender='hombre', exhibition_month=5)
    trainer.train_models(X_train, y_train, item_list=['itemA', 'itemB'])

Evaluation

Each model is evaluated with a confusion matrix. Comparing predictions with the actual ratings shows exactly where and how often a model is right or wrong, which classes need more attention, and whether certain patterns are being systematically misclassified.

Running it

  1. Install the dependencies: pip install -r requirements.txt
  2. Run the scripts in order: data_preparation.py → data_processing.py → model.py
  3. Tune gender, category and exhibition_month in model.py to match the segment you want to predict.

Key libraries: scikit-learn, pandas, numpy, sqlalchemy, openpyxl and matplotlib.