Implementing a Hierarchical Deep Learning Approach for Simulating Multilevel Auction Data

Abstract

We present a deep learning solution to address the challenges of simulating realistic synthetic first-price sealed-bid auction data. The complexities encountered in this type of auction data include high-cardinality discrete feature spaces and a multilevel structure arising from multiple bids associated with a single auction instance. Our methodology combines deep generative modeling (DGM) with an artificial learner that predicts the conditional bid distribution based on auction characteristics, contributing to advancements in simulation-based research. This approach lays the groundwork for creating realistic auction environments suitable for agent-based learning and modeling applications. Our contribution is twofold: we introduce a comprehensive methodology for simulating multilevel discrete auction data, and we underscore the potential of DGM as a powerful instrument for refining simulation techniques and fostering the development of economic models grounded in generative AI.

Appendices

Appendix A: SEAO Dataset

Table 5 outlines the variables used in our study. It is important to note that geographical variables like countries or states were excluded because, although present in the raw data, only contracts from the province of Quebec are actually recorded in the subset we accessed.

Table 5 Description of the variables used in the study

The cleaning of the raw data involved selecting relevant variables. Many columns in the raw data were excluded as they did not provide informative signals due to their nature (e.g., web links) or redundancy with other variables. The quality of the signals was another selection criterion. Columns like temporal variables or textual entries were omitted due to low quality, evidenced by inconsistencies, excessive missing values, and non-uniform entry formats. Despite centralized dataset management by SEAO, data inconsistencies and missing values are common due to manual updates by various public entities’ administrators. These are significant limitations of this dataset.

The raw data, is available at https://www.donneesquebec.ca/recherche/dataset/systeme-electronique-dappel-doffres-seao, and the official PDF description file at https://www.donneesquebec.ca/recherche/dataset/d23b2e02-085d-43e5-9e6e-e1d558ebfdd5/resource/af41596c-b07f-4664-82c8-577e1ef9a6f3/download/seao-specificationsxml-donneesouvertes-20171010.pdf. Data for each year, and in some cases each month, must be fetched separately. The raw data, provided in XML format, needs conversion into a workable tabular array. We utilized the Python "xml" library to convert and save the data in pickle (.pkl) format. The code for processing the original XML files is available in the associated GitHub repository for this manuscript. Note that we also provide the cleaned and preprocessed pickle file.

Appendix B: Methodology Overview

Initially, the SEAO dataset underwent a thorough cleaning process. This involved handling missing values, removing irrelevant columns, and reformatting specific columns to ensure their consistency and reliability.

Following the data cleaning, preprocessing was conducted to transform the dataset and make it suitable for machine learning applications. Discrete variables were transformed using one-hot encoding techniques, while continuous bid values were standardized.

To generate synthetic data, two primary generative models were utilized: CTGAN and TVAE. The CTGAN model was trained using an array of hyperparameters, including distinct embedding dimensions, generator and discriminator dimensions, learning rates, and specific decay rates. Likewise, the TVAE, a variant that incorporates a variational autoencoder structure, was trained with specific parameters, including hidden and latent dimensions.

Once trained, these models were then employed to sample synthetic datasets, replicating the patterns and distributions seen in the original SEAO dataset.

Subsequent to the synthetic data generation, we introduced BidNet neural network model. This model was designed to predict bid values using both discrete and continuous inputs. For training efficiency, the model utilized cross-validation and early stopping methodologies. Several hyperparameters, including learning rate, batch size, and number of epochs, were tuned manually to enhance the model’s performance. Alternatively, rigorous automated tuning procedures (e.g., Bayesian optimization) can be used, provided enough computational resources and time are available.

To assess the quality of the synthetic data produced by CTGAN and TVAE, a series of classifiers, including Decision Trees, k-Nearest Neighbors, and Neural Networks, were trained on both the real and synthetic datasets. Performance metrics from these classifiers provided insights into the fidelity and utility of the synthetic data.

Finally, BidNet model’s performance was critically evaluated using various metrics. These metrics, namely the Root Mean Square Error (RMSE), Jensen-Shannon distance (JS), and Wasserstein distance (WS), compared the synthetic and real bids, giving a comprehensive understanding of the model’s accuracy and effectiveness.

Throughout the entire process, special attention was given to reproducibility. Foundational functionalities ensured consistent random states, allowing for deterministic behavior across runs. Additionally, capabilities were established to save intermediate results, trained models, and to manage computation across various devices, whether CPU or GPU.

Appendix C: Algorithms

Algorithm 1

Training GANs-based auction features generator

Algorithm 2

Training tabular VAE for auction features

Algorithm 3

K-folds cross-validation BidNet training procedure

Algorithm 4

Synthetic bid validation / Double validation