Multi-Objective Evaluation Utilities

This repository contains two Python scripts used to post-process multi-objective optimization experiments:

• quality_indicator.py computes Pareto-based quality indicators across multiple runs and generates comparison plots.
• resource_usage.py aggregates runtime and resource metrics from experiment logs and produces trend visualizations and summaries.

Prerequisites

• Python 3.10+ (tested with Python 3.11)
• Dependencies listed in requirements.txt (install with pip install -r requirements.txt)

Expected Data Layout

Both scripts assume experiment outputs exist in sibling folders to the project root. By default the experiments are named:

• nsgaii-ccm-eval-102-surrogate-50
• nsgaii-ccm-eval-102-surrogate-false

Each experiment should contain multiple runs structured as:

<experiment>/
  run1/
    experiment.json
    algo_perf_stats.json
  run2/
    experiment.json
    algo_perf_stats.json
  ...

Adjust the experiment names in the scripts if your folders differ.

Usage

Quality indicators

quality_indicator.py merges Pareto fronts from all runs, computes metrics (HV, IGD+, GD+, epsilon) with pymoo and jMetalPy, and saves per-metric comparison plots.

Run from the project root:

python quality_indicator.py

Outputs: PNG figures named like hv_quality_indicator_comparison.png in the current directory.

Resource usage analysis

resource_usage.py loads algo_perf_stats.json files, normalizes differing JSON shapes, and computes mean/std trends for detected numeric resource columns. It also derives execution time and memory summaries when available.

Run from the project root:

python resource_usage.py

Outputs are written under results/resource_trends/, including per-resource plots, an overview grid, optional CSV summaries, and markdown/ASCII tables for execution times.

Notes

• The scripts rely on Matplotlib and Seaborn; a non-headless environment or appropriate backend may be needed for figure generation
• No external credentials or user-specific configuration are required; paths are relative to the repository root.