Skip to content

Add fio, fio_scale, hammerdb_scale, sysbench workloads to summary report #1240

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ebattat merged 1 commit into from
Jun 22, 2026
Merged

Add fio, fio_scale, hammerdb_scale, sysbench workloads to summary report #1240

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
212 changes: 17 additions & 195 deletions benchmark_runner/jupyterlab/templates/summary_report/summary_report.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,21 +14,7 @@
"cell_type": "markdown",
"id": "465cbea8-2033-4569-9712-ad494eaa459d",
"metadata": {},
"source": [
"This notebook is designed to perform a PerfCI VM workloads comparison between OpenShift versions using the Geometric Mean \n",
"between the 2 most recent major versions.\n",
"\n",
" * HammerDB: Database TPM [Transaction Per Minute] for Mariadb, MSSQL, PostgreSQL\n",
" * Uperf: Network between two VMs\n",
" * Vdbench: VM Application storage: oltp1, oltp2, oltphw, odss2, odss128\n",
" * Bootstorm: Time till VM login\n",
"\n",
"How to use it:\n",
"\n",
"1. Run all and get the percentage results at the end.\n",
"2. If you want different versions instead of the default, choose them and rerun the 'Workloads comparison' cell.\n",
"3. The default number of fetch days is 365 days. It can be configured in the prerequisite variable ES_NUM_FETCH_DAYS = 365."
]
"source": "This notebook is designed to perform a PerfCI VM workloads comparison between OpenShift versions using the Geometric Mean \nbetween the 2 most recent major versions.\n\n * HammerDB: Database TPM [Transaction Per Minute] for Mariadb, MSSQL2025, PostgreSQL\n * HammerDB Scale: Database TPM (scaled 6 VMs)\n * Uperf: Network between two VMs\n * FIO: VM storage I/O performance: IOPS, Latency\n * FIO Scale: VM storage I/O performance (scaled 6 VMs)\n * Vdbench: VM Application storage: oltp1, oltp2, oltphw, odss2, odss128\n * Vdbench Scale: VM Application storage (scaled 6 VMs)\n * Bootstorm: Time till VM login\n * Sysbench: CPU and Memory performance\n\nHow to use it:\n\n1. Run all and get the percentage results at the end.\n2. If you want different versions instead of the default, choose them and rerun the 'Workloads comparison' cell.\n3. The default number of fetch days is 365 days. It can be configured in the prerequisite variable ES_NUM_FETCH_DAYS = 365."
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -139,186 +125,22 @@
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/html": [
"<style type=\"text/css\">\n",
"#T_ec1b0 th:not(:last-child) {\n",
" text-align: left;\n",
"}\n",
"#T_ec1b0 td:not(:last-child) {\n",
" text-align: left;\n",
"}\n",
"#T_ec1b0_row0_col3, #T_ec1b0_row3_col3, #T_ec1b0_row5_col3, #T_ec1b0_row6_col3, #T_ec1b0_row7_col3, #T_ec1b0_row8_col3, #T_ec1b0_row9_col3, #T_ec1b0_row11_col3, #T_ec1b0_row12_col3, #T_ec1b0_row13_col3, #T_ec1b0_row14_col3 {\n",
" color: green;\n",
" text-align: center;\n",
"}\n",
"#T_ec1b0_row1_col3, #T_ec1b0_row2_col3, #T_ec1b0_row4_col3 {\n",
" color: black;\n",
" text-align: center;\n",
"}\n",
"#T_ec1b0_row10_col3 {\n",
" color: orange;\n",
" text-align: center;\n",
"}\n",
"</style>\n",
"<table id=\"T_ec1b0\">\n",
" <thead>\n",
" <tr>\n",
" <th class=\"blank level0\" >&nbsp;</th>\n",
" <th id=\"T_ec1b0_level0_col0\" class=\"col_heading level0 col0\" >workload</th>\n",
" <th id=\"T_ec1b0_level0_col1\" class=\"col_heading level0 col1\" >metric</th>\n",
" <th id=\"T_ec1b0_level0_col2\" class=\"col_heading level0 col2\" >storage type</th>\n",
" <th id=\"T_ec1b0_level0_col3\" class=\"col_heading level0 col3\" >('4.13.29', '4.14.10')</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th id=\"T_ec1b0_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
" <td id=\"T_ec1b0_row0_col0\" class=\"data row0 col0\" >hammerdb</td>\n",
" <td id=\"T_ec1b0_row0_col1\" class=\"data row0 col1\" >TPM (mariadb)</td>\n",
" <td id=\"T_ec1b0_row0_col2\" class=\"data row0 col2\" >ODF</td>\n",
" <td id=\"T_ec1b0_row0_col3\" class=\"data row0 col3\" >3.58%</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_ec1b0_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
" <td id=\"T_ec1b0_row1_col0\" class=\"data row1 col0\" >hammerdb</td>\n",
" <td id=\"T_ec1b0_row1_col1\" class=\"data row1 col1\" >TPM (mssql)</td>\n",
" <td id=\"T_ec1b0_row1_col2\" class=\"data row1 col2\" >ODF</td>\n",
" <td id=\"T_ec1b0_row1_col3\" class=\"data row1 col3\" >-0.15%</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_ec1b0_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
" <td id=\"T_ec1b0_row2_col0\" class=\"data row2 col0\" >hammerdb</td>\n",
" <td id=\"T_ec1b0_row2_col1\" class=\"data row2 col1\" >TPM (postgresql)</td>\n",
" <td id=\"T_ec1b0_row2_col2\" class=\"data row2 col2\" >ODF</td>\n",
" <td id=\"T_ec1b0_row2_col3\" class=\"data row2 col3\" >-0.83%</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_ec1b0_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
" <td id=\"T_ec1b0_row3_col0\" class=\"data row3 col0\" >hammerdb</td>\n",
" <td id=\"T_ec1b0_row3_col1\" class=\"data row3 col1\" >TPM (postgresql)</td>\n",
" <td id=\"T_ec1b0_row3_col2\" class=\"data row3 col2\" >LSO</td>\n",
" <td id=\"T_ec1b0_row3_col3\" class=\"data row3 col3\" >12.07%</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_ec1b0_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
" <td id=\"T_ec1b0_row4_col0\" class=\"data row4 col0\" >uperf</td>\n",
" <td id=\"T_ec1b0_row4_col1\" class=\"data row4 col1\" >Throughput</td>\n",
" <td id=\"T_ec1b0_row4_col2\" class=\"data row4 col2\" >Ephemeral</td>\n",
" <td id=\"T_ec1b0_row4_col3\" class=\"data row4 col3\" >-4.6%</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_ec1b0_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
" <td id=\"T_ec1b0_row5_col0\" class=\"data row5 col0\" >uperf</td>\n",
" <td id=\"T_ec1b0_row5_col1\" class=\"data row5 col1\" >Latency</td>\n",
" <td id=\"T_ec1b0_row5_col2\" class=\"data row5 col2\" >Ephemeral</td>\n",
" <td id=\"T_ec1b0_row5_col3\" class=\"data row5 col3\" >2.83%</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_ec1b0_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
" <td id=\"T_ec1b0_row6_col0\" class=\"data row6 col0\" >vdbench</td>\n",
" <td id=\"T_ec1b0_row6_col1\" class=\"data row6 col1\" >Iops</td>\n",
" <td id=\"T_ec1b0_row6_col2\" class=\"data row6 col2\" >ODF</td>\n",
" <td id=\"T_ec1b0_row6_col3\" class=\"data row6 col3\" >4.52%</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_ec1b0_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
" <td id=\"T_ec1b0_row7_col0\" class=\"data row7 col0\" >vdbench</td>\n",
" <td id=\"T_ec1b0_row7_col1\" class=\"data row7 col1\" >Latency</td>\n",
" <td id=\"T_ec1b0_row7_col2\" class=\"data row7 col2\" >ODF</td>\n",
" <td id=\"T_ec1b0_row7_col3\" class=\"data row7 col3\" >6.15%</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_ec1b0_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
" <td id=\"T_ec1b0_row8_col0\" class=\"data row8 col0\" >vdbench_scale</td>\n",
" <td id=\"T_ec1b0_row8_col1\" class=\"data row8 col1\" >Iops</td>\n",
" <td id=\"T_ec1b0_row8_col2\" class=\"data row8 col2\" >ODF</td>\n",
" <td id=\"T_ec1b0_row8_col3\" class=\"data row8 col3\" >27.43%</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_ec1b0_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
" <td id=\"T_ec1b0_row9_col0\" class=\"data row9 col0\" >vdbench_scale</td>\n",
" <td id=\"T_ec1b0_row9_col1\" class=\"data row9 col1\" >Latency</td>\n",
" <td id=\"T_ec1b0_row9_col2\" class=\"data row9 col2\" >ODF</td>\n",
" <td id=\"T_ec1b0_row9_col3\" class=\"data row9 col3\" >23.11%</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_ec1b0_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
" <td id=\"T_ec1b0_row10_col0\" class=\"data row10 col0\" >bootstorm</td>\n",
" <td id=\"T_ec1b0_row10_col1\" class=\"data row10 col1\" >240 VMs run time (fedora37)</td>\n",
" <td id=\"T_ec1b0_row10_col2\" class=\"data row10 col2\" >Ephemeral</td>\n",
" <td id=\"T_ec1b0_row10_col3\" class=\"data row10 col3\" >-33.83%</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_ec1b0_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
" <td id=\"T_ec1b0_row11_col0\" class=\"data row11 col0\" >bootstorm</td>\n",
" <td id=\"T_ec1b0_row11_col1\" class=\"data row11 col1\" >111 VMs run time (windows10)</td>\n",
" <td id=\"T_ec1b0_row11_col2\" class=\"data row11 col2\" >ODF (virtio)</td>\n",
" <td id=\"T_ec1b0_row11_col3\" class=\"data row11 col3\" >23.8%</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_ec1b0_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
" <td id=\"T_ec1b0_row12_col0\" class=\"data row12 col0\" >bootstorm</td>\n",
" <td id=\"T_ec1b0_row12_col1\" class=\"data row12 col1\" >111 VMs run time (windows11)</td>\n",
" <td id=\"T_ec1b0_row12_col2\" class=\"data row12 col2\" >ODF (virtio)</td>\n",
" <td id=\"T_ec1b0_row12_col3\" class=\"data row12 col3\" >8.17%</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_ec1b0_level0_row13\" class=\"row_heading level0 row13\" >13</th>\n",
" <td id=\"T_ec1b0_row13_col0\" class=\"data row13 col0\" >bootstorm</td>\n",
" <td id=\"T_ec1b0_row13_col1\" class=\"data row13 col1\" >111 VMs run time (windows_server_2019)</td>\n",
" <td id=\"T_ec1b0_row13_col2\" class=\"data row13 col2\" >ODF (virtio)</td>\n",
" <td id=\"T_ec1b0_row13_col3\" class=\"data row13 col3\" >25.13%</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_ec1b0_level0_row14\" class=\"row_heading level0 row14\" >14</th>\n",
" <td id=\"T_ec1b0_row14_col0\" class=\"data row14 col0\" >bootstorm</td>\n",
" <td id=\"T_ec1b0_row14_col1\" class=\"data row14 col1\" >111 VMs run time (windows_server_2022)</td>\n",
" <td id=\"T_ec1b0_row14_col2\" class=\"data row14 col2\" >ODF (virtio)</td>\n",
" <td id=\"T_ec1b0_row14_col3\" class=\"data row14 col3\" >30.53%</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<div><span style=\"color: black; font-weight: bold;\"> Network Speed: </span><span style=\"color: blue;\">25</span></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<div><span style=\"color: black; font-weight: bold;\">** For more details Grafana url:</span> <a href=\"http://cnv-intel-15.perf.eng.bos2.dc.redhat.com:3001\" style=\"color: blue;\" target=\"_blank\">http://cnv-intel-15.perf.eng.bos2.dc.redhat.com:3001</a></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# There is an option to choose specific workload comparison\n",
"report_df = summary_report_widget.analyze_all_workload(workloads = ['hammerdb', 'hammerdb_lso', 'uperf', 'vdbench', 'vdbench_scale', 'bootstorm', 'windows'])\n",
"summary_report_widget.display_df(report_df)"
]
"outputs": [],
"source": "# There is an option to choose specific workload comparison\nreport_df = summary_report_widget.analyze_all_workload(workloads = ['hammerdb', 'hammerdb_lso', 'hammerdb_scale', 'uperf', 'fio', 'fio_scale', 'vdbench', 'vdbench_scale', 'bootstorm', 'windows', 'sysbench'])\nsummary_report_widget.display_df(report_df)"
},
{
"cell_type": "markdown",
"id": "1f3c306d",
"source": "## Product versions",
"metadata": {}
},
{
"cell_type": "code",
"id": "21f28a06",
"source": "summary_report_widget.get_product_versions()",
"metadata": {},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
Expand Down
Loading