Skip to content
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 23 additions & 24 deletions docs/src/2-benchmark.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ using SolverBenchmark

Let us select equality-constrained problems from CUTEst with a maximum of 100 variables or constraints. After removing problems with fixed variables, examples with a constant objective, and infeasibility residuals.

``` @example ex1
```julia
Comment thread
arnavk23 marked this conversation as resolved.
Outdated
_pnames = CUTEst.select(
max_var = 100,
min_con = 1,
Expand All @@ -43,7 +43,7 @@ using DCISolver, NLPModelsIpopt

To make stopping conditions comparable, we set `Ipopt`'s parameters `dual_inf_tol=Inf`, `constr_viol_tol=Inf` and `compl_inf_tol=Inf` to disable additional stopping conditions related to those tolerances, `acceptable_iter=0` to disable the search for an acceptable point.

``` @example ex1
```julia
#Same time limit for all the solvers
max_time = 1200. #20 minutes
tol = 1e-5
Expand Down Expand Up @@ -78,20 +78,17 @@ stats = bmark_solvers(solvers, cutest_problems)

The function `bmark_solvers` return a `Dict` of `DataFrames` with detailed information on the execution. This output can be saved in a data file.

``` @example ex1
```julia
using JLD2
@save "ipopt_dcildl_$(string(length(pnames))).jld2" stats
```

The result of the benchmark can be explored via tables,

``` @example ex1
```julia
pretty_stats(stats[:dcildl])
```

or it can also be used to make performance profiles.

``` @example ex1
```julia
using Plots
gr()

Expand All @@ -101,38 +98,40 @@ legend = Dict(
:neval_grad => "number of ∇f evals",
:neval_jac => "number of ∇c evals",
:neval_jprod => "number of ∇c*v evals",
:neval_jtprod => "number of ∇cᵀ*v evals",
:neval_jtprod => "number of ∇cᵗ*v evals",
:neval_hess => "number of ∇²f evals",
:elapsed_time => "elapsed time"
)
perf_title(col) = "Performance profile on CUTEst w.r.t. $(string(legend[col]))"

styles = [:solid,:dash,:dot,:dashdot] #[:auto, :solid, :dash, :dot, :dashdot, :dashdotdot]
styles = [:solid,:dash,:dot,:dashdot]

function print_pp_column(col::Symbol, stats)

ϵ = minimum(minimum(filter(x -> x > 0, df[!, col])) for df in values(stats))
τ5 = minimum(minimum(filter(x -> x > 0, df[!, col])) for df in values(stats))
first_order(df) = df.status .== :first_order
unbounded(df) = df.status .== :unbounded
solved(df) = first_order(df) .| unbounded(df)
cost(df) = (max.(df[!, col], ϵ) + .!solved(df) .* Inf)
cost(df) = (max.(df[!, col], τ5) + .!solved(df) .* Inf)

p = performance_profile(
stats,
cost,
title=perf_title(col),
legend=:bottomright,
linestyles=styles
[cost(df) for df in values(stats)],
styles,
legend,
perf_title(col),
col
)
display(p)
end

print_pp_column(:elapsed_time, stats) # with respect to time
```

``` @example ex1
print_pp_column(:neval_jac, stats) # with respect to number of jacobian evaluations
print_pp_column(:neval_obj, stats)
print_pp_column(:neval_cons, stats)
print_pp_column(:neval_grad, stats)
print_pp_column(:neval_jac, stats)
print_pp_column(:neval_jprod, stats)
print_pp_column(:neval_jtprod, stats)
print_pp_column(:neval_hess, stats)
print_pp_column(:elapsed_time, stats)
```

## CUTEst benchmark with Knitro

In this second part, we present the result of a similar benchmark with a maximum of 10000 variables and constraints (82 problems), and including the solver [`KNITRO`](https://link.springer.com/chapter/10.1007/0-387-30065-1_4) (Byrd, R. H., Nocedal, J., & Waltz, R. A. (2006). K nitro: An integrated package for nonlinear optimization. In Large-scale nonlinear optimization (pp. 35-59). Springer, Boston, MA.) via [`NLPModelsKnitro.jl`](https://github.com/JuliaSmoothOptimizers/NLPModelsKnitro.jl). The script is included in [/benchmark/script10000_knitro.jl)](https://github.com/JuliaSmoothOptimizers/DCISolver.jl/blob/main/benchmark/script10000_knitro.jl). We report here a performance profile with respect
Expand Down
2 changes: 1 addition & 1 deletion src/DCISolver.jl
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ function cons_norhs!(nlp, x, cx)
return cx
end

export dci
export dci, feasibility_step
Comment thread
arnavk23 marked this conversation as resolved.
Outdated

include("param_struct.jl")
include("workspace.jl")
Expand Down
5 changes: 3 additions & 2 deletions test/Project.toml
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
[deps]
ADNLPModels = "54578032-b7ea-4c30-94aa-7cbd1cce6c9a"
BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
DCISolver = "bee2e536-65f6-11e9-3844-e5bb4c9c55c9"
Comment thread
arnavk23 marked this conversation as resolved.
Outdated
HSL = "34c5aeac-e683-54a6-a0e9-6e0fdc586c50"
Krylov = "ba0b0d4f-ebba-5204-a429-3ac8c609bfb7"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
Logging = "56ddb016-857b-54e1-b83d-db4d58db5568"
NLPModels = "a4795742-8479-5a88-8948-cc11e1c8c1a6"
NLPModelsTest = "7998695d-6960-4d3a-85c4-e1bceb8cd856"
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
SparseArrays = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
SolverCore = "ff4d7338-4cf1-434d-91df-b86cb86fb843"
SolverTest = "4343dc35-3317-4c6e-8877-f0cc8502c90e"
SparseArrays = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[compat]
Expand All @@ -23,7 +24,7 @@ Logging = "1.10"
NLPModels = "0.21"
NLPModelsTest = "0.10"
Random = "1.10"
SparseArrays = "1.10"
SolverCore = "0.3"
SolverTest = "0.3"
SparseArrays = "1.10"
Test = "1.10"