Benchmarking constraint satisfaction problems with the OptaPlanner Benchmarker
Why Stack Overflow recommends it
Stack Overflow users link it when asking how to benchmark, tune, or diagnose the performance of OptaPlanner solvers.
Recommended by 1 people across 5 different months, 2014 to 2020.
What it is
A 24-minute walkthrough by OptaPlanner creator Geoffrey De Smet on using the OptaPlanner Benchmarker to compare solver configurations on constraint satisfaction and planning problems. It shows how to set up benchmark configs, use templates, and read the generated HTML reports.
What people said
Recommends Geoffrey De Smet · 2014-11 · Stack Overflow · on “OptaPlanner on Solaris is slower 5 times than on Linux os”
Interesting observation - I 'd like to understand what happens there too. Create a benchmark config [this video] for your use case (if you haven't already), run it on both machines and share both benchmark html reports here. I'll diagnose them. Typical causes can be: JDK version differences, JVM -server mode, undetermined termination config (unimprovedTimeLimit), ... But none of that explains a 5x difference.
Recommends Geoffrey De Smet · 2015-04 · Stack Overflow · on “Looking for ways to make Drools / Optaplanner faster”
See docs "Score performance tricks" [LINK] . Especially keep an eye out on your "average score calculation count per second". It should be above 1000 , preferably above 10000 . Use the OptaPlanner benchmarker [this video] . It will be your best friend in this endevour, thrust me :) Note: Ask yourself: What do you mean with "faster"? Do you want better results in the same time? Do you want the same results faster? And/Or do you want to scale out more? There might be trade-off choices to make there - use the OptaPlanner Benchmarker to see the pure gains on all 3 fronts, but also to visualize trade-offs between those 3 fronts.
Recommends Geoffrey De Smet · 2016-03 · Stack Overflow · on “optaplanner performance report”
Yes, configure the OptaPlanner Benchmarker [this video] for your implementation and look at the generated html report.
Recommends Geoffrey De Smet · 2018-08 · Stack Overflow · on “First Fit Decreasing Algorithm on CVRP in Optaplanner”
(braindump) Unlike with non-VRP problems, the choice of "difficulty comparison" to determine the "Decreasing Difficulty" of "First Fit Decreasing", isn't always clear. I've done experiments with several forms - such as based on distance to depot, angle to depot, latitude, etc. You can find all those forms of difficulty comperators in the examples, usually TSP. One common pitfall is to tweak the comperator before enabling Nearby Selection: tweak Nearby Selection first. If you're dealing with large datasets, an "angle to depo" comparator will behave much better, just because Nearby Selection and/or Paritioned Search aren't active yet. In any case, as always: you need to be using optaplanner-be
Recommends Geoffrey De Smet · 2020-10 · Stack Overflow · on “Run optaplanner at AWS Lambda is good enough?”
OptaPlanner Benchmark [this video] to the rescue! Seriously, run OptaPlanner Benchmark (see OptaPlanner docs) for 30 minutes and look at the "BEST_SCORE" graph to know what 15 minutes or more or less will get you on your machine. Then rinse and repeat on a cloud like OpenShift or AWS. As for AWS Lamdba (serverless): OptaPlanner Quarkus [LINK] is your friend.
Cites rinde · 2016-01 · Stack Overflow · on “Benchmark a real-time planning algorithm using Optaplanner”
I'm trying to benchmark a real-time planning algorithm but can't seem to find how to do it, is this supported in Optaplanner? I've successfully run a benchmark using an offline version of my problem. I've implemented SolutionFileIO that reads my problem instances and converts them to a solution. I've read the docs and saw the video [this video] related to benchmarking but couldn't find what I'm looking for. Alternatively, I can run the real-time algorithms using my own framework, but that would require me to manually define all Optaplanner heuristics that I want to run (which is quite cumbersome when using a matrix setup). Is there a way to instantiate the solvers (in Java) based on the benchmark
Cites F. Sonntag · 2020-04 · Stack Overflow · on “Modularize optaplanner benchmark config”
The question is: Is there a way to write a solver config file that it contains insertions of configuration parts from other xml-files within Optaplanner? Explanation: I try to perform some rather complex benchmark runs with partially very different solver confugrations like different local search stages filter classes drool rule sets etc. Because of that the template way shown in this benchmark constraints salvation video [this video] is not sufficient. Since XML has the DTD Entity insertion, I thought about modularize the complex configuration patterns and reuse them in different sets, but unfortunately Optaplanner seems not to support this feature when it is reading a solver configuration file.