Uploaded image for project: 'OptaPlanner'
  1. OptaPlanner
  2. PLANNER-627

Benchmarker: quick and easy way to create a benchmarker from an existing solver configuration and problem

    XMLWordPrintable

Details

    • 2017 Week 32-33
    • NEW
    • NEW

    Description

      Setting up a benchmarker currently takes an hour or so. Many users are too time pressured for that (they're too busy tracking down their performance issue... sigh). So we should make it quicker for them to set up benchmarker.

      Something like this should work:

      SolverFactory sf = SolverFactory.createFromXmlResource(".../nqueensSolverConfig.xml");
      CloudBalance problem = ...;
      PlannerBenchmarkFactory bf = PlannerBenchmarkFactory.createFromSolverFactory(sf, problem);
      bf.buildPlannerBenchmark();
      

      If anyone has a perf problem and then still doesn't use the Benchmarker, well then ...

      Details:

      • default to 30 secs warmup
      • default to "local/benchmarkReport" for benchmark report output. (One can argue that this should default to "target/benchmarkReport" but as that's cleaned during "mvn clean", I wouldn't mind promoting the local directory in .gitignore approach (which is great).)
      • PLANNER-724 Definitely including the BEST_SCORE problem statistic. We can worry later if we want to add the others too (due to perf impact).
      • Yes, this benchmark report has only 1 solver configuration and 1 dataset, but it does show the best score graph, the perf, the warnings, etc

      Attachments

        Issue Links

          Activity

            People

              gdesmet@redhat.com Geoffrey De Smet (Inactive)
              gdesmet@redhat.com Geoffrey De Smet (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: