Types of Broken

Engines, like any system, may have bugs. This is normal and natural. But engines, especially heuristics that approximate optimal, have a difficult time around subjective complaints. The engine produces a 99.9% good solution. Yes, it’s missing the 0.1%, so it’s not perfect. Is this a reason to disregard the result, stop using the engine, or cancel a contract? Really?

For engines, we have a scale for broken.

• Broken

  • No Solution

    • Crash

    • Infinite Run Time

  • Infeasible

    • Violates real-world constraints

      • Cannot drive a truck to two different locations at the same time

• Undesirable

  • Obviously Suboptimal

    • Users of moderate experience can immediately see issues with no research

  • Suboptimal

    • Experienced users can export results, conduct deep analysis, and derive problems with solutions

  • Slow

    • Results are acceptable, but it takes longer than desired to get results

• Desirable

  • Near perfect in a reasonable time

Within this scale, all users (and developers, quite frankly) want desirable results and only desirable results. But the reality of most optimizations is that this is not achievable in all scenarios. A Mixed Integer Program can get bottled up in the Branch and Bound algorithm. Dykstra’s algorithm can fail to find the optimal route.

The results from Slow are still good. The results from Suboptimal are not ridiculous. These are real and useful systems. If the user is experienced and knows how to work around the performance issue or knows what to look for in the suboptimal solution, the user can gain significant benefit from the system.

So while NEO strives for 100% Desirable results from all engines all the time, the truth is that slow and suboptimal are the reality for a wide range of scenarios.