Estimated reading time: 4 minutes
When designing a complex procurement, lots of time and money can be spent on the design phase of the tender documents: consultants hired to write various aspects of the requirements, stakeholder meetings to review the evaluation framework and make sure the weights are just right, and so on.
In this design phase of the evaluation framework, owners will often think about the characteristics they want to see in an organization that wins the contract and will consider these in developing their criteria. Experienced procurement professionals will understand that it is unrealistic to expect any bid to score 100% on each and every evaluation factor you have developed. More likely, some will score well in certain areas and more moderately in others, all with an impact on the bid price. The owner will have one or more score/price profile in mind for desired optimal bids.
However, no matter how carefully you craft your evaluation criteria and your weights, without proper analysis it will be hard to determine whether your most optimal bid profile will win your competition. While there is no way to guarantee the outcome of a solicitation, modelling your evaluation structure will help you understand your anticipated results more fully.
We call this process “war gaming”.
War gaming is a decision-making technique that provides structured but intellectually liberating safe-to-fail environments to help explore what works (winning/succeeding) and what doesn’t work (losing/failing), typically at relatively low cost.
A war game is a process of adversarial challenge and creativity, delivered in a structured format and is usually umpired or adjudicated.
It is meant to deal with the “what ifs” of your evaluation schema. A good war gaming exercise looks at your model and challenges it: What happens if none of the bidders can really demonstrate what your asking for in that mandatory criterion? Are you willing to let them fail on that one factor? What if they all present exactly what you ask for in rated criterion 3.2? Is it really a differentiator? Is it useful?
As the procurement professional leading the solicitation, it is your responsibility to stick-handle this process. It does not mean that you need to be the one asking all the questions. Indeed, it is often useful to get other stakeholders to ask questions and challenge the framework, as well as suggesting other options, with the procurement professional acting as the ‘referee’ in the process.
The war gaming exercise will help the decision makers understand that there are ranges of potential outcomes.
To start the exercise, you need to develop scenarios: The types of responses you expect to get from the competition. Scenarios should always be based on plausible variations, wherever possible backed up by detailed market knowledge or previous experience. They should include a price and technical scoring in as much detail as practical. For example, you may only include scores for overall sections, or for individual factors, depending on the structure of your framework and the kind of modelling and scenario you want.
Once you have a series of potential scenarios, you can chart them against your framework. Using our AWARD® System, such a chart might look like this:
This chart above is based on these characteristics:
- A 60/40 Technical/Price scoring split;
- A Best Value calculation using the Proportional Scoring method; i.e. Financial Scores calculated as the Lowest Bid Price ÷ The Bidder’s Price x the Weight (40).
The chart shows that the Premium Solution (with the highest technical score, but also the highest bid price) wins the competition with 75.98% overall. The line on the chart shows all the price/quality score combinations that would result in 75.98 points. That line is important because it shows that the Premium Solution would still score the same should it have been priced at $100,000; that is $10,000 more than its actual bid price.
Once you have this data, you can adjust your weights and your Best Value calculation (the technical/price split, or the methodology/calculation on the whole) to see what can be done to lead to a more optimal bid winning the competition. You should also be able to remove various bids from the model to see how that might affect the outcome. Alternatively, you may also adjust your scenarios if, on review of the results, you find that certain aspects may not be that realistic after all.
War gaming helps protect against sub-optimal outcomes: “Did we mean that to happen?” or “Did we really want to discount/reject that bidder/solution from the competition?”
Worst case sub-optimal outcomes may include all bidders being non-compliant at the end of tender evaluation, or the preferred bidder being clearly identifiable at the end of tender evaluation, but key senior stakeholder(s) are unhappy, viewing the outcome as intolerable (for example, it is over budget).
Our AWARD® Software, together with our expert services, can help with this modelling for procurements that might be beyond the reach of a spreadsheet.
However you complete it, proper modelling is key to ensuring a quality outcome to your complex procurement. Get in touch below to find out more on how we can support your future sourcing projects with solutions to deliver the best possible outcome.
