LITIGATION SUPERFORECASTING SURVEY RESULTS PART 2: SOLICITORS MORE CONFIDENT THAN BARRISTERS

Click here to download article as a PDF

Imagine you’ve been to see your solicitor about a legal dispute brewing between your company and one of its major suppliers.  Your solicitor has provided her “preliminary view” on how the key contract terms should be construed and concludes that you have “strong prospects” of obtaining “significant damages”.

Litigation is a big financial decision for your company and so you decide to get a second opinion from a silk at one of the top commercial sets.  The QC provides her opinion that your claim has “strong prospects”.

Naturally, you’re relieved to find that your barrister agrees with your solicitor.

However, were the QC and the solicitor using the phrase “strong prospects” in the same way?

A recent survey by Balance Legal Capital indicates they may not be.

In fact, solicitors tend to attach higher confidence to phrases like “strong prospects” than barristers do.

LITIGATION SUPERFORECASTING SURVEY

We issued a survey to solicitors, barristers, corporates and other users of legal advice around the world, asking them to put a percentage number against the following ten probabilistic phrases commonly used by lawyers:

1. “near certainty”
2. “significant likelihood”
3. “serious possibility”
4. “strong prospects”
5. “good prospects”
6. “reasonable prospects”
7. “reasonably arguable”
8. “more likely than not”
9. “fair chance”
10. “on balance”

The phrases were displayed in a random order each time respondents visited the online survey to reduce any biases that might be induced by the order in which the phrases were presented.

Balance recruited the assistance of cognitive neuroscientist Dr. Stephen Fleming of University College London, to analyse the data from the 250 responses as at 1 August 2018.

Take the survey here. 1

SOLICITORS V BARRISTERS

A key result from the Litigation Superforecasting Survey was that there is substantial variability in the way that probabilistic phrases are interpreted by all respondents.

When we zoned in on just solicitors and barristers, we found that solicitors gave all 10 of the verbal labels higher scores than the barristers, leading to a statistically significant difference between the groups (see Chart 1 and Table 1).

While solicitors generally attached higher scores to all phrases (on average, 6 percentage points higher), the phrase showing the greatest mismatch in confidence is “serious possibility”, with barristers interpreting this as a 41% probability, compared to the solicitors’ 54%. In contrast, the best-matched phrases were “on balance” and “reasonable prospects”.

“solicitors gave all 10 of the verbal labels higher scores than the barristers”

CHART 1: SOLICITOR V BARRISTER - “RAINCLOUD” PLOTS SHOWING SCORES BY PHRASE

TABLE 1: SOLICITOR V BARRISTER - MEAN SCORES FOR EACH PHRASE AND VARIANCE

‍

“Your solicitor and barrister may have substantially divergent views on the prospects of success of your claim, currently obscured by these common legal advice phrases.”

Robert Rothkopf, Managing Partner of Balance Legal Capital commented:

“In our experience, barristers are generally more reluctant than solicitors to put numbers on their prospects advice. Often a resistant barrister or solicitor will argue that applying percentage prospects is an artificial exercise because a 70% case can still lose so what’s the benefit of the number?  Of course a 70% case has a 30% chance of losing, but the benefit is one of precise communication of the adviser’s confidence.  With the number, we can better understand what the barrister or solicitor really thinks about our case than we can from a phrase like “strong prospects.”

Your solicitor and barrister may have substantially divergent views on the prospects of success of your claim, currently obscured by these common legal advice phrases.  Without asking them to “put a number on it” you will never know how they really feel about your case.

Notes:

1. Balance’s Litigation Superforecasting Survey was inspired by the research of Philip Tetlock (the author of Superforecasting) which, based on studies of behavioural economists and forecasting tournament data, endorses a number of techniques found to improve forecasting accuracy. One of these is the use of numbers rather than phrases when making probabilistic assessments.
2. Raincloud plots are data visualizations which show raw data, probability density, and key summary statistics such as median, mean, and relevant confidence intervals while preserving the ‘inference at a glance’ nature of barplots. For more, see: Allen M, Poggiali D, Whitaker K, Marshall TR, Kievit R. (2018) Raincloud plots: a multi-platform tool for robust data visualization. PeerJ Preprints 6:e27137v1.
3. SD = Standard Deviation

‍