Moneyball for law: picking the best fantasy league legal team

Improving the likelihood of success

Our guest for this blog post, Mark Verhagen, is a PhD candidate at Nuffield College at Oxford (https://www.sociology.ox.ac.uk/people/mark-verhagen). In addition to his PhD, Mark also operates a machine learning consultancy focusing on optimizing people and talent.

We met in late 2020 on an Oxford University Law Facility Zoom presentations, and struck up a conversation about combining the use of computer vision and natural language processing to analyze legal documents. We decided we had to find a project to tackle together. While we waited for the right project, we decided to toy with the questions – how do we optimize the likely performance of legal teams, and therefore how do we improve the likelihood of success of legal work?

Much of legal technology seems to be focused on solutions that seem to “robotize the lawyer”, but Mark and I want to use this blog post to explore what could happen if we apply technology and data science to the layers above the expert services, and “robotized” those layers?

In other words, what if we optimized:

  • team selection
  • information preparation

What if we “robotize” everything up to the point when the lawyer goes into a room with a client?

Picking the fantasy league legal team

There is no single “best legal team” for all matters. Each matter will require different expertise, personalities and tendencies.

Mark notes during our conversations that there is an emerging practice in the management consulting world where data is used to improve hiring and assembling a team in order to ensure “case success” (assuming we know what “case success” means).

Unlike management consulting, when lawyers are picking their team for transactions or litigation, partners at law firms typically pick their team based on:

  • who they know in their firm
  • who they have worked with in the past
  • who they know might have relevant industry experience

In other words, the process is based almost wholly on intuition or “gut feelings” of senior lawyers. While it is possible that some firms have adopted a data-driven approach to pick the best team for each piece of work, the majority of lawyers seem to select their team using subjective metrics, or, at least, based on imperfect information.

… I wonder, can we explore the question of “what if the whole management team of your law firm is replaced by robots?”

This is an optimization problem – how do we pick the best team for any piece of legal work?

If the management team is replaced by robots, what decisions would be made a robot management team? How would we design a “training problem” to ensure the robot management team makes the best decisions?

What stops us from immediate experimentation?

If we had data about lawyers, such as their skills, their strengths and weaknesses, and their experiences, then perhaps we can immediately conduct experiments… but unlike sports, vital statistics and information about lawyers are not widely available. Even where data is privately available, the data tend to be either self-reported or trapped in the minds of those people who are within two degrees of separation from a lawyer.

How do we optimize everything up to the point the lawyer picks up the work?

Since we do not have data, Mark and I discussed how we can theoretically address this optimization problem, assuming that we could access relevant data, such as:

  • education and training history for each lawyer
  • documents opened by each lawyer
  • hours billed by each lawyer
  • narration of work done by each lawyer
  • anonymized annual performance review scores for each lawyer

This is how we would pull together our heist crew

Watch Mark and I discuss how we might solve this optimization problem in our unscripted conversation here:

Leave a Comment

Your email address will not be published. Required fields are marked *