So let’s answer the question first, absolutely.

Hype over the last few years has been the issue. How many times have we heard artificial intelligence, big data or blockchain mentioned. Some of these have had a few case studies, but none have caused a revolution,

But taking a look at the legal tech start-ups now, is almost like being back in the late 1990’s and wandering the floor of ” The Solicitors’ & Legal Office Exhibition” when a whole host of new technology firms were starting to implement on that beige thing with a screen that was in the corner of a lawyers office at the time.

This image is just a small number of the start-ups in one area that you’ll be familiar with in legal.

Perhaps not all these will work, but the difference now from the hype cycle above is that these are targeting business problems.

And that is where we are seeing the big change, innovation as a synonym for “new shiny technology” has started to morph into finding problems, applying techniques like lean six sigma or design thinking and working out ways to solve business challenges. Sometimes this is business change internally, sometimes it is people creating a start-up to deliver a solution to a problem.

With so many new options available to law firms and many options to solve the same problem, how do you chose? Or more importantly how can you try out these solutions quickly? Fail fast if you will.

Well this goes back to part 3. If we’ve got a platform that presents all our data and key services outwards and provides standard interfaces to access these, then plumbing in these new tools to test functionality can be much easier than in the past where you might have the spiderweb of interconnections between systems needed.

I want to go back to AI though to finish. A technology that I have may have been cynical of meeting it’s initial hype.

But a recent listen to this Forrester podcast last month shows that AI as a tool is now being put in some great solutions. One quote jumped out:

“29% of developers are using AI and machine learning technologies to build application that are infused with AI”

Infused with AI. This to me is AI moving beyond hype, it’s no longer buying “AI” as a product, realising that It’s not a “thing” but think of it more like a database, it’s becoming a core piece of a solution that solves a problem.

But here’s some takeaway to think about.

So we build software and what we are doing is giving the software more capabilities using machine learning, so rather than defining our input and getting a defined output, we build in machine learning and allow the software itself to learn and define the output. So the same input could give different outputs.

And so here we have a challenge, how do we test AI solutions to ensure we can be confident of the results. It’s not the end of the world if your Alexa get’s things wrong, but a decision in a legal case?

There are a number of cases of AI bias causing unwanted results when the solutions were released in the wild. Two well known ones are Micorsoft’s “racist bot” Tai and the US Compas system used to do risk assessments during sentencing.

Testing will need to change, the results are not deterministic and so a test engineer would need to run a test many times and make sure that statistically the conclusion is correct. As more and more legal technology infuses AI it makes sense to start looking at the test frameworks that are starting to be creating for AI testing, especially functional testing to limit the risk in bias.

How we test AI will certainly be a topic that will need to become part of the law firm IT dept arsenal in coming years as more legal tech get infused with machine learning. But as you ponder this new world of AI testing I will leave you with one quote from a testers blog

“What happens when both testing applications and systems under test use AI?”