The path to 99%: what we measure, what we publish, and how far we have to go

Every serial-capture tool will tell you it is accurate. Very few say accurate at what, measured how, on which devices. Here is ours. We will re-publish this page each time we re-measure.

What we are actually measuring

A drive label carries a dozen strings that look like a serial number: part numbers, model numbers, firmware codes, factory batch codes, and often an owner’s asset sticker with its own “S/N”. Reading the label is one job. Knowing which string is the serial is the harder one.

We tested on hundreds of real storage devices, each checked by a person, in our own lab. Every label had a visible serial. We tested the model five times, each time on devices it had never seen.

Where we are today

Our serial identification model picks the right serial on 97.7% of the drive labels it can read. We count an identification as correct when it is within one character of the true serial. 87.4% were an exact match.

That number grades the model, the part that reasons and decides. It does not include the labels where the print could not be read at all. The test was conducted in our own lab rather than with a customer.

Why one character difference? Simply, AI and hardware just aren’t there yet economically. A ‘0’ is frequently mistaken for an ‘O’, and an ‘l’ for a ‘1’. These errors are easy to anticipate, identify and correct, which is why we have allowed this tolerance. This does not mean we won’t keep reducing it, but the first job is an algorithm that identifies the right serial every time.

With a supplied asset list, matching is exact. When a customer gives us the serials on the job, we only accept an exact match.

The model behind this figure is the one running in our apps and tablet today.

What got us here

We did not get here by trying cleverer AI. What moved the number was building in what an experienced operator already knows about drive labels.

Most of the errors that remain are in reading faint or small print, not in the choice. That tells us where the next big gains are: less blurry photos (sorry!), an improved camera, better lighting and a better training set (again, sorry!).

What 99% means, and how we will prove it

Our goal is that 99% of serials confirmed without review are correct.

We will publish 99% when it has been measured in our lab environment, rather than on customer scans, with at least 2,000 different devices. And we will keep checking a random sample of confirmed scans by hand, so the error rate is always measured and never assumed.

For context on what proof costs: to show an error rate of 1 in 100 with confidence, you need about 300 checked confirmations without a single error. For 1 in 10,000, which is 99.99%, you need about 30,000. That is why this page is a path rather than a press release.

What we don’t publish yet

• A speed figure. Our sub-one-second statement is an engineering constraint rather than a goal. To be truly useful on the shop floor, at the loading dock and at the cutting head, it must be quick, not just accurate. A slow but accurate model does not make a measurable improvement on the alternatives available today. Fixing speed as a constraint rather than an aim means we keep innovating towards our goals within it.
• Results from customer sites. Our figures are lab measurements. Our customers run this in different conditions on different hardware, so recording those results would leave too many variables uncontrolled. We will run benchmark tests with real operators in a consistent, realistic space and share the results when they come in.
• Anything we cannot show the working for. Just like Colonel Sanders, our secret sauce is ours. Sorry.

What happens next

1. Benchmark test in a live environment.
2. Measure everything again on the devices themselves, and on customer sites.
3. Let each customer set how sure the system must be before it confirms without review.
4. Re-publish this page with the new numbers, including any that go down.

We are not at our goal yet, but we are closer than we were yesterday.

We will keep working with our awesome customers to hold up our end of the deal, by improving your ability to do your job.