Quantitative comparisons refined

For some time already we’ve been considering ways to help our users with their quantitative comparison studies. Earlier, we had a study for comparing instruments running same assays, and another study for comparing test methods regardless of the instruments used for running them. Technically speaking, these are two separate use cases with different kind of needs, so having separate studies for them felt initially like a good idea.

On the way we’ve found out that this is often not the case. Users are interested in comparing instruments even if the main goal of the validation is to evaluate bias between different methods. And the same data that is used for comparing different test methods, is often also used for comparing the performance of the parallel instruments.

So it became evident that the quantitative comparison studies needed to be combined. But how to do that? We wanted to create a study that would enable a wide variety of different kind of comparisons. We didn’t want to make the existing workflows more complicated. We wanted to make it look simple. So we’ve been drawing optional designs for user interface and charts of different kind of workflows, and after quite a few discussions and changes of plans we’ve finally published the new, better, Measurement Procedure Comparison study. We still have a few ideas how to make it even more flexible to use, but already we feel it will give you better possibilities for comparing your quantitative data.

So what did change?

You are now able to compare methods and instruments on the same study. For example, you can first evaluate how your new method compares against your old one. Then, with the same data, you can check that the parallel instruments used to run the new method, are giving similar results.

Biggest study differences are in planning the comparison study

• From now on, you always need to define compared instruments. We tried to make that as easy as possible, so you can create multiple comparison pairs at once.
  
• Validated tests and their reference methods are defined on the same page as instruments. You can add the compared methods for multiple instrument pairs at once. And if your instruments support the same tests but some of the tests are run with only some of the instruments, you can select the exact instrument pairs you want to use for validating each test.
• If you are interested in comparing instruments running the same tests, you can access an easier way of adding tests by selecting the radio button with text Compare same methods between different instruments. After that you can just select your validated tests from a list and they are added to your study.

What will change in near future?

There are some tweaks to be done in setting goals and showing information in work index to help you in your work. We are also planning to add a possibility to give additional information to your instrument comparison pairs for cases where you want to study the performance of a single instrument in different kinds of environments.

Did you notice already?

Recent changes in Measurement Procedure Comparison also include a possibility to add goals for medical decision points. This way you can make sure that the performance of a method is good enough at medically relevant concentrations, instead of making your conclusions relying on average bias calculations over the whole measurement interval.

Detailed report contains your results visualized in a bias plot, a difference plot and a regression plot. Calculated results are presented in tables, highlighting whether your set goals have been met or not. There is also room for your result analysis.

Give it a try and tell us what you think!