Users who are new to Validation Manager often ask us for a recommendation on how to organize their upcoming big verification in Validation Manager.
This is a very good question to consider. How to make it easy to gather your data, review your verification results, and create your verification report?
Most laboratories are used to organizing their verification so that each analyte will have their own report files. The usual reason being that it’s practically the only possible way to manage the data.
In Validation Manager, on the other hand, you can make deliberate choices based on your needs.
But, if you’ve never got to do that, it may be difficult to know how to do it. After all, you would need to recognize the patterns that currently add unnecessary complexity to your work.
Being used to the restrictions related to the old data management system probably affects the way you think about verifications. That’s why, when talking about verifications, people often think of a verification of one analyte on one instrument.
The best success stories come from organizations utilizing Validation Manager to take their verifications to the next level, rather than to maintain their old workflows. Our customers are able to cut more than 95% of data-management time and reap more benefits from their verifications than ever.
How to define a verification project?
Validation Manager gives you detailed reports for single analytes run on single instruments, as you probably expected it to do. But that’s not the level of reporting you should be thinking about when creating your project in Validation Manager.
A better way to approach a verification project would be to think about the folder that contains all your data related to, e.g., verifications of a new instrument system. Or you could think about a binder into which you have gathered all the printed verification reports related to the instrument system.
Generally, we advise thinking of one project as something that gives answers to high level questions. A good question could be “can we take the new line of chemistry instruments into use.” Your project aims to answer this question, and therefore it should contain pretty much everything that’s relevant for that decision.
How many studies to have within a project?
A study is for one type of statistical analysis to describe a performance characteristic (e.g., precision). One project can have many of them.
Within a verification project, each study can handle multiple instruments and analytes. So, when considering the scope of a study, you shouldn’t divide your work into single analytes. Instead, you should include everything in one study that is supposed to answer a question like:
How does the behavior of the new method differ from the old method when measuring patient samples of different concentrations?
How much variance is there when doing multiple replicate measurements over multiple days?
So basically, most verification projects can be taken through with one project containing only two studies:
One ANOVA study to measure precision of all new methods.
One comparison study to compare all the new methods to old ones.
In some cases, adding a third study for parallel instrument comparisons is a good idea. And, sometimes, you may want to have a precision study for pooled patient samples and quantitative accuracy study for controls. But in the end, all you need is a handful of studies, even if you are verifying a large set of tests on multiple instruments.
So, if you want a rule of thumb on how to organize your verification project, it’s here: Have only one study for estimating each type of performance characteristic. Include all the instruments and analytes within each study.
However, experience has shown this is not the whole truth. How to best organize the verification is pretty much up to what you really need to get out of the report. Sometimes, it may be meaningful to divide the analytes into groups, for example having one comparison study and precision study for electrolytes, and a separate comparison study and precision study for metabolites. That way, each group will have their own chapters on your report. And the study names are used as chapter headings on the report.
The most important things to remember
When you are new to Validation Manager, it may feel troublesome to make these kinds of choices. The most important things to remember are:
One report can only include content related to one project. So, to make sure that it’s easy for you to fill your reporting needs, think big when considering the scope of a project in Validation Manager.
Avoid studies that only contain one analyte, especially in big verifications, where there are lots of analytes that need to be verified.
It’s not always easy to find the best solution before you have tried it. But there are many ways to do things well enough.
If you really need to change something in the structure of your ongoing verifications, it’s easy to create new studies and copy content from one study to another within the same project.
They say that practice makes perfect, and this certainly applies to using Validation Manager.