The Precision Vs Accuracy challenge in 3 examples
Some may not understand that an instrument that is very precise can be extraordinarily inaccurate. The key to accuracy lies in properly applying an instrument or test, regardless of its precision, in an informed context. This topic is a continual challenge to all who advance science, and every now and then questions of precision, accuracy, sensitivity, uncertainty, specificity, and bias can bring a project to its knees. So it always merits discussion. When I explore other online definitions of the topic of precision vs accuracy, I always seem to find the same old example of a bullseye target. That doesn’t begin to capture the variety of ways that precision runs head on against accuracy, so I’ve posted a few examples here, grounded in my own professional experiences, that seem important.
The featured image is almost a great example, given that the PACIFICA pH values (cyan) were reported at 4+ decimal places, while NOAA’s WOD pH values (green) were only reported at 2 decimal places. It appears that the less – precise WOD values more accurately capture our selected ocean pH at depth than the more precise data source. The PACIFICA pH data is literally all over the map.
This is almost a great example, because the high precision numbers from the PACIFICA resource are partly based on a simple calculation. The lack of accuracy remains, regardless of calculation or instrumentation it seems. The PACIFICA profile of pH is based on the sampling of dissolved inorganic carbon (DIC). That DIC ultimately comes from the geostrophically circulating organic masses of carbon associated with phytoplanktons, bacteria and viruses whose carcasses rain across the planet and settle to the ocean floor. The PACIFICA pH calculation doesn’t appear to factor this settlement origin into the equation and that may be why their numbers are all over the map.
The cyan data may at least give a helpful understanding of the phytoplankton plus virus carcass rainfall rates at various times and locations as some references appear to describe. I think other references  also describe that the massive contributions of tiny photosynthesizing marine flora are the reason for the impressive overlay of a solar-driven pH effect upon the natural ocean geostrophic chemical gradients. That pelagic effect does not appear to receive its due weight in any estimation of ocean health. The glass electrode pH (geph) measurements in green on the other hand are all based on actual direct electrochemical measurements of the pH at the given depths into the ocean in that same SE Pacific quadrant.
This battle between precision and accuracy continues to play out, because it has consequential impacts with regard to assertions of climate change. It may seem strange that something so seemingly non controversial as precision versus accuracy, turns controversial topics on their heads. This seems to apply to today’s Covid-19 surveys, all of which are based on highly precise PCR test results. A more accurate surveillance of respiratory virus spread is commonly accepted however to be based on antigen tests. They are less precise but more relevant.
I have found some helpful resources that compare both precision versus accuracy and also sensitivity versus specificity at this link. I think they are quite similar comparisons, depending upon the context. That resource is produced by the Association for Clinical Biochemistry and Laboratory Medicine (UK) and includes a statement that
“Specificity is the ability of a test to correctly exclude individuals who do not have a given disease or disorder.“
By the way, flu tests for antigens and flu tests for specific viral genetic strains follow these same recommendations. If you wanted to “know” if a subject (seemingly healthy or not) had or could transmit a certain named respiratory disease, you might want to swab their nose for a quick set of tests that are specific. As noted, antigenic tests can be impressively specific. As a first response, an array of antigenic tests can tell whether you are infected with Influenza A or Influenza B (two broad categories), or with one of the five virus categories above. They can also be rapid and cheap.
If you wanted to “know” if a subject (seemingly healthy or not) had or could transmit a certain named respiratory disease, then you might want to avoid a PCR test, UNLESS, you have already narrowed the categories of disease down through prior Antigenic testing. A PCR test may through its amplification (concentration) process find a bit of RNA wiring. But it cannot verify if that has anything to do with the subject’s health or their capacity to transmit that targeted disease. In other words, the PCR surveillance approach alone appears to fall short in specificity goals that can be easily met through antigen tests.
When the PCR cart comes before the Antigen horse, all kinds of misinterpretation appear to be likely. If I’m not mistaken, a PCR test will lead to both more false positives and false negatives*, unless it is preceded by the Antigen specific tests that help to narrow options down in the first place. The current exclusive focus on PCR tests for one strain is somewhat like an election where only one candidate is permitted to be voted for. Yet universally we are instructed that PCR tests are more sensitive, without the above context. PCR tests number in the millions both positive and negative for Covid-19. Antigenic tests for Covid-19 are reported nowhere.
This is only a blog and please note that I am only thinking out loud. I don’t have a single grain of health science training. I’ve addressed the information based only upon what I’ve read, some intersecting work, and upon my experience in ferreting out questions of precision, sensitivity, and accuracy in the earth sciences, such as ocean hydrology. But if I have been wrong, that should be easy for any actual expert to debunk and I always welcome such criticism and correction.
Ocean pH and virus tests cover the first two of three comparisons. The third is the outstanding collection of hydrologic errors associated with climate change models. In the example below I’ve charted the accuracy of my own hydrologic projections for Rocky Mountain streamflows (green and white) against those of the West Wide Climate Assessment. Remarkably, the WWCA product prominently features “bias correction” in their lead title, and yet their errors are not only heavily positively biased, they are extraordinarily inaccurate.
Accuracy means getting something generally right, now matter how that assessment is arrived at. To produce the inaccurate spray of red dots in the chart above, the small army of researchers supporting the West-Wide Climate Assessment deployed one of the most sophisticated hydrologic computer software codes and systems available today, and they performed thousands of expensive, time-consuming, runs on powerful ultra-fast computers, in the hopes of avoiding bias and bracketing uncertainties. For whatever reasons, they failed at both. In contrast, I used one laptop, along with a simple statistical approach for my more accurate and less biased projections, signified by the green and white dots.
More about measurements, accuracy and precision
The featured chart and the chart above are found at other posts at this site and also through a CSIM parody project. The sense of huge numbers of inaccurate results smothering smaller and more accurate products, inspired me to develop a satirical narrative which the CSIM page links to at its end.
Returning to real concerns, the interesting green “thermocline” profiles above were derived from pH measurements associated with the cruises mapped below.
The NOAA WOD captures the complete instrumental records for measurements at customary depths. This ensemble typically includes temperature, salinity, and dissolved O2 and more. I once used the WOD interface to query for O2 across the same map subset but for a later decade.
And the resulting profile for the shallower pelagic O2 measurements:
While the map of O2 sampling sites seems very distracting to enjoy, that was for a later decade. Returning to one example sample site location for 1994 again, if I recall correctly, I had plotted comparisons to ocean pH as monitored by gephs. In reviving this below, everything seems to be in harmony, speaking purely scientifically. 😀 At least, the thermoclinic inflections with depth are most consistent for pH, temperature, dissolved O2 and more.
I like very much how the dissolved O2 and pH form a symmetric fork to the phosphates and nitrates. This appears to complement explanations of “Low nitrate:phosphate ratios in the Global Ocean” which happened to look at the same WOD class database for the same year of 1994.  In that coincidentally timed study, the authors Tyrrell and Law noted the persistent low ratio, which we can see here because the two curves of nitrate and phosphate practically plot on top of each other.
I think this post complements due to the pH and O2 curves. Each pattern shown above aligns with Tyrrell’s and Law’s indication that “The main trend in the data (between nitrate and phosphate) is due to phytoplankton assimilation of the nutrients in surface waters followed by aerobic decomposition of the sinking organic matter, which recycles nitrogen and phosphorus back into dissolved form in deeper water according to the reaction..”.
This was no detour. The fact that glass electrode pH meter readings, arrived at from a random location and date selection, fully align with the dissolved oxygen, and nutrient molecules, may be as much proof as one needs to verify that the pH readings are accurate. But in any case, proof is abundant. Here is the same primary chart of this post, cropped to fit the new background image, which itself was cropped from Figure 1 of Nelson et al (1998) . That’s a random grab from a pH paleoreconstruction paper. Their reconstructed pH and “mine” are compellingly aligned, would you agree? That cannot be said for the otherwise highly precise cyan colored PACFICA calculated pH dots.
These types of vital graphic comparisons of pH based on only 2 decimals of “precision”, are not to be found so far as I know in ocean acidification publications. In the bigger picture of accuracy and precision, one can find that anywhere around the globe one examines the NOAA WOD geph records, one will find a thermocline and the same relations as above for dissolved O2, phosphate, nitrate, and pH. If the pH zigged when it should have zagged, then THAT would indicate poor accuracy. But to simply omit 2 million records of glass electrode pH measurements covering 80 important years from ocean science studies, without examining that accuracy, and without disclosing this omission, seems unacceptable.
Much like the Covid-19 testing paradigm, these concerns of “sensitivity” and “precision” appear to have led to the most wild inaccuracies imaginable. Yet they are the foundation that all now overtly rely upon. Here’s yet another example, easy to find, where pH values are posted to the third decimal place (so, highly precise) and yet their accuracy is astoundingly poor. In fact the majority of pH measurements shown range far above any values found in nature. They are truly off the charts. Consult any pH reference, even those by ocean acidification enthusiasts who nominally embrace precision over accuracy, to confirm.
Returning to the general reasons for the actual ocean profile thermocline patterns, it would be logical to assume that a significant fraction of Tyrell’s “sinking organic matter” includes again, the remains/carcasses of RNA viruses, which are excellent sources of both nitrates and phosphates and/or their redox appropriate cousins. The nitrates and phosphates are withdrawn from the oceans by living matter, and returned to the oceans, by decaying matter. And although the NOAA WOD won’t include pH contours in their parameters – contouring and profiling content, the nitrates and phosphates help us to also see the most likely high and low pH regions across the world.
 Limits of the Natural Environment in Terms of pH and Oxidation-Reduction Potentials G. M. Baas Becking, I. R. Kaplan and D. Moore The Journal of Geology Vol. 68, No. 3 (May, 1960), pp. 243-284
Published by: The University of Chicago Press
Article Stable URL: http://www.jstor.org/stable/30059218
 Tyrell, T., and Law, C.S. 1997 Low nitrate:phosphate ratios in the global ocean Nature 387, 793-796 https://www.nature.com/articles/42915#Sec1
 Palmer, M.R., Pearson, P.N., and Cobb, S.J. 1998 Reconstructing Past Ocean pH-Depth Profiles. SCIENCE Vol 282
*After thinking that through, I checked and found this: False-positive COVID-19 results: hidden problems and costs (thelancet.com) which, if I’ve correctly interpreted, describes that both false positives and false negatives are a concern for PCR tests.
7650total visits,2visits today