Will Certification Kill the IHE Connectathon?

Summary: IHE Certification has been announced, is dubious but probably unavoidable; the for-profit ICSA Labs will execute it, possibly gutting the participation in future Connectathons and leading to their demise

Long Version.

With unseemly haste, and little time to go before the IHE North America Connectathon starts at the end of January 2013, IHE USA has announced a new IHE Certification program to be executed by ICSA Labs, a for-profit division of Verizon that has significant experience in security certification and is also a Meaningful Use ONC-Authorized Testing and Certification Body (ONC-ATCB), which competes with the likes of CCHIT.

There is no point in debating whether certification adds any value to the customer of interoperability products; that cat is already out of the bag, for MU at least, since ONC has a hard-on for certification (and asks standards organizations like IHE and DICOM uncomfortable questions when there is no corresponding formal certification program). Compared to all the other ludicrous costs of bring a product to market that do little if anything to improve quality or assure safety or interoperability, this is just yet another check box (and lifetime employment program) for the army of quality, safety and regulatory personnel. I don't have a strong opinion about the merits of certification per se; my anecdotal experience as a vendor and a customer and a worker in regulated industries would lead me to believe that it is neither necessary nor sufficient, unless it is conducted with such thoroughness (by which I mean depth and coverage of testing, rather than volume or rigor of documentation) as to be impractical. But I don't have any real evidence one way or the other.

I have no direct knowledge of the IHE USA Board's discussion of the matter, so can only guess as to their motivation. It isn't likely to be the potential for a significant revenue stream, since the price point precedent established by the ONC-ATCB market isn't very high, and the cost of doing the vast amount of ISO-standard inflicted paperwork isn't trivial. I am not sure whether or not IHE USA is getting a cut from ICSA Labs, or perhaps is even funding them.

The haste smacks of desperation, so another guess would be a desire by IHE USA to try to remain relevant in the US national "discussion" on what standards to adopt (if you can grace the ONC autocratic decision-making process with that term). Since a nice thing about standards is that there are so many of them to choose from, and anyone can make up a new one and call it a standard (as the DIRECT folks have clearly demonstrated), there is no question that choices need to be made. IHE USA's difficulty may be that, much to their chagrin, the ONC does not accept as a no-brainer that IHE's choices are always the best choices.

There is also little point in bemoaning what, as far as I can tell, has been a distinct lack of consultation with the IHE community at large. Almost nobody I have talked to knew anything about this before it was announced, and even those meetings that I am supposed to participate in (being Rad TC co-chair) like the Domain Coordination calls (which discuss such boringly bureaucratic and tedious topics that even I with my love of standards documentation can't stomach), were not, I am told, briefed about this. Nor was there a peep on the Testing and Tools mailing list. Clearly the lowly workers did not merit either the opportunity to comment nor advance warning. Strangely enough, the vendors I have talked to were not aware of it either, and indeed the vendors have a long history of opposing the additional burden of certification and have lobbied against it on the basis that self-certification has a demonstrated successful track record, and certification adds cost without value. Perhaps the IHE Board knew how controversial this would be and did not want their decision second-guessed, or perhaps they just got snowed by a plausible argument by a talented evangelist for the concept.

Or perhaps ONC just issued one of its edicts and the board had no choice ... conspiracy theories abound given the lack of transparency in the decision making process.

For better or for worse, unless the pilot ICSA Labs project overlaid on top of the IHE NA Connectathon somehow fails spectacularly, IHE Certification, now that it has been "announced" may be here to stay. If few vendors sign up for certification, then it might fail, but given the loss-leader price of $1,500 for each actor (in packs of 5) that they are offering, it seems unlikely that there won't be a sufficient critical mass of product managers who are tempted to get an additional potential marketing advantage over and above their normal Connectathon passing "gold star", or are just scared of being left behind if certification does become a trend. Today's briefing by ICSA Labs was certainly laced with dubious but blatent teasers for marketing folks. Besides, like UK National healthcare IT projects, it may be sufficient to "declare" victory even in the face of obvious failure, if the political motivation to proceed is strong enough.

So, with that background, on to the question of what this means for the upcoming and future Connectathons. One of my initial concerns, that the work product of the army of volunteer monitors who staff the Connectathon would somehow be "re-labeled" as a certification result, was clearly assuaged by ICSA Labs on today's call - their tests will be separate from the Connectathon tests, executed by their own staff (who they are calling "inspectors" to distinguish them from "monitors"), though they may repeat the same test, or add more steps or tests, and will certainly add more documentation. So, the volunteers will not be "exploited" in the sense of their work being reused for someone else's commercial gain. Nor will ICSA Labs interfere with the process; the certification work will be additional, and later in the week, though predicated on a Connectahon pass.

So far, so good, but what about in the long term? If I, as a vendor product manager, feel the need to obtain certification for any or all of my products, why would I bother to participate in the Connectathon at all? If I have to do the same amount of work, gain the same experience, and the additional cost is small related to the real or perceived marketing benefit, why do it twice? Though the certification testing will take place at the Connecthon itself for the pilot, in the long term ICSA Labs plan to do this separately as needed, virtually (over the Internet) if possible. Arguably, the Connectathon serves as dry run, but those can probably be done just as easily at home or informally.

Similarly, why would I ever volunteer to staff a Connectathon, when I could probably charge my hourly rate to ICSA Labs, or get a job with them, to do exactly the same thing (unless of course ICSA Labs treats its employees like parent company Verizon treats its linemen)? There is an educational component to volunteering, but for experienced folks, the supply is small and now there is a competing consumer.

That is why, at this date, I am opposed to IHE Certification process as announced; not because I think it is a waste of time and money (which I sort of do), but because it puts at risk what I think is much more important, the opportunity to gather and learn and test and even experiment at the Connectathon, in a relatively informal manner that encourages iteration, improvement and collegiality between competitors. In my opinion, there will inevitably be a dramatic reduction in Connectathon participation as a consequence of the need to become certified; regardless of how much the engineers might want to test at such an event, their marketing bosses who hold the purse strings just will not have the will or the resources to do both.

If I were one of the IHE decision makers, and felt bound to jump on the certification bandwagon to survive, I might have made a different choice, one which did not put at risk the Connectathon, but instead leveraged it, and found a means to certify Connectathon results, building on the experience and dedication of the existing staff, and added the necessary documentation (preferably automated) to satisfy the ISO processes. This sort of thing has been discussed before, including by the people who do the work. I dare say I would have been skeptical about the feasibility and initial cost of this, and tempted by the offer of an existing organization like ICSA Labs with the ISO infrastructure already in place, but I would have prioritized the future of the Connectathon, and seen the modest revenue stream from certification as a means to (belatedly) beef up the tests and tools that have languished somewhat for lack of resource investment over the years.

Not to mention avoiding the slap in the face to the core Connectathon staff, who, for well over a decade, have been putting on this event with diligence and dedication, and who are now probably destined for unemployment or a future as underpaid mistreated Verizon drones, in addition to seeing their many person-years of work in the form of openly available tests and tools handed off to a for-profit to reap the benefits, as well as being denied the opportunity to turn their vast talent to performing certification themselves. They are obviously not able or willing to speak about this, but observation of their reaction would suggest that they were not consulted or warned about this either; if that is the case, it is shameful treatment by the IHE organization that they have served so loyally.

Given my upcoming New Year's resolution to be more of a glass half full kind of guy though, perhaps I should view this as a precedent and take the opportunity to start a new initiative for DICOM Certification, probably the last thing the world really needs, but hey, there is no point in standing on principle in the face of a tidal wave, and if you can't beat them join them. Give the customer what they think they want, not what they actually need.

I still haven't decided whether or not to withdraw from volunteering as a monitor for the upcoming Connectathon. I would hate to miss it, but given that it is one of the few tangible ways to express my displeasure, I don't think I have much choice other than to boycott the event.

David

PS. An interesting side question, is why IHE USA did not select the non-profit CCHIT organization as a partner in this, particularly since they are doing the certification for Healtheway (formerly the Nationwide Health Information Network NWHIN). Perhaps it is just because Mike Nusbaum, a long time IHE player and expert has joined ICSA Labs as Program Consultant and seems, judging by today's briefing, to be a key player in this effort; they certainly have someone who knows what they are taking about. Not that I have any great love for what CCHIT does either, just wondering.