Starting the COVID Alert App Postmortem — Part III COVID Alert Series
Understanding the Timeline, Processes, and Workflows
Please see part one and two of this series. Also, today in the Toronto Star I suggest that beyond getting Health Canada to shut down the app, we need to have a formal postmortem about COVID Alert to document the lessons learned for future public tech efforts, particularly in terms of public accountability, oversight, and policy development prior to launch.
I’m not sure what the processes are for a formal postmortem, as in, from a democratic perspective, (will learn and share). So in the meantime, going to start trying to organize some of what we know about how this app came to be, its weak points, and publicly available data related to efficacy. This will help us to think about what kind of accountability frameworks we can insist upon *before* any kind of public health tech (or other public tech) is launched.
The App’s Origin Story
To consider the history of the app we have to look globally, nationally, provincially, and to Ottawa.
First, globally: Apple and Google introduced their protocol in April 2020. From a (long) wikipedia entry: “The (Google/Apple) Exposure Notification (GAEN) system, originally known as the Privacy-Preserving Contact Tracing Project, is a framework and protocol specification developed by Apple Inc. and Google to facilitate digital contact tracing during the COVID-19 pandemic.”
The trick with reading into the technical parts of this app story is not to get too mired in them or you’ll lose track of the point fast as well as the bigger picture. For a shorter read, and quick reminder of the rhetoric that was flying around at that time, see the Apple and Google protocol page: “All of us at Apple and Google believe there has never been a more important moment to work together to solve one of the world’s most pressing problems.”
So we’ve got Apple and Google making this possible with 1. their protocol and 2. their ability to make this app available through their app stores/reach a mass amount of people. What we don’t know is how much Google and Apple were chasing governments around globally to do something with it, or the other way around. Or both.
Next up, we head to Canada and Ottawa, where we’ve got the federal government and we’ve got Shopify. The COVID Alert app is the official government version of something called COVID Shield, which was built by Shopify volunteers, from its site: “We are a group of Shopify volunteers who want to help to slow the spread of COVID-19 by offering our skills and experience developing scalable, easy to use applications. We are releasing COVID Shield free of charge with a flexible open-source license.”
Here again we need to understand the dynamics. Did the government approach Shopify to see if they would do this? Did Shopify approach the government with the idea? Did Shopify just make the app knowing it would likely be picked up? What is the timeline and process that occurred between Shopify launching their app and the government deciding to use it?
Then we zoom out from Ottawa and look to the province of Ontario. The province of Ontario was the only province on board with this idea when it was launched by the federal government in July 2020. It took until October 2020 for the rest of the provinces and territory that joined to get into it. And then, remember, Alberta, British Columbia, Nunavut, Yukon have never joined. Alberta went on to make its own app — ABTraceTogether.
What were the conversations going on between the federal government and each of the provinces? Why did some join and others not? What was with the lag between Ontario and the rest of the joiners?
This is clearly not an exhaustive list of actors, but to start we can see we’ve got: Apple, Google, Shopify, the federal government, and the provincial governments.
The COVID-19 Exposure Notification App Advisory Council
Next up, we’ve got what was supposed to be some public oversight in the form of an advisory council. This council’s first meeting was held in August 2020, and the council met often until May of 2021. Then it went quiet. I was told by one of the council’s former co-chairs that “Council had come to the end of its term”. All the meeting notes are here. As is the terms of reference for the group. From the terms of reference:
“The Advisory Council will provide reports to the Deputy Minister of Health, Deputy Minister of Innovation, Science and Economic Development, Deputy Minister of Intergovernmental Affairs, the Secretary of the Treasury Board (responsible Federal Deputy Ministers) and relevant Provincial and Territorial Deputy Ministers on a regular basis. The reports will be on subjects tasked by the responsible Federal Deputy Ministers. The responsible Federal Deputy Ministers shall make these reports public, except to the extent that they compromise the security of the app.”
Few things here — apart from the obviously troubling one, which is that the app is still up and running and the council is gone. As we know from the terms of reference, this council was supposed to provide guidance on wrapping the app up, specifically that the council was for, among lots of other things:
“Providing advice for eventual wind-down of the app, including recommendations for the timely destruction of data.”
Looking at their reporting lines gives us a bit of an idea of how many people were involved in this project on the government side. The notable one of this group is the ISED — Innovation, Science and Economic Development.
It’s notable because part of why this kind of “health tech” (again, cringe, there is good health tech, this app is not it) is appealing is the idea that it opens up new market opportunities. This is not inherently problematic, but for the very narrow problem of a government intervention during a public health crisis, it’s worth hiving off to consider.
The App’s Component Parts and Related Workflows
Ok, last part for today — the first high level go at how at considering the shape of how this app worked in order to understand all of its pieces. This will need a schematic which I’ll draw by hand for the next post.
The federal/provincial jurisdiction piece of the puzzle of this app is highly significant. As we know, the app relies on PCR test results. Those PCR test results were the point where a one-time key was generated so that if you had covid you could upload your positive status to the app so it could do its notification thing with people you were around. I’m putting a pin in this part to come back to because the “science” behind this model was problematic from the start and only got worse as transmission science evolved. Who was engaged, on the government side, before launch, to assess efficacy of the proposed design? What documentation exists of these considerations?
So here we need to being to break the app into parts and when we do that we’ll be able to better follow accountability lines/governance approaches for different parts. For now, let’s consider 1. the app 2. the PCR testing/one-time key generation workflow
The COVID Alert App
This part is federal, Health Canada is the business owner of the app. The app has reams of technical documentation available at the Canadian Digital Services GitHub repositories. I had hoped to be further along in looking at them but not there yet. There are 13 results results there when you search COVID Alert. In the weeks ahead, and with the help of a trusty little schematic, we’ll look at all the bits and pieces there and look at the how the Google/Apple protocol comes into play as well.
It’s great to have this documentation available, but when I was looking at it, it reminded me of the early days of open data. Making this transparent and available is great, but on its own it’s not accessible. A schematic and an easy to understand diagram would be a great addition to what is there.
In terms of metrics for the app writ large, the number of downloads was the first of two main numbers of focus. Here is an open data set that lets you see the number of downloads of the COVID Alert app on a daily basis, and breaks the number of downloads down by operating system — ios for Apple and Android for Google.
Interesting aside here: that dataset, for national # of downloads, in hosted by the province. When I searched for the same at the federal open data portal, it circled me back to this one from the province of Ontario. Why is this data held/stewarded there?
The PRC Testing and One-Time Key Workflows
This part is provincial. Though I’m not sure if there is any formal accountability at the provincial level for the app. And when I say formal accountability, I mean like Westminster System governmental accountability. And are there memorandums of understanding between the provinces/territory and federal government?
The one-time keys are the part of the app that make it “work” — they’re what you get from a province/territory when you test positive for COVID. This means that there is an entire workflow going on outside of the app that may involve other software. Here the documentation is weak from what I’ve seen so far and it’s confusing to even think about without a schematic so more on this soon.
One-time keys used has been the second major metric of focus. There is an open data set for the one-time keys that have been used in the province of Ontario. What that means is the number of times someone got the code from public health, and uploaded it to their phone. Note: this data set is only for Ontario. I have not found if the other provinces/territory provide this yet.
Next Steps
Going to get a dedicated timeline tool going, as well as a first go at what will be a laughable schematic/workflow/processes diagram.
