COVID Alert App Postmortem — Part IV COVID Alert Series
High Level Mapping and Reframing
Please see parts one, two, three, and this oped as prior information.
To continue to try to understand what went on/is going on with Health Canada and the COVID Alert app, we’re going to try to construct a map of the different pieces of the puzzle. To keep things simple, I am going to focus in on Ontario as the case study. As we know, the app is active in 7 other provinces and one territory, but early information about how the app works in each province is that it’s not the same across them. So we’ll start with the way the app works as a collaboration between the government of Canada and the government of Ontario.
As an aside, it has been brought to my attention that in Switzerland, there was jurisdictional issues as well. It would be good to start to compile a list of how the problem in Canada played out in other countries.
The Start of a Map and Complexity as a False Flag
It has been interesting to start to review the documentation, code, and data made available about COVID Alert. I stopped pretty early in looking at it because I could feel that the issue was not being framed in a way that put adequate importance on the most unclear parts.
As I mentioned, the documentation for the COVID Alert app, at the federal level, appears to be pretty exhaustive and comprehensive. I’m nowhere near done my review there, but that’s good so far.
But when we turn to the provincial level, we have a different story. There are two data sets I’ve found — one that has information about downloads, one that has information about the use of one-time keys. Those data sets appear to tell us nothing about about items 3, 4, and 5 in the schematic.
There are also two small things of note here. One, for such a high-visibility data set, the labelling is sloppy, including grammatically. It reads more like marketing than something mathematically rigorous. The data about the data is scant, you can’t tell much about the process behind its creation. This has been a long topic of discussion in the open data community, the need to provide adequate context for what is shared.
Two, the information about the stewards of the data is lacking:
I am going to start sending questions to the open data team about what isn’t here, but suffice it to say, one is supposed to be able to understand a lot more about the process that drives the data shared than what is provided here.
The Provincial Software and Process We Need to Know More About
As outlined in my oped in the Toronto Star from earlier in the week, there are so many different points of failure to consider in a broad postmortem, and the most important ones are cultural and governance related. But to inform those conversations, it helps to understand how this app was functioning so that all the appropriate parties are identified for the parts they had control over. The interplay between the federal and provincial governments comes together around the part of the process that is called one-time key provision and use.
The two main numbers that the governments have been pointing at to understand uptake and use of the app are 1. downloads and 2. one-time keys used.
Downloads
The downloads part — the marketing and money behind the federal government’s campaign to get people to use the app. This is fairly well tread ground. The official number of downloads across the country, as of February 1, 2022 (the number has not been updated since) is 6,893,423.
There have been people telling me this number is inflated for a range of reasons and that there are other ways to get to this number that are more accurate so let’s park that for now.
This downloads number focuses on how many people have the app on their phone. There is extensive documentation for the app in this state, just existing on your phone. A good number of the consequences of you downloading the app and having it on there have been widely considered. This state of use, just having it on your phone, is informed by the Google/Apple Exposure Notification protocol (8) and by how the COVID Shield app (9) was originally designed by Shopify volunteers.
I would like to make an observation at this point in my reading. There is a wild amount of ink spilled and documentation related to these parts of the app in this state. Sitting on your phone, listening for other phones. While this is important, it seems as though the amount of attention and energy into making that part of how the app worked so transparent has no equivalent on the provincial side. This feels as though it’s an early indication of the problem with transparency. The “what” of transparency can not only be framed in ways that are confusing, it may completely overwhelm the space with the wrong, or at least the less important information.
One-Time Keys and the Province of Ontario
The number of one-time keys used across the country, as at February 1, 2022, is 57,704 (source). The number of one-time keys used in the province of Ontario, as at April 24, 2022, is 48, 764 (source).
While there are technical specifications for what the app (both the code on government computers (2) and the code on your phone (1)) does with a positive test result, I have been unable to far to find a workflow document that describes the following process which leads up to that event:
- person gets a PCR covid test at a provincial lab
- PCR test is positive
- person gets news of positive test and one-time code from the province
From what I understand, there is a web-portal called ‘Lab Results Viewer’ where someone in Ontario could access both their test results and the one-time code, it’s a web portal. I don’t know if this is up and running for the special cases of people that still qualify for PCR tests. And remember, this process may look different in another province/territory. This is strictly looking at Ontario.
In October 2020, there is reporting about people not getting their codes. Here’s a local example from Toronto, from the CTV article, in italics:
“I started calling the clinic, Toronto Public Health, the app help line. I kept going around in circles,” she told CTVNews.ca over the phone Tuesday. “In the end, they told me ‘don’t worry about it, we’ll take care of the tracing.’”
At the point, this issue did not appear to be making its way up to political leadership. From the same article, in italics: “Deputy Prime Minister Chyrstia Freeland said that the government isn’t relying on “a single magic bullet” like a smartphone app and that she was not aware of concerns related to missing COVID Alert key codes.”
This is two full months into the launch. And this is a good example of how the most important parts of this post-mortem, which are about government accountability, are about VERY different information and data flows than how the app manages personal data.
(Aside: if you successfully received a one-time code from the province of Ontario, can you please get in touch with me?)
Adventures in Mapping Ontario Health and their Workflow for the Keys
So here I am going to stop for today. I have emails out to Public Health Ontario and the Information and Privacy Commissioner of Ontario to try to understand more about this process. The Information and Privacy Commissioner of Ontario, Patricia Kosseim, endorsed the app on July 31, 2020. As such, one can imagine that they were able to do a forensic-level analysis of the one-time key process when the app was launched. This is also a great case of seeing how the information part of their remit matters a lot, as this inquiry is much more about how the thing was working than an assessment of privacy issues. I am also going to reach out to eHealth Ontario to see if/what they know about this part, as well as Ontario Health.
On Process
Last thing. I am writing in small little bursts to try to work and think in the open. This first step of the efficacy information is but one tentacle of several and to be frank, trying to hold even one instance of this (government of canada/province) is overwhelming. I started to get more carried away with the schematic with colours and more but wanted to stop and leave it up for missing parts before trying to make it more functional. it’s not all apples to apples even but it’s a start.
There is a law that I can’t remember about the internet, but it basically says that the fastest way to get the right answer is to share the wrong one. So while I’m not trying to share anything wrong here, I am aware that I am poking away in a mountain and hope people will continue to share corrections, additional information, etc. because I am absolutely for sure missing a lot. So as a working method, from here on out, I’m going to see if this schematic and its groupings prove helpful. They may not, or it may need to evolve. I will also start each next post with explicit corrections or expansions to the one prior.
