Coronavirus (Covid-19) and vaccine proof: What’s working and what isn’t?

You need to prove you got vaccinated — maybe at the border, maybe at the door of the bar, or at the airline gate or the sports arena. What to use?

There’s always your paper vaccine record. And yet a number of electronic vaccine passes have sprung up, some more useful and reliable than others. In the United States alone, there’s a hodgepodge of solutions, some run by states (New York’s Excelsior Pass), some by other states contracting with private entities (Docket in New Jersey and Utah), some by city governments (NYC Covid Safe) — and some states that refuse to observe vaccine mandates at all. (Here’s a list of states and their vaccine passport stances, dating to July 2021.)

We looked at the landscape and talked to some experts, including a man who checks the passes at a New York venue, which he says produces anxiety because of the wide variations. We talked to people using the New York Excelsior Pass with varying degrees of success.

Here’s what the experts say.

The 50 states and their vaccine policies

“This was a future that we saw coming, that there would be a world where there was a vaccinated population and an unvaccinated population, and that the vaccinated population would need to prove that they were vaccinated,”  Hana Schank, director of strategy for public interest technology at New America, said in a video interview. “Unfortunately, even before the first vaccine was administered, we knew this was going to be a problem, because there is no centralized data source that could actually provide this information. So the states have largely been left to do so even though this was like a federal initiative.”

A similar view comes from Noam Arzt, founder and president of HLN Consulting, a health informatics firm that works with public health agencies, at the federal, state, municipal and other levels on a wide variety of projects, mostly what he described as deep technology support and systems development.

“The U.S. is markedly different than almost every other country,” he said in a video interview, in that in almost every other country, health care delivery, policy, financing and information are top down. “But in the United States, it’s completely bottom up. There is no federal public health law in the United States.” Instead, he said, there are state, local, tribal, territorial and other public health authorities. The C.D.C. does have some influence, he said: “The C.D.C. gets states to comply with things because it pays them to comply with things and puts stipulations on the block grants that it that it gives them to do things in a certain way.”

“When it became clear that shortly, citizens were going to have to prove their vaccination status, a number of efforts popped up to try to make sense of those requirements and develop some solutions,” he added. “In most countries, this was conceived of and and thought about at a national level. In the United States, it was not and continues not to be. They simply do not speak about this and do not have a public strategy.”

The result is the distributed and decentralized approach we see today: State by state, largely, with some cities (like New York City) having their own approach.

It’s not a one-time miss, Schank said: “There’s a missing gap in the ecosystem of how our democracy works, which is that Congress passes stuff and things get kicked to the states. And every single time there’s technology involved in the delivery, somehow people are surprised that every single time the states are like, ‘We don’t know how to do this.’ And it’s just a cycle that keeps like repeating, repeating, repeating.”

Data collection

One of the fundamental problems is the data collection in the immunization information system, Arzt said. The C.D.C. primarily distributes vaccines through the states, though there is a separate path for national pharmacy chains, and a separate path for long-term care places, which are primarily serviced by the national pharmacy chains. There are also large national and other health-care systems, like the Veterans Administration, the Defense Department, U.S. prisons and the like. But the systems do not   communicate among each other.

More important, the C.D.C. does not collect “identified data,” with names and verification, on who got the vaccine and when —  they don’t have the right to do so, and many states don’t permit it. so the C.D.C. has deidentified Covid vaccine data. That leaves the patchwork of jurisdictions collecting data, and a common scenario, he explained: A person lives in New Jersey and gets vaccinated there, but works in New York, and the state records don’t necessarily communicate.

The problem goes back even further, he added. Most immunization records systems are built on childhood registries, so it’s not about the patient but rather the patient’s parent or guardian, with the provider supplying this data about the patient to the parent or guardian.

“With Covid, it became clear that there was going to be a pretty acute need to have patients have this data, and that the current infrastructure really hadn’t been providing it up until this point,” he said. In the absence of clear federal guidance, “Many jurisdictions sort of took matters into their own hands. And that led to a variety of solutions. In some cases, they didn’t have to do anything at all. Because the Covid shots were coming into the [Immunization Information Systems], they were then going wherever a patient’s record was going,” but not in a consistent pattern — to their provider’s MyChart portal if through a provider, but if not to some state or pharmacy repository.

This problem presented itself, he noted, during a pandemic, when the underfunded public  health care system was peopled by workers who are maybe sick themselves, struggling to work from home with kids on Zoom classes and dogs barking in the background, and parents in elder-care facilities besieged by Covid — just as the scale and importance of the work skyrocketed.

Vaccine data anomalies

Meanwhile, the data from various providers is feeding state dashboards, he said, and every time there is an anomaly in the state data, the governor’s office calls. Or the various insurance companies want to query the Immunization Information System, which, he said, “is a huge technical burden on a system that was never created to do that.”

Simple mistakes can point up the big problem. He told of one individual at the very beginning of the vaccine effort who got vaccinated at a state clinic, and then went to tell the health-care provider, who also dutifully recorded the vaccine but got the date wrong — resulting in a record that this person was fully vaccinated, with two shots, when only one had been administered (but it was recorded twice).

Israel, on the other hand, has a unified system, he pointed out. It’s a small country, where the overwhelming amount of health care is delivered by three HMO’s, and it’s not that hard to collect meaningful data centrally. The World Health Organization and the European Union are also convening around consistent standards, he said — while the United States maintains its state-by-state approach.

“For some states, they perceive this to be an important problem,” he said. “Some states frankly, have outlawed it. In Florida and Texas,  these things are outlawed by legislative or executive action, because of the politics around vaccination and Covid.”

The underlying record-keeping is a problem

“Anybody who is a technologist will tell you that you want data sources to be centralized,” Schank said. “The idea that you’re going to have 50 different data sources, doing 50 different things is a terrible idea. There isn’t really a validated, centralized database, of which vaccines have been administered, and which haven’t,

“I live in New York. So clearly, the state has some kind of database where they’re storing stuff. But it’s not validating your identity, along with the vaccination data. So technically, not to sound weird, but that kind of makes it not real. In the sense   that you can’t fully trust the data, you don’t actually know who the people are, or who have been vaccinated — beyond that they showed up and told you their name.

“So the state of New York decided, even though they don’t really have the data, they’re going to throw some money at it and pretend that this is a real source of data, the single source of truth. So the Excelsior Pass really is an effort to put the cart before the horse. There isn’t a single source of validated data that is 100% accurate, or even 98% accurate. But the state has chosen to rely on what they have.”

On a New York vaccine site on Facebook recently, people were complaining that their CVS vaccine records were wrong — a misspelled name for one, and another said she checked her records to see that CVS had recorded her birth date wrong, and “it shows that I took a booster yesterday and today.”

Another woman wrote that she had scheduled and received a booster Dec. 6. Then on Dec. 7, she received an email from CVS saying her appointment had been canceled. The email said if she believed this was an error, she should confirm. She went to the CVS and explained, and indeed she was not in the system. She asked to be reinstated, and said the person on duty added her vaccine date. She then got an email saying the booster was dated Dec. 7, and she went back and insisted that it be changed to the proper date.

Electronic apps

Examples of misfires on the electronic apps are myriad. We recently wrote a post about New Yorkers’ experiences with the Excelsior Pass, the state’s vaccine credential.

Built at a reported cost of $27 million to IBM from New York State by mid-August, the pass has had a spotty history. Many people are finding it to be useful — downloading it and filling it out successfully, and flashing it to an admissions staff at a restaurant or venue.

But many people had problems. One woman’s husband’s vaccine record had two letters transposed in the record of his vaccine, and he could not get it fixed. One woman said her CVS vaccine record had her name correct but used a 15-year-old address in Virginia, and she could not get the two married up.

One woman wrote: “I cannot get the Excelsior Pass to recognize my info. I tried several times. I called the local pharmacy where I got vaccinated and we went through all the info painstakingly, checking for misspellings or variations. Everything was correct. They resubmitted it, just in case, but it has made no difference.”

Others had different experiences; you can read about them here. 

Few scans, more paper

We heard from a number of New Yorkers that they had abandoned the Excelsior Pass and were instead using their paper vaccine record.

Vox’s Rebecca Heilwell wrote on Recode in late October of skimpy usage: “While New York state has issued 6 million total Excelsior Passes in the 29 weeks since launch, those passes have been scanned just over 314,000 times, the governor’s office told Recode on Thursday. That amounts to an average of about 10,800 scans per week. The app that businesses use to scan Excelsior Pass QR codes has been downloaded around 156,000 times. …

“Neither New York state nor New York City require businesses to scan the Excelsior Pass, and the low number of total scans suggests the Excelsior Pass is too complicated for its own good. Meanwhile, workers don’t have a strong incentive to prolong the process of checking people’s proof of vaccination when they already face harassment from customers angry about Covid-19 guidelines.”

The New York Times reported in mid-August that the pass “will likely cost far more than originally expected, with projected costs nearing $27 million, according to newly obtained documents shared with The New York Times.”

Checking the apps is a problem

The actual act of checking is also fraught, Schank said.

“When you think about who are the people who are being tasked with that labor, those are very often either minimum wage workers or people who don’t want to be in that position,” she said. “They don’t want to start a fight.

“It’s different if you’re a bouncer at a bar, and that’s your job. But the waitstaff at your local restaurant? They didn’t sign up for that, and they’re not trained on it. So I think there’s a huge issue with this: who is doing the validating?”

A New York woman who went to a bar recently showed her Excelsior pass, and only later did she realize that she had displayed a version of the pass that was expired, with a big notation “Expired” on the screen. Only after she had been there for a while did she realize she had displayed an expired pass. She went back to the front to show her un-expired pass — but the front desk people dismissed her.

She wondered later how many other people had displayed expired or fake credentials.

The idea of the pass was to obviate falsified C.D.C. credentials, it being true that making a fake paper record is cheap and easy. (Here’s the 167-page Excelsior Pass contract, reportedly, from the Surveillance Technology Oversight Project.)

Documented and undocumented

In the early part of the rollout, when vaccines were doled out on a state-by-state basis, many vaccine sites were requiring that people document that they were a resident of New York, or Connecticut, or New Jersey. In fact at one point, New Yorkers heading to Connecticut with vaccine appointments were turned away.

This early documentation requirement eventually was relaxed, but in a non-linear way, depending on the site. This happened partly because vaccine supplies became larger, and also partly because authorities realized that undocumented people needed to be vaccinated — and that requiring them to show identification was a definite deterrent for those who wish not to tangle with immigration authorities.

“It’s a really important point,” Schank said. “Part of why New York in particular didn’t know what to do is because they’re stuck between, ‘well, on the one hand, we need to know who the people are in order to actually track what’s happening and who’s getting what,’ and then, take that a step further to target areas where people aren’t getting vaccinated.

“But there’s a real tension between that and large undocumented population who we want to keep safe — not knowing what a federal entity might do with a giant repository of names of people who are undocumented.”

Data from several states was revised recently by the Centers for Disease Control and Prevention, in what was described as an update by The New York Times. But if those state databases are being used to drive eligibility on apps, then isn’t it a problem of garbage in, garbage out?

“In the largest revision of state vaccination numbers to date, the Centers for Disease Control and Prevention updated those for Pennsylvania, which had counted about 1.2 million more doses than had actually been administered,” The Times wrote.

“The C.D.C. said the data, updated almost every day on its website, had been corrected. As of Tuesday evening, about 81 percent of people in Pennsylvania had received at least one shot of a vaccine, according to C.D.C. data, whereas on Monday the data indicated that about 84 percent of people in the state had gotten a shot.

“The agency has been periodically revising vaccination numbers in states since July 14. Altogether, the C.D.C. and the states have reduced the number of reported doses in the U.S. by about 2 million.”

Issuer, holder and verifier

Arzt pointed out that California’s state government made a strategic decision different from New York’s: With no state mandate for use of the data, the focus became making the data available to citizens. In the vaccine credential world, he pointed out, there are three main parties: The issuer, who issues the credential in the data; the citizen or holder, who gets that data about themselves to show to others; and the verifier, who could be a bouncer at a bar, the ticket-taker at the sports stadium, or the airline desk person at flight check-in.

That seems easy until you understand that all three have different motivations, he said.

“Also, a lot of this movement is being driven by people with a very strong consumer rights perspective, so this is about empowering consumers, not protecting public health,” he said. “The other thing is that really from the start, very few people involved in this know anything about immunization. There are a lot of technologists, but not a lot of healthcare people. They could just as easily be trying to figure out and create an application for you to prove that it’s time for the oil change on your car.”

The result, he said, is that the credential is supposed to substantiate the vaccinations you receive: What they are and when you got them. “But there is nothing in the credential that substantiates anything about whether those were appropriate vaccines, or whether you’re actually fully vaccinated or not. Those are value judgments that are applied to the raw data. So it’s up to the verifier — ‘how am I supposed to understand this data?’ And that’s far from straightforward.”

With three vaccines and a clear regimen for each (two shots for Pfizer, two for Moderna, one for Johnson and Johnson) it was clear — until booster advice appeared, confusing the picture for the verifier.

And, of course, the rules can change. For now, two doses of Pfizer or Moderna make you “fully vaccinated,” though of course the C.D.C. is urging a third, “booster” shot for most, and definitely a third shot for immunocompromised people. Could that change? Yes. In a White House briefing in late November, the C.D.C. director said for right now, two shots is “fully vaccinated,” though that could change “as the science evolves.”

Further complicating the picture are overseas requirements. In Israel, he said, three Pfizer shots makes you fully vaccinated; in the U.S., the C.D.C. still considers two shots a full vaccination regimen. So if you go to Israel from the U.S., with two shots, you may not be approved.

“We have lots of people coming into the United States who have other vaccines,” he said. The specific vaccine from from Russia, several from China, the AstraZeneca vaccine that’s actually manufactured here, but not authorized for for us here. In fact, W.H.O. recognizes something like 10 vaccines, and there are some vaccines, like the Russian Sputnik vaccine, that aren’t even on that list, and one of Chinese vaccines isn’t. So  what do you do with a foreign college student who comes into the United States with a W.H.O.-approved vaccine but not a U.S.-approved vaccine?”

Wide variations, statewide bans

In some places, like New York, there’s the state’s Excelsior Pass, downloadable as the NYS Excelsior Pass Wallet, and the NYC Covid Safe app, which also stores your ID inside the app. The New York scene is complicated by the fact that many people live in New Jersey and work in New York, so there might be a vaccine registered in the New Jersey system that doesn’t show up in New York’s.

Many places will accept only a paper card, Arzt said, as for example in a wedding he went to in the Hudson Valley recently.

A number of states have made vaccine mandates illegal. “If I took my Excelsior Pass to Florida, it wouldn’t matter, because they’re not checking anyhow,” Arzt said. “In Florida, you barely need to wear masks.”

Here’s a list of states with bans; it is from mid-July, so things might well have changed. Keep in mind too that some cities have policies at variance from state policies. Also individual businesses may have requirements.

When we mentioned wide state variations, Schank said: “It’s noticeable if you have traveled to other cities or states that every place has their own rules that they are really, really adamant about, and you have no idea what those rules are.  I was visiting my brother in L.A., and in L.A., they  are really good at masking —  they’re masking a level that we don’t mask here.

“But then, when you go from here to Maine, for example,  nobody’s masking. I’m not surprised that people are   chattering about what is happening where, because   we’re not used to this.  When you drive from New York to New Jersey, the highway quality doesn’t change. So to have these disparities   in such an immediate way, I think, is something that we haven’t encountered before.

“I think you could make an argument that  maybe this was the vision for the  federated model, that things would be super different everywhere. And we kind of  didn’t live up to that until, until   masking and vaccines —  and this is what we needed to push us into all of our crazy little variations all across the spectrum.”

Iowa’s ban

In Iowa, the ban is having unintended consequences.

Axios reports: “Vaccine passports are becoming the new standard for the music industry, but Iowa’s law banning businesses from asking about vaccination status creates a lose-lose situation for venues booking artists, said Sam Summers, owner of Wooly’s and First Fleet Concerts. …

“Iowa’s music and entertainment venues are trapped in a corner — either lose ticket sales or, if they break the law, lose state funding.

“Iowa venues are deciding for themselves how to handle bookings if an artist requires ticket-holders show proof of vaccination for entry. But there are risks involved in each decision.

“The Iowa Events Center is requiring either proof of vaccine or a recent negative COVID-19 test for concerts where the performer asks for such a policy. So far, those have included Michael Bublé and James Taylor.”

BUILDING THE APPS

Building the apps is a problem. As Arzt noted, “1,000 companies are trying to jump into this space because they see opportunity — mostly companies not from the health care world, but social media companies, all kinds of companies,   think that there’s money to be had. And these folks most often don’t know anything about  vaccination, let alone the nuance that I’m talking about — which one, which matters, and that will only get worse as time goes on.”

One company Arzt said does have vaccine expertise is Docket, which works with New Jersey, Minnesota and Utah. It lets people download and carry their vaccination history, as much or as little as they want. It’s accessible by QR code. But Docket has had some security issues, notably a QR security flaw found by TechCrunch and reportedly immediately fixed.

Consumer Reports noted that ,”As if things were not confusing enough, MyIR, an app working with seven states — Arizona, Louisiana, Maryland, Mississippi, North Dakota, West Virginia, Washington, and West Virginia, as well as Washington, D.C.—announced in late October that digital vaccine credentials were being temporarily suspended (PDF). (Residents of these states can still obtain other digital vaccine credentials.) The problem is that MyIR aggregates data from various states and providers rather than directly on the behalf of each, according to VCI, the group working on standards for SMART Health Cards.”

The shifting landscape includes newcomers like Clear, the company that you pay for a shortcut through airport security, which has a new vaccine app that requires facial recognition: A picture of your face to accompany your vaccine card and your government ID to allow entry. But privacy issues are raising some questions.

“That facial recognition that you give to CLEAR to get into a game today could end up allowing some other company to ID your face next year,” NPR reported. “But CLEAR says that’s not what’s happening. Its privacy policy promises not to rent or sell personal information, though it can share the data with companies it calls service providers.”

While Clear says it’s optional for entry into some venues, NPR said, “In practice, though, you can find events that say everyone should use CLEAR, or they make it seem impractical for you not to. And it’s understandable that big venues prefer this uniformity. In this, CLEAR is filling a void left by the government when the White House pledged in April not to create a national vaccine passport system. “

Arzt said a number of companies are also jumping into the market to supply verification services for the passes. “There are a million companies trying to monetize that,” he said. “One of the things they’re struggling with is that the states by and large, given everything else they’re doing, can’t be bothered to give them access to the immunization registries, or it’s not within their law or regulation to give these third parties access to data in the immunization registry.” So there’s a lot of confusion in the market, he noted.

International travel

In Germany, teams of police spot-check riders’ medical status to see if they are protected. Here’s a video. This could not be done in the United States.

On Nov. 28, “Swiss voters on Sunday gave clear backing to legislation that introduced a system with special COVID-19 certificates under which only people who have been vaccinated, recovered or tested negative can attend public events and gatherings,” ABC News reported, with 62 percent of voters supporting the legislation.

What can be done?

It’s hard to imagine a global solution right now.

Schank said she took part in a press conference at MIT as the passes started getting rolled out. MIT had done studies determining that paper was going to be the best solution, she said, but “nobody paid any attention to that.”

“I don’t think it’s too late though,” she added. “I think there could still be a mayor who came in and said, ‘O.K., in New York, we’re going to get this cleared up — we’re going to actually have a database set it up so that we are not putting people in danger but protecting people’s health.’

“Yes, it’s very, very challenging. It would have been better to do it a year and a half ago. But it’s not too late. It’s just that it would require somebody to come in and really be a leader. But we’re not seeing that yet.”

“There’s a gap in the ecosystem of how our democracy works, which is that Congress passes stuff and things get kicked to the states,” Schank said. “And every single time there’s technology involved in the delivery, somehow people are surprised that every single time the states are like, ‘we don’t know how to do this.’ It’s a cycle that keeps like repeating, repeating, repeating.”

To solve the problem, Arzt said, depends on defining what you think the problem is. If you’re not going to sports events or dining inside, this may not actually be a big problem for you — at least not right now.

“There’s a difference between a credential and a pass,” he said. “The credential is simply the statement that you received three doses of this vaccine on these dates.  The pass says, for a particular use, you’re good to go. What’s in between those two? Rules that say, in order to get from the raw data to the thumbs up, you  have to evaluate that data through a set of rules that are in part influenced by the context.”

The rule for entering Aruba could be different from the rules for entering the United Kingdom or Iceland, he explained. So you can’t just show the pass and say that’s enough, because the credential of vaccination should be examined in the context of the pass.

“There’s an under-appreciation that there’s a different  meaning for a credential and a pass, and an under-appreciation that these rules are only going to get more complicated as time goes on,” he said. “More vaccines. More people at more ages, in more combinations.

“Right now in the US, you can’t mix and match your primary series of two doses for for any of them. Well, that’s not  a self-evident set of rules, right? Now if you’re looking at a card and you see two Pfizer doses and and one Moderna dose, that’s O.K. — well, it’s O.K., depending on the order of them, right?” If the sequence was Pfizer-Moderna-Pfizer, without a booster, then that’s not acceptable under current rules, he noted.

“It will get more complicated,” he said. “We have foreign students who came to U.S. universities this fall with vaccinations from products that are not authorized for use of the United States. What do you do about that? If a student comes from the UK with an asked AstraZeneca vaccine. Is that O.K.? Or do you re-vaccinate them?”

—

Read more:

Our piece on the New York Excelsior Pass

What’s it like to be the person checking credentials