The purpose of this note is simple: it provides a snapshot of what an independent review could do and shows why one is necessary for the bus redesign's study report.

In reporting results from the survey performed in the first phase of research in 2021, Table 3 on page 6 of Phase I Community Engagement Report reports that 17% of the entire sample thought “maximize access to jobs and opportunities” was one of transit's top objectives. Many more - 29% - thought "basic access to everyone who needs it, wherever they are" was one of transit’s top objectives. Thus the report muses thoughtfully (p.6): “This particular response data … suggests the desire for more of a coverage network among all respondents, including frequent transit riders, lower-income individuals, people of color, persons with disabilities, seniors, and young adults."

Also according to the figures in Table 3, roughly 58% (1,606/2,764) of the respondents are "frequent transit riders" defined as riding "often" or "every day" pre-pandemic. Yet according to figures in the very next table (#4), only 14% of the respondents were “frequent transit riders” pre-pandemic: 3% riding almost every day and an additional 11% riding 2-4 days a week. In contrast, 57% rode “rarely.”

As someone who has spent most of her professional life analyzing data, often from social surveys, I am used to spotting unintentional errors of all kinds, errors in which one could come to a conclusion diametrically opposite to the one that should be arrived at. In this situation, “roughly 58%” and 57% of somewhat different sample sizes (presumably without “unknowns” or “no answers”) are too close to each other not to suspect a simple data entry error. But which table is in error? Furthermore, if an error occurs so soon into a review, who knows how many other errors are sprinkled throughout or what the impact is of that on any conclusions?

The risk of simple error is unfortunately common and is why it is so important that everything undergoes an independent review before being widely accepted. Everyone makes mistakes, but good policy is based on reality, not fiction. And in this case, it is unclear as to what is true and what might be the opposite from true. Instead of catching the error, it is repeated without correction in the appendix to the Alternatives Report. How many other errors lurk somewhere?

While good practice would have called for a third-party review much earlier, it is never too late.