March 30, 2007              

Debra Bowen
Secretary of State of California
1500 11th Street
Sacramento, CA 95814
ATTN: Voting Systems Review, 6th Floor

Enclosed are my comments on the Draft Criteria for the Top-to-Bottom Review of Voting Systems. Thank you so much for your effort to bring back integrity to our elections.

Jerry Berkman
3136 Eton Avenue
Berkeley, CA 94705
510-547-0985

Comments on Draft Criteria of March 22, 2007 in:

TOP-TO-BOTTOM REVIEW OF ELECTRONIC VOTING SYSTEMS
CERTIFIED FOR USE IN CALIFORNIA ELECTIONS

Page 1, Title:

The Draft Criteria's title says:
      "Electronic Voting Systems Certified"
while the first paragraph says:
      "voting systems currently certified".
"Electronic Voting Systems" may be interpreted as excluded optical scan systems. All voting systems certified for use in California should be reviewed. Delete the word "Electronic" in the title.

Page 1, paragraph 1:

The first paragraph says:
The goal of the review is to determine whether currently certified voting systems provide acceptable levels of security, accessibility, ballot secrecy, accuracy and usability under federal and state standards.
The draft criteria include sections/criteria for security, accessibility, and usability, but not sections/criteria for ballot secrecy and accuracy.

Pages 1, 2, and 4, use of word "standards"

Page 1, paragraph 1: "federal and state standards" should be "federal and state laws and guidelines". Page 2: "voluntary federal voting system standards" should be "federal voluntary voting system guidelines".

Page 2: "Security Standards" ... "For purposes of these standards ..." Page 4: "Disability Access Standards". I would use "requirements" instead of "standards" in these three places. "standards" might have a legal meaning and requirements for setting.

Page 2 and 3 Vote Tampering:

"untraceable vote tampering" is defined and then used in 4 additional places. Vote tampering, whether traceable or not, must be prevented. This is especially true as the public is not allowed to look at the logs or other places traceable vote tampering might have left its mark. So, I'd recommend changing the definition to:
"vote tampering" means preventing the accurate electronic recording of votes, or altering the record of votes, to change the result of an election."
After changing the definition, change "untraceable vote tampering" wherever it occurs later in the document to "traceable and untraceable vote tampering" or just "vote tampering".

Pages 2-3. I. 1. Security Standards

I'd like a subsection d. which would apply to any part of the voting system not covered by a., b., and c. In particular, some optical scanners may just convey ballot images or ballot selections and not do any counting and may not be covered by a., b., and c. Also the voter access card enablers wouldn't be covered.

Page 3. I. 2. b. Source Code Review

This says "prior to, during or after completion of the risk assessment", but that is the only place "risk assessment" appears; i.e. it is not defined and therefore I don't know what this sentence means.

Also, in my opinion, it will be impossible to review all the source code of all the certified systems. Perhaps the paragraph should be rephrased to indicate partial source code review, as time and resources permit. The findings at the end of the process must make clear exactly how much source code review was done. We don't want a partial source code review with vendors claiming California proved the code is bug free and vulnerability free. Actually, they probably will whatever you do, but you should try to make it clear that your source code review is incomplete.

Page 5. II. 2. (f) Accessibility

II.2.(f) states:
(f) In the case of a DRE, the capability to permit a voter to verify electronically, through a nonvisual method, the information that is contained on the voter verifiable paper record copy of that voter's ballot. This requirement is satisfied by a method of nonvisual confirmation that draws the information provided to the voter from either (1) the paper record copy itself or (2) the same electronic data stream used to print the voter verifiable paper record copy.
II.2.(f)(2) is not sufficient. The Elections Code says that the voter may verify:
      "the information that is contained on the paper record copy" (19250(d))
      "the information provided on the paper record copy" (19251(a))
via a nonvisual means.

II.2.(f)(2) at best conveys to the voter the information the machine expects to be on the paper record copy, regardless of what actually is on the paper record copy.

There are many situations in which the paper record copy does not contain the proper information:

For how to "game" the VVPAT, see: for more info.

Actually, I'm not even sure what "the same electronic data stream" means. When the VVPAT is printed, the "electronic data stream" is not saved. If you mean recreating the data stream sent to the printer, that stream will contain printer control characters, so it can not be sent to an audio device. If you mean the data stream before it gets to the printer driver, this would not protect against malicious code in the printer driver or sending a different data stream to the audio device.

Page 5. IV. Usability

"respond to voting system error message" - these should be in comprehensible English, not just error numbers.

"print end-of-day vote totals" - this should be independent of other printouts; i.e. not on sealed VVPATs.

Counties should also be able to set up elections, e.g. ballot definition files, etc. to avoid problems like those with ES&S being so late in the last election.

I've heard counties using ES&S systems have to have ES&S in Omaha, Nebraska, program the firmware for their county's machines before each election. If that is true, the software for programming the firmware may not be in escrow in California and may not even be certified for use in Claifornia. That can be tested simply by testing creating a new election.

Not Covered in the Draft Criteria:

1. Ballot Secrecy/Privacy

An item needs to be added as to how well the equipment preserves the secrecy of the ballot, as required by the California Constitution.

The continuous roll VVPAT does not seem to protect the secrecy of the vote, since: - anyone can observe at the polls (EC 2300 and EC 19362), - each voter must audibly state their name before voting (EC 14216), - anyone can request a recount of a precinct (EC 15620 and 15621), and - anyone can watch the recount of the precinct (EC 15629),

Michael Shamos, Pennsylvania's Certifier of Election Systems, won't certify any continuous roll VVPATs due to concerns about the secrecy of the ballot.

This is an even worse problem in counties like Alameda County, where on election day, only the disabled will use the DRE. Last November, most of the DREs in Alameda County held only one or two votes. If the results tape for the DRE is posted as required by the Elections Code and only one person used the DRE, then that person's ballot is displayed.

2. Security - Testing if code is escrowed.

EC 19223:
19223. The Secretary of State shall conduct random audits of the software installed on direct recording electronic voting systems, as defined in Section 19251, to ensure that the installed software is identical to the software that has been approved for use on that voting system.
Although required, I doubt the random audit has been performed recently. To do this, take random machines from the Counties and compare the software to that in escrow. To make a true comparison, you must compary binary executables. According to "Voting System Requirements", 10/5/05, http://ss.ca.gov/elections/voting_systems/requirements.pdf, these should be on file with the Secretary:
VOTING SYSTEM REQUIREMENTS

Any new voting system to be considered for certification for use in California elections will be required to have the following features:
...
5. In addition to depositing the source code in an approved escrow facility, each vendor must deposit a copy of the system source code and binary executables with the Secretary of State. ...

There have been articles stating that ES&S installs a different version of the executable in each precinct, e.g.:

"ES&S Programming Is Unverifiable", by John Washburn, http://www.washburnresearch.org/archive/ESSFirmware/ESS-Firmware-001.pdf

This can be tested by comparing the binaries and firmware on several machines used in different parts of a county or in a different county. If true, this would violate EC 19103. (a):

No voting system may be used for an election unless an exact copy of the ballot tally software program source codes is placed in escrow.

3. Security - Code on removable media.

The security tests and code review should include testing whether there are binaries/executables/scripts or equivalent on removable media, or whether the machine would use such code if it were there. If so, this is a security risk and should result in decertification.

4. Security - Testing mode

All testing should be done in election mode, not test mode.

5. Suitability/Usability - MTTF

The MTTF (Mean Time to Failure) in the VVSG would allow 10% of the machines to fail on an election day, where the machines are used for about 15 hours. This is far too many failures, putting stress on pollworkers and elections department staff. We must require a higher MTTF; a machine with a 163 hours MTTF may be suitable for a polling place at the elections department office, but is not suitable for offsite polling places.

6. Suitability - Calibration

Test how often a machine requires recalibration. Test to see if transportation affects calibration.

I don't think pollworkers should need to do recalibration. If that is the case, the MTTF of the machine with respect to calibration must be evaluated, and should be on the order of 100 hours or more, or this is not a suitable machine.

If pollworkers are expected to do recalibration as needed, the recalibration procedure must be simple. Otherwise, it fails the suitability requirement. After a machine is recalibrated, I would expect Logic and Accuracy testing to be redone. Is this required by the procedures?

7. Security - Storage/Sleepovers

How does this security review relate to how machines are stored between elections, transported to the polls, stored at election sites (often in public places), and so-called sleepovers with pollworkers?

8. Security - Early Voting and Absentee Vote Counting

For early voting, equipment must be secured for days or weeks after voting has begun. The equipment is probably left unattended overnights and weekends. Likely the equipment in the Registrar's main office is well secured, but the equipment in satellite locations is probably much less well secured. I have heard some counties have portable absentee voting stations on buses. Use conditions must spell out what security precautions are necessary.

Tabulating of absentee voting has similar problems when done before election day. Since the optical scan equipment is left unattended and vote counts are not allowed to be accessed until 8p.m. election day (EC 15101.), stringent security precautions are necessary to protect the vote. Since many counties have purchased new high speed scanners, it may no longer be necessary for them to start tabulating votes before election day.

9. Suitability - SOVC

In larger counties, the Statement of Votes Cast (SOVC) is huge. It should be simple for interested parties to access the numbers in the SOVC and put them in spreadsheets, etc. However, the SOVC is often only provided as a PDF file which can not be processed electronically. In Alameda County in the 2004 general election, the SOVC was 75 Megabytes, 6600 pages. Most of the data is just zeros, e.g. listing zero vote counts for Oakland precincts for a Livermore council race.

The SOVC should also be available in a more usable format, preferably as a CSV (comma separated value) file. In the current environment, I would say any system which can not do this would fail the suitability requirement.

If this can not be required as part of the current review, it certainly should be made a requirement for any future certifications.

10. Suitability - Recounts

The continuous paper roll VVPATs must be tested for how hard it is to count the votes by hand, as is done in the 1% manual tally and recounts.

Registrar Jill LaVine tested one vendor's equipment in early voting in Sacramento County. As a test, they manually counted the VVPATs for one of the early voting polling places: It took 127.5 hours to recount the 114 ballots, or approximately an hour and 15 minutes for each ballot. (http://www.eac.gov/docs/LaVine%20Testimony%204-20-06.pdf)

This would fail the suitability requirement for any county with significant early voting or vote centers.

11. Accuracy - Optical Scanners

The optical scanners should be tested for accuracy. Testing should include how dark and what sizes marks must be to register as votes, whether there is a difference when ballots are flipped end-to-end or upside-down or both.

Testing of this type is described in:

"Regarding the Optical Mark-Sense Vote Tabulators in Maricopa County" by Doug Jones, University of Iowa. http://www.cs.uiowa.edu/~jones/voting/ArizonaDist20.pdf

In addition, there should be testing with of various types depending on the types of choices on the ballot.

This is described in John Washburn's Election Integrity site:
"Testing Voting Equipment", http://www.washburnresearch.org/ElectionIntegrity.html

12. Accuracy - DREs

There have been many cases of vote flipping. The DREs must be tested for vote flipping and whether the VVPAT agrees with the voters choices and with the electronic ballots.

13. Security - Logs, Data files

File of vote totals, logs, data files, etc. must be public. Otherwise there is no transparency and no real security. These files must be available electronically and at nominal, if any, cost.

14. Usability - California AVVPAT Standard

California develped a standard for AVVPATs. The June 15, 2004 version is no longer available on the Secretary of State web site. However, it is available via the "WayBack" archives at:

http://web.archive.org/web/20051124102850/http://www.ss.ca.gov/elections/ks_dre_papers/avvpat_standards_6_15a_04.pdf

This includes more details than the Elections Code, e.g. it says in section 2.4.3 that the Paper Record Display Unit must include an audio component.

This standard disappeared from the web without notice. Is that appropriate? Is this still a standard?

15. General Thoughts

I find it ironic that the Government Code, Secretary of State Duties, section 12168.7, wants the Secretary of State to develop standards for a "trusted system" for electronic storage of records:
"trusted system" means a combination of techniques, policies, and procedures for which there is no plausible scenario in which a document retrieved from or reproduced by the system could differ substantially from the document that is originally stored."
The "no plausible scenario" seems a much stronger criterium than that accepted by most for election equipment.

16. General Thoughts

Once a system passes this test, what sort of upgrades will be allowed? Will minor upgrades be allowed without federal testing, or will any upgrades require compliance with the VVSG 2005?