Forecasting Critical Violations

A key finding of any food inspection is whether a food establishment commits a critical violation. These violations, which normally pertain to improper temperature control for food, are most susceptible to accommodating the start or spread of food borne illnesses. Just one critical violation will result in a failure, and must be remedied by the establishment to the satisfaction of the Department of Public Health.

Locating restaurants with critical issues is an important priority for the Department of Public Health. Approximately fifteen percent of inspections result in at least one critical violation. Given the large number of inspections that inspectors have to complete, the time and effort it takes to discover critical violations can mean prolonged exposure to potential disease, illness, and unsanitary conditions at some food establishments.

Predictors of food Inspection outcomes

Establishments that had previous critical or serious violations

Three-day average high temperature

Nearby garbage and sanitation complaints

The type of facility being inspected

Nearby burglaries

Whether the establishment has a tobacco license or has an incidental alcohol consumption license

Length of time since last inspection

The length of time the establishment has been operating

Inspector assigned

Open Data and Analytics

The Department of Public Health and Department of Innovation and Technology have partnered to explore a combination of datasets to prioritize which establishments are more likely to yield a critical violation during an inspection. Staff from Allstate Insurance have also assisted with the research project.

After conducting interviews with the Department of Public Health's food inspection team, several data sources—ranging from 311 data to food inspections and weather—were explored. A dozen variables had substantial relationships with the likelihood of an establishment failing a food inspection.

Information about the food establishment, such as its CDPH-assigned risk level and whether the establishment had failed previous inspections, served as important predictors. Information about the establishment's community, such as its location and nearby sanitation complaints made through 311, was also related to the most severe violations.

When factoring all of these items together, the research team was able to provide a likelihood of critical violations for each establishment, which was developed to prioritize which ones should be inspected first.

Collaboration was a key component of this project, with researchers at the Department of Innovation and Technology, Department of Public Health, Allstate Insurance Company, and Civic Consulting Alliance all working together. Each variable used in the model, with the exception of widely available weather and inspector data, was available on Chicago's open data portal. The portal provides access to hundreds of datasets, which have been used to improve transparency, allow application development, and catalyze commercial activity for start-ups and large enterprises.

Looksy

The portal was an effective tool to allow for such collaborative research. This project was able to leverage Chicago's key data assets: its large volume of data, the transparency and size of its open data portal, and its ability and willingness to conduct research to improve city services, introduce savings, and increase engagement with Chicago-area businesses.

Alt
Almost 16 percent of establishments visited had at least one critical violation

Evaluating the Effectiveness of the Model

The analytical model was tested using a double-blind retrodiction. The Department of Public Health conducted inspections via its normal operational procedure during September and October of 2014. During these two months, food establishments were visited in their normal order.

During this time, the Department of Public Health visited 1,637 food establishments. Almost 16 percent of them—258 establishments—yielded at least one critical violation during the experiment. Over half, 55 percent or 141 establishments, were found during the first month of the evaluation, whereas 117 establishments (45 percent) were found during the second half.

After all of the inspections were completed, the Department of Innovation and Technology used data to estimate the likelihood of each establishments having a critical violation. Researchers applied a probability to each establishment using historical data, and then investigated if these probabilities could be used to make the inspection process more efficient.

Percentage of critical violations found in the first half of work

Percentage of critical violations found in the first half of work

Standard Workflow: 55%
Data-driven Workflow: 70%

The simulation would show if riskier establishments would be inspected first. Researchers found that food inspectors could be allocated more efficiently using the computer algorithm. During the simulation, 69 percent of inspections—178 establishments—with critical violations were found during the first half of work, compared to 55 percent during normal operations. Over the two month pilot, establishments with violations were found, on average, 7 and a half days earlier. That is, an additional 37 establishments would have been cited for violations in the first month, as opposed to being discovered later, potentially after patrons became ill.

Open Source Solutions

Food inspection forecasting is also made available as an open source project. An open source approach helps build a foundation for other models attempting to forecast violations at food establishments. The analytic code is written in R, an open source, widely-known programming language for statisticians. There is no need for expensive software licenses to view and run this code.

  • Chicago Health Department

    Our Vision: A city of thriving communities where all residents are able to live healthy lives. Our Mission: To promote and improve health by engaging residents, communities, and partners in establishing and implementing policies and services that prioritize residents and communities with the greatest need.

  • Illinois Department of Public Health

    The Department of Innovation and Technology (DoIT) was introduced in 2008 to add innovation to the charter of the former Department of Business and Information Services. As the central information technology organization for the City, DoIT provides a number of technology and telecommunications services to departments, the Mayor, Aldermen, other city agencies, residents, businesses and tourists.