Project Goals

At the outset of the DiSTRIBuTE project, limitations in data sharing were evident in national influenza surveillance and biosurveillance practices, as reflected by position statements from within the public health community [13][17] and Federal legislation and directives [14][15]. Specific to ILI surveillance, limitations included: delays in reporting, high provider drop-out rates, burden on clinical and public health practice, limited flexibility with case definitions, lack of age-specific denominator, and lack of year-round reporting. Specific to biosurveillance practice, limitations included: creation of multiple separate data silos, lack of feedback to original data holders, legal and practical restrictions to sharing personal identifiable information, delays in accessing or disseminating collected data, considerable cost to acquire data, and concern over state and local jurisdictional autonomy. In an effort to address these limitations and needs, the ISDS DiSTRIBuTE project sought to develop a simple, low cost network for sharing aggregate data from ED syndromic surveillance systems that would protect privacy and allow jurisdictional data control.

The primary goals of DiSTRIBuTE were to establish the feasibility of sharing aggregate population health data; and to assess the utility of regional and national sharing of ED syndromic surveillance data for influenza surveillance. These goals were also in line with the Markle Connecting for Health principles [16].

Project Principles

In order to achieve the original goals of the project, a core set of principles were followed. These are summarized in Box 1, and discussed below.

Use Aggregate Data. Collection of aggregated data from health jurisdictions has advantages over individual-level raw data because it can sufficiently represent populations while protecting personally identifiable information. DiSTRIBuTE employed a simple data format that included an agreed upon minimum level of data detail for the epidemiological question and public health action of monitoring febrile, respiratory and ILI syndromes at the population level.

Maintain Jurisdictional Control. Public health departments are more likely to share surveillance data among trusted collaborators in an environment that protects jurisdictional control and where policies are defined in a participatory manner. The intent of this principle was to ensure that both the data framework and the exchange of surveillance findings and interpretation were suitably controlled by the participating jurisdictions.

Minimize Barriers to Participation. The use of a simple aggregate data format minimized one barrier to participation in the project. Participants were initially asked to submit counts of cases measured according to the definitions and standards used in their existing systems. This flexibility lowered the barrier to entry into the project, deferred to state and local authority, and leveraged existing practices with local syndrome definitions and standards. This approach acknowledged the role of local context in extracting public health information from local clinical data. In other words, we assumed that local syndromes were defined based on regional variations in data collection standards, idiom, language, syndrome coding, hospital information systems, as well as other factors, and we wanted to build on this experience and expertise.

Create a Collaborative Network. Data exchange and information sharing in the DiSTRIBuTE project was community-based, and participating jurisdictions were collaboratively involved in the development, specification, and implementation of the system, and in the data analysis and interpretation. The intent of this principle was to ensure that the data exchange framework was based on jurisdictional needs and priorities, and to foster data sharing among trusted collaborators in an environment where they defined the policies and controls in a participatory manner.

Syndrome Definitions

While participating sites generally did not use identical syndrome groupings to monitor influenza-related ED visits in their jurisdictions, the data were requested to be submitted as the preferred local syndrome grouping used by the health departments, with the intent of representing the patterns and trends that the jurisdiction wished to have shown for their region. During the proof of concept period, as sites shared data and observed each other’s trends, there was interest in comparing syndromes based on concepts rather than strictly on defined coding standards [18]. There also was interest in applying more broad and sensitive febrile and respiratory influenza-related groups, and more narrow and specific influenza-like syndromes that were more closely analogous to the traditional clinical surveillance definition of ILI (a presentation with influenza, or fever with cough and/or sore throat) [11][12].

In a pilot comparison of two DiSTRIBuTE jurisdictions, each used the other’s syndrome coding and applied it to their own data to compare the resulting surveillance trends with the syndromes they used locally. The results suggested that using the locally defined syndromes created surveillance time-series that better matched the viral isolate data of confirmed influenza cases locally [19][20]. As the project progressed, participating sites began to reevaluate their own syndrome definitions, and in many cases began to move toward common syndrome concepts through stepwise adaptations of local syndrome definitions, coding, and preferences. As sites looked at each other’s data more, they better understood the heterogeneity of the syndrome groups. And when they assessed their own syndrome characteristics, many reported making minor coding changes that were consistent with making the syndrome concepts and definitions closer and more comparable. This followed a similar process to standardization in other information technology efforts, where standards were aligned with the measurable and practical needs and interests of the users - and standards became “standard” because they became the normal case in the field [21].

During the proof of concept period, work focused on comparison of influenza surveillance systems, evaluation of the model, and harmonization of common ILI syndromes [19][20][22][23]. Similarly, acute gastroenteritis syndrome indicators were implemented in the project, with the goal of being able to monitor winter-seasonal increases believed due primarily to norovirus and rotavirus epidemics across the DiSTRIBuTE network. Implementing and sharing acute gastroenteritis trends also served to assess to what degree the DiSTRIBuTE model could be generalized from monitoring population level ILI morbidity trends to monitoring trends in another constellation of syndrome groups believed closely tied to two recurring winter-seasonal epidemic viral diseases. The implementation of these new groups required participants to adopt or modify syndromic output to include an additional column of data and report historical baselines of the new syndrome groups. Additionally, during the proof of concept period, evaluation efforts and informal ad hoc comparisons were done within the network, notably with respect to an assessment of DiSTRIBuTE data with Google Flu Trends [23] and with the emergence of novel A/H1N1 influenza in the spring of 2009.

Technical Infrastructure

The technical infrastructure to initially support DiSTRIBuTE employed an incremental approach to automating and securing the manual processes initially used in the pilot, and then by early participants. Data were originally exchanged by email attachments using comma separated value (CSV) files in similar formats. An automated system was implemented to permit secure file transmission using the secure shell file transfer protocol (SFTP) and utilizing existing commercial software at the health jurisdiction, or equivalent free software identified by the project. The users sent their data files to a central server where automated software services received the data files, performed basic error detection (including data type errors, simple range checking, and improper coded values), and transformed the data into a common format (allowing for site-specific variation in the composition and layout of the CSV file). The resulting files shared a common syntax that were free of both format and simple content errors. Examples of the latter included violations of simple data type, basic range checking of results, and constrained enumerated values (such as age ranges for age stratification), and were free of format and simple content errors, including violations of simple data type. The standardized files were parsed and their content imported into a simple database, which was implemented using the open source MySQL database package. Open source software was employed for the entire centralized server, using the "Linux, Apache, MySQL, PHP, Perl" development framework.

Visualization of Data

The display of DiSTRIBuTE trends was created as the primary means for sharing and presenting data from participating health jurisdictions. Initially, weekly aggregate ratios of febrile respiratory or influenza-like syndromes, based on each participating site’s routine syndromic criteria for monitoring seasonal influenza, to total ED visits, were visualized as regional weekly time-series (Figure 1), and as age-specific temporal epidemic response surface (TERS) plots (Figure 2). The regional time-series plots presented variation in relative baseline levels during non-influenza periods during the proof of concept period, from lower quartile weekly ratios of 1.4% to over 6% of total ED visits. Peak seasonal epidemic influenza levels by participating health departments, ranged from 6% to over 16%. The variation in non-influenza period baseline levels and peak seasonal epidemic levels indicated that the overall magnitude of the data were not directly comparable. However, the relative pattern of the time-series reported by participating systems were noted as being consistent with regional, state and local influenza surveillance systems and measures [6][7][8][9][10][11].

While participating DiSTRIBuTE sites were not representative of the whole nation on a population basis, taken as an aggregate 13% convenience sample of the U.S., the data for the 2006-2007 and 2007-2008 influenza seasons were highly correlated with national ILI surveillance data [22][23] (Figure 1).

DiSTRIBuTE time-series visualizations by jurisdiction, February 16, 2009.

ILI syndrome time-series are plotted as ratios for DiSTRIBuTE and national CDC sentinel ILI, and as counts by subtype for viral influenza isolate data [12]. DiSTRIBuTE data were typically reported one or more weeks ahead of sentinel reporting data. Pearson correlation between all DiSTRIBuTE sites and CDC ILI for the 2006-2007 and 2007-2008 influenza seasons was 0.96 (p<0.01).

DiSTRIBuTE age-specific visualization, February 16, 2009.

ILI syndrome time-series as age-specific temporal epidemic response surface (TERS) plots for 2006-2009 are shown as relative increase, calculated as observed over the lower-quartile baseline ratio by age-group and jurisdiction [11]. Age-groups are stratified into ranges representing infants and toddlers (age <2 yrs), preschool-age (2-4 yrs), school-age (5-17 yrs), younger adults (18-44 yrs) older adults (45-64 yrs), and senior citizens (age 65+ yrs).

The age-specific visualizations presented participant jurisdiction data as an interpolated surface gradient of the relative magnitude of visits, calculated as observed ratios over lower-quartile baseline, by age-group through time [11]. The plots presented a snapshot of age-specific trends and intensity by jurisdiction, with notable characteristics such as the age-specific timing and relative magnitude of the predominant circulating epidemic viruses in a particular jurisdiction (Figure 2).

National surveillance trends in ILINet from 2008 and later were highly correlated with DiSTRIBuTE data, however ILINet included three DiSTRIBuTE participating health department ED systems during this period. The inclusion of these three state and local syndromic systems represented a large portion (roughly 27%) of the total national ILINet visit volume, and this resulting overlap prevents direct comparison of the systems without disaggregating the regional data. A suitable gold standard for evaluation of population level influenza trends, whether from clinical sentinels or syndromic ED data, is time-series of laboratory confirmed influenza infections. The national viral isolate data, as a proportion of weekly viral tests positive for influenza, were highly correlated with the combined DiSTRIBuTE data (Figure 3).

DiSTRIBuTE and CDC ILINet and viral surveillance data, 2007-2009.

DiSTRIBuTE ILI syndrome time-series are shown as ratios, with national CDC sentinel ILI and viral influenza isolate data [12]. Pearson correlation between all DiSTRIBuTE sites and CDC viral surveillance were significant for the 2007-2008 (0.90, p<0.01) and 2008-2009 (0.86, p<0.01) influenza seasons.

Originally, visualizations and findings were summarized and disseminated via email, on the ISDS website www.syndromic.org, and through meetings, webinars and presentations. Later visualizations were presented on a user website hosted at the University of Washington, where a series of automated queries and statistical and graphing routines were implemented as scripts in the R statistical programming language and run on a daily basis to produce images for both public and restricted-access web applications. The public site showed time series of daily and weekly syndrome ratios. The restricted site was accessible by only those jurisdictions contributing data to DiSTRIBuTE (Figure 4). It included both daily and weekly ratios for several ILI and gastroenteritis syndromes, as well as their constituent data, syndrome counts and visit totals.

As the number of participant sites grew, features were added to enhance the ability to create custom visualizations, as well as to enter descriptive data, or metadata, specific to each site. These data included information needed to manage the creation of combined or composite time series, control the flow of data based on specific terms of participation for particular sites, specify order of appearance for menu items and visualizations, provide denominator information such as estimates of the population covered or number of hospitals in a particular jurisdiction, and contact information for epidemiologists and IT staff with each jurisdiction and site.

National and HHS Regional DiSTRIBuTE time-series, 2007-2011.

Daily trend data are shown for ILI counts, ILI ratios and total ED visits as presented on the ISDS restricted website (Accessed April 6, 2011). The 42 DiSTRIBuTE participating state, county and city jurisdictions are aggregated by federal surveillance region: Region 1, Boston, Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, Vermont; Region 2, New Jersey, New York City, New York State; Region 3, Maryland, Pennsylvania, Virginia; Region 4, Florida, Georgia, North Carolina, South Carolina, Tennessee; Region 5, Cook County Illinois, Indiana, Michigan, Minnesota, Ohio, St. Louis Illinois, Wisconsin; Region 6, Arkansas, North Central Texas, Oklahoma City and Tulsa Counties Oklahoma; Region 7, Missouri, Nebraska; Region 8, Denver Colorado, North Dakota, Utah; Region 9, Arizona, Los Angeles, Napa and San Diego Counties California; Region 10, Portland Oregon, Seattle and King County and Washington State.

The Pandemic Phase

The development and evolution of the DiSTRIBuTE project changed dramatically with the emergence of novel A/H1N1 influenza in April of 2009. With the first influenza pandemic in over 40 years, the demands on local, state, and national public health practice increased. There were notable lessons learned with the DiSTRIBuTE effort during the emergence and progression of the pandemic in the spring of 2009 and with the public health concern over its anticipated recrudescence the following summer and fall - these lessons are presented in the sections below. The DiSTRIBuTE findings and the federal surveillance needs in response to the pandemic [24] led to the implementation and expansion of DiSTRIBuTE nationwide through a collaborative effort with the Centers for Disease the Centers Disease and Prevention (CDC), as part of an existing cooperative agreement with the Public Health Informatics Institute (PHII).

Feasibility of the Data Sharing Model

The DiSTRIBuTE project demonstrated that it was feasible to share population health ILI data nationally in a manner that addresses the needs of local jurisdictions [13][17] and is consistent with the Markle Connecting for Health principles [16]. Although it was clear that submitting aggregate data, as opposed to individual records, encouraged data sharing among participants, it did not altogether eliminate concerns about control and use, particularly with regard to sharing provisional and near real time data.

The principle of keeping barriers to participation low facilitated the growth of the DiSTRIBuTE network. One of the other apparent effects of this principle was that data submitted by different regions often reflected different syndromic definitions, with varying sensitivity and specificity for monitoring conditions of interest. Some epidemiologists and other observers perceived this difference as a limitation to overall data quality, particularly with regard to comparing data .across regions. However, others viewed the heterogeneity as something that was recognized and accounted for by the jurisdictions, where definitions were derived empirically or in response to a local surveillance need. Recognition of these differing perspectives led almost immediately to ongoing collaborative assessment of syndrome definitions and other surveillance practices.

Applying the Model

The success of the DiSTRIBuTE project in demonstrating the feasibility of implementing an innovative low cost model for national syndromic influenza surveillance suggests that it is worth considering expansion of this model to other surveillance activities and to other fields in public health and health care. In DiSTRIBuTE, the data collection process was generalized to acute gastroenteritis syndromes, and potential expansion, for instance, to the monitoring of quality of care would be more ambitious and rewarding. In fact, elements of the DiSTRIBuTE model could conceivably be applied to any system that relies on collaborative networks, rigorous data collection, detailed and comprehensive data, and ongoing technical support.

The lessons regarding standardization of influenza surveillance are also instructive. The issue of standardization remains a major challenge for syndromic surveillance. While relative disease patterns and trends can be compared between systems with heterogeneous syndrome definitions, the ability to compare more direct epidemiological measures is limited. ISDS and collaborators continue to work on strategies to find common ground for syndrome definitions and coding that can inform national agendas for syndromic surveillance.

Providing an emphasis on establishing data transmission while accepting variations in data standards and then allowing the community to move to collaborative harmonization appears to be a viable approach and one that offers an alternative to top-down standards-setting approaches, which can be slow to complete and result in high technical barriers to participation. It also needs to be acknowledged that population surveillance of aggregate data has inherent limitations for queries and analysis at deeper levels of detail.

The question of cost is also important to consider. The DiSTRIBuTE model demonstrated that a national network can be organized to support existing state and local systems, expertise and infrastructure for both public health surveillance and emergency preparedness and response with modest funding (i.e., less than $200,000 per year during the proof of concept period, 2006 to 2009). For the contributing systems, the necessary costs incurred by state and local jurisdictions to support syndromic surveillance were offset in part or in total through federal funding by the CDC through Public Health Emergency Preparedness (PHEP) cooperative agreements, and Public Health Emergency Response (PHER) grants, and through foundation support or direct spending by the jurisdictions themselves. While the total costs required for developing and operating the existing infrastructure of state and local syndromic surveillance through federal PHEP and PHER funding is unknown, federal level efforts to create a national centralized network, such as BioSense, cost roughly $30 million per year during the period 2003 to 2009 [25][26].