Bureau of Communicable Disease New York City Department of Health and Mental Hygiene New York, NY, USA
Bureau of Communicable Disease New York City Department of Health and Mental Hygiene New York, NY, USA
Bureau of Communicable Disease New York City Department of Health and Mental Hygiene New York, NY, USA
Bureau of Communicable Disease New York City Department of Health and Mental Hygiene New York, NY, USA
Bureau of Communicable Disease New York City Department of Health and Mental Hygiene New York, NY, USA
Bureau of Communicable Disease New York City Department of Health and Mental Hygiene New York, NY, USA
Many local and state public health departments have come to rely on syndromic surveillance to monitor and characterize large-scale, seasonal disease trends and have reported it to be especially informative for tracking influenza trends,
The categorization into syndromes of electronic data captured at the time of a health-related encounter (e.g., hospital emergency department (ED) chief complaints, pharmacy medication sales) is the basis of both the simplicity and lack of specificity of syndromic surveillance. Syndrome assignment occurs outside of the health encounter system, most often performed by an automated algorithm at local and state health departments. Syndromic surveillance systems offer a snapshot of care-seeking patterns but do not represent diagnoses, as the syndromes are neither clinically nor laboratory confirmed at the time of analysis. For example, a patient presenting to the ED in New York City with a complaint of fever and cough would be categorized into the influenza-like illness (ILI) syndrome, without information about laboratory test results.
The NYC Department of Health and Mental Hygiene (DOHMH) developed a chief complaint-based emergency department syndromic surveillance system in 2001. Not until the 2009-10 influenza season (as part of enhanced surveillance established after the initial wave of pandemic 2009 H1N1) did DOHMH have the opportunity to evaluate the validity of its influenza-related syndromes. The system regularly monitors six major syndromes, several of which have been shown to correlate well with expected seasonal respiratory and diarrheal disease trends.
· Synchrony of the ILI definition with the Centers for Disease Control and Prevention ILI definition.
· High ecological correlation between citywide ILI syndrome and influenza isolate trends.
· Low baseline ILI levels during non-epidemic periods.
· Simplicity of the definition for communicating information to the public. The respiratory, fever-flu, and ILI syndromes are all monitored, but only ILI syndrome trend data are shared with the public during influenza season.
Because of concerns about a second wave of 2009 H1N1 influenza during the 2009-2010 influenza season, NYC DOHMH selected 5 sentinel hospitals for surveillance of the epidemiology and severity of hospitalized influenza-positive cases in NYC. The DOHMH linked patient ED visits with their laboratory influenza and RSV testing results in order to evaluate the correlation, sensitivity and specificity of the influenza-related syndromes. We share the methods and findings to offer guidance to other jurisdictions that might consider evaluating their syndromes, as well as to describe the implications of monitoring ILI in ED surveillance.
We conducted two main analyses of NYC ED syndromic data and laboratory-confirmed data captured during the 2009-10 influenza season: (1) a citywide, non-matched correlation analysis, and (2) a patient-specific matched analysis of syndromes and laboratory test results from the five sentinel hospitals.
Laboratory-positive influenza and RSV of all NYC residents are reportable to DOHMH. Electronic reports of positive influenza and RSV tests from laboratories are sent via the New York State Electronic Clinical Laboratory Reporting System (ECLRS). In 2008 there were 1436 distinct case reports of influenza; however, in 2009 (because of 2009 H1N1 influenza), 14,523 cases were reported. Laboratory-positive RSV was made reportable in NYC in 2008; in 2009 more than 5,000 cases were reported.
Total ED visits
ILI
Fever/Flu
Respiratory
All Hospitals
1,939,417
52,359 (2.7)
154,339 (8.0)
203,717 (10.5)
5 Sentinel Hospitals
264,532 (13.6)
4,996 (1.9)
24,100 (9.1)
29,859 (11.3)
During the same period, we received confirmatory laboratory reports of 5137 unique cases of influenza and 5030 cases of RSV citywide. The correlation coefficients between the syndromes and positive influenza or RSV results were all high and not appreciably different, as reflected when they are displayed on the same graph with shared scales (Figure 2). Influenza across all ages correlated best with ILI (0.92) and fever/flu (0.91) followed by respiratory (0.84), whereas RSV correlated best with fever/flu (0.82), followed closely by ILI (0.78) and respiratory (0.77) syndromes. Despite similar correlation coefficients throughout the influenza season, the ILI syndrome more closely mirrored the trajectory of influenza cases when the number of cases spiked at the beginning of the season (Figure 3). Even though among the 0-4 age group influenza was equally correlated with ILI and respiratory syndromes (0.92), whereas RSV correlated best with the fever/flu syndrome (0.90) (Figure 4), influenza was driven primarily by ages 5 and over (Figure 5).
At the time of their ED visit, the majority of influenza cases had been categorized in the fever/flu syndrome with only 15% assigned to ILI (Table 2). In total there were 179 ED visits with matching lab reports classified as ILI; 109 (61%) were RSV reports and 70 (39%) were influenza.
Lab-confirmed
Lab-Confirmed Sentinel
ILI‡
Fever/Flu‡
Respiratory‡
Other‡
Influenza (N=655)
466
70 (15)
260 (56)
165 (35)
102 (22)
RSV (N=1348)
1017
109 (11)
426 (42)
539 (53)
187 (18)
Most of the 1992 matches had been categorized as respiratory (47.6%), fever/flu (33.0%) or ILI (9.4%) syndrome. Only 197 (9.9%) of the matched ED visits were categorized in a non-flu-related syndrome; review of these syndromes revealed no particular pattern. There were 167 influenza A positive cases among the 1992 matched cases. The ILI syndrome had the lowest sensitivity but the highest specificity for laboratory confirmed influenza (Figure 6).
Data were further analyzed by month of diagnosis and age (Figure 6). There were no significant differences in trends by county (data not shown). Visits to the ED earlier in the influenza season (November and December) when citywide influenza activity was highest revealed a greater sensitivity and lower specificity of ILI than visits later in the season (February and March) when influenza activity was lower. By age group, ILI was a more sensitive indicator for influenza among the 5-17 and 18-64 age groups. Across syndromes, though, the respiratory syndrome was most sensitive among 18-64 year-olds, whereas fever/flu was a more sensitive measure among 0-4 year-olds. Of note, in the 65 and over age group, ILI was not assigned to any of the 15 patients in our sample who tested positive for Influenza A, resulting in a 0% sensitivity of ILI in this age group. A larger sample would be required to determine a more precise estimate.
The ILI syndrome in the NYC ED syndromic surveillance system served as a specific but not sensitive indicator for influenza during the 2009-2010 influenza season. While the ILI syndrome provided a more precise estimate of the initiation and trends of influenza activity, the syndrome greatly underestimated the magnitude of illness. ILI was most sensitive in the 5-17 and 18-64 age groups. Influenza illness among patients 65 and over was better represented by the respiratory syndrome, whereas patients aged 0-4 were better represented by the fever/flu syndrome. Overall, the sensitivity of ILI was higher in the months of high influenza activity (November and December 2009) as compared to the months of lower activity (February and March 2010).
Despite its limited sensitivity, the ILI syndrome might be more informative for monitoring overall trends in influenza activity because it is less likely to capture cases of other respiratory viruses. Because citywide influenza surveillance usually focuses on activity at the population level, with additional interest in specific age groups or local subgroups (e.g., county), syndromic ED data can provide useful tracking information on the trajectory of an outbreak. Among age subgroups, though, ILI in children 0 to 4 years should be interpreted with caution and alongside laboratory data as both the literature and our data show that the ILI syndrome in this age group reflects both RSV and influenza activity.
There are several limitations to this analysis. First, the majority of all laboratory reports of influenza came from a single sentinel hospital. The overrepresentation of this hospital might have skewed the analysis of sensitivity and specificity among the matches between ED visits and a laboratory diagnosis, which may not accurately reflect citywide trends. The sensitivity and specificity of ILI for four of the five hospitals were within the 95% confidence interval range of the overall ILI results presented in Figure 6, with the exception being one of the smallest hospitals, so the one hospital overrepresented in the sample did not seem to influence these results. In addition, because hospital EDs differ in their methods (e.g., free-text chief complaint vs. pull-down menu) and idiomatic expressions when coding chief complaint, this high-volume hospital could over-represent different illness patterns as compared to the other four hospitals. In addition, a limited number of hospitals in the city perform testing for RSV; one of these hospitals was included in the five sentinel hospitals, therefore weighting the RSV results towards this hospital’s ED practices and patient population. In general, the five sentinel EDs might not be representative of citywide EDs or the NYC population.
Second, the analyses were conducted on data captured during the 2009-10 influenza season during the H1N1 pandemic, an atypical influenza season in terms of the distribution of disease by age group, symptom presentation, and timing of peak activity. Different influenza seasons produce more or less extreme trends depending on circulating strains, susceptible populations and health care access behavior. The season described in this manuscript is unique in that it followed the massive off-season 2009 H1N1 pandemic, which likely resulted in a larger immune population for the regular 2009-10 influenza season. Furthermore there could have been a change in healthcare utilization patterns during the fall/winter as a result of the pandemic outbreak during the spring and summer of 2009. Because syndromic surveillance can only monitor healthcare-seeking behavior of those who visit an emergency room, changes in syndrome patterns from season to season may reflect changes in healthcare behaviors and not necessarily changes in illness spread or severity. Therefore, this analysis may not be generalizable to other influenza seasons when other influenza strains may be circulating. In addition, we were comparing the syndromic data only to influenza and RSV reports and did not account for other respiratory viruses that were likely circulating during the same period, such as rhinovirus. The sensitivity and specificity analysis also has limited generalizability because this analysis was conducted only with data from patients admitted to the hospital rather than both inpatients and outpatients.
Finally, these results are subject to testing biases as well as variability in emergency department utilization by age and socio-economic status. During times of high influenza activity, providers may opt not to test patients for influenza because of the need to make immediate treatment decisions. These untested ED patients may or may not be captured as ILI but would not have been included in this analysis because of the lack of testing. In seasons of relatively mild influenza activity, emergency department activity might be reduced, therefore limiting the ability to monitor ILI trends. Furthermore, because RSV is a disease predominantly recognized in young children, older children and adults are infrequently tested for RSV; similarly, children aged 0-4 years may not be tested for influenza as frequently as they are tested for RSV. Therefore, the true prevalence of RSV in older children and adults and the true prevalence of influenza in young children might be biased by testing behavior. In our matched analysis, 41% of the influenza results and 95% of the RSV results come from the 0-4 year age category. In contrast, the 0-4 year age group made up only 12% of the total ED visit volume during this period and 2.5% of patients admitted to the hospital.
This analysis of matched laboratory test results for specific pathogens with specific ED syndromes has shown that use of the ILI syndrome as a marker for trends in influenza activity is valid in NYC. Syndromic surveillance of ILI, while not a useful method alone for identifying all cases, can provide trend data that can be used to inform public health messages and clinical practice — once ILI activity has risen above a pre-determined threshold, clinicians can be certain to include influenza in their differential diagnoses of patients with flu-like symptoms. As another example, among the 0-4 year age group, the rise of ILI and fever/flu syndromes can be monitored in relation to laboratory reports for influenza and RSV to identify the introduction and amplification of each virus each season.
From an overall public health perspective, the ILI syndrome might not provide added benefit to current surveillance activities beyond the timely identification of the beginning of influenza season. Further enhancements to syndromic surveillance could better describe trends in epidemic infections by taking into account age groups, variable hospital utilization patterns, and perhaps also different hospital practices in coding patients’ reasons for visiting the ED and in laboratory testing. In addition, the introduction of novel viruses or illnesses, or the changing epidemiology of a known seasonal illness, warrants regular monitoring of how well syndromes track with laboratory-confirmed diseases and how changes in sensitivity and specificity of syndromes might affect ongoing surveillance and the ability to estimate influenza burden, both citywide and by subgroups. Given that most laboratories around the country are already routinely reporting cases of influenza (and, in some cases, RSV), jurisdictions who already utilize syndromic surveillance could conduct a similar analysis with minimal additional input. Furthermore, as health care information technology improves, routine linkage of laboratory and syndromic data in real-time might be soon possible, which might add public health value to ongoing surveillance activities.
This analysis highlights the need to further understand hospital utilization patterns as well as hospital-specific systems and practices to better interpret syndromic data. Although the ILI syndrome may not serve as the most sensitive measure of influenza activity, ILI remains a valuable tool for monitoring influenza activity and trends as it facilitates comparisons nationally and across jurisdictions and is easily communicated to the public.
The authors would like to thank all those who contributed to the quality of the data used in this analysis including Deborah Kapell, Ana Maria Fireteanu, Brooke Bregman, and Nimi Kadar. The authors also thank the following for their insight into and support of the project: Marci Layton and Scott Harper. Lastly, this project would not have been possible without the dedication of the staff at the Public Health Laboratory and the sentinel hospitals as well as the hard working data-entry staff at the Bureau of Communicable Disease.