Introduction

Earliest clinical manifestations of Huntington disease (HD) are poorly characterized, and there is a need for clinical scales specifically designed to measure early changes in HD gene expansion carriers. The Functional Rating Scale Taskforce for pre-Huntington Disease (FuRST-pHD) is a multinational, multidisciplinary collaboration to develop a valid functional rating scale to assess changes in symptom severity in HD gene expansion carriers who do not yet meet criteria for a formal clinical diagnosis (prodromal HD or prHD) or are early manifest. [1] Indeed, although formal research diagnosis is currently based on motor impairment, longitudinal data have demonstrated that there are many changes that occur before the presence of motor signs that meet conventional diagnostic criteria for manifest disease, including motor and non-motor signs and symptoms. [2] [3] Furthermore, scales developed to assess signs and symptoms and used to make a formal diagnosis (i.e., UHDRS) [4] were developed in manifest gene expansion carriers and thus may not be sensitive to changes in prodromal HD. [5] [6] Such a measurement tool is essential to better understand the earliest manifestations of HD and to evaluate novel therapies early in the course of disease.

In assessing therapeutic benefit, improvement can occur after treatment with respect to signs and symptoms, functional ability, and quality of life. As outlined by the US Food and Drug Administration (FDA):

“… evidence of improved symptoms alone will only support claims specific to improvement of the symptoms and would not support a general claim related to improvement in a patient’s ability to function or the patient’s psychological state.” [7]

Therefore, functional assessments, particularly in combination with symptomatic assessments, would support broader claims with respect to patient outcomes.

Measures of functional impact assess the impact of disease on the ability of the patient to function normally; changes that should be meaningful/impactful to the patient. Two commonly used measures of function in people with prHD are Total Function Capacity (TFC) [4] and Functional Assessment Scale (FAS). [8] The TFC is a clinician rating scale with questions on occupation, finance, domestic chore, activities of daily living and level of care. The FAS contains more detailed questions on occupation, financies, activities of daily living, domestic chorse, level of care, and physical activities, with binary yes/no responses. Although each assesses important components of day-to-day functioning, in a multinational, multisite study of people with prHD, over 88% of participants scored at ceiling levels of the TFC and FAS. [6] Despite these limitations, occupational decline, and loss of ability to independently manage finances or to drive safely were most commonly noted among people who had not yet been diagnosed with HD. [9] The TRACK-HD study includes the SF36 which assesses physical and social interaction aspects of function. [10] Exploratory qualitative studies with people who have prHD document that functional changes can encompass work, as well as a wide range of daily activities, some of which require social interactions. [11] These findings support the need for measures that are sensitive to changes in daily function in people with prHD and those in the early HD stages.

FuRST-pHD has established an inclusive process for scale development using input from numerous sources, including prHD and early HD individuals, family members, companions, and experts from a variety of fields, as well as from ongoing large-scale HD studies using existing clinical scales. [1] As part of the process, an inclusive series of “Working Groups” of individuals with clinical and/or scale development expertise were formed to review existing data and develop interview questions within the specific domain under study. Once these interview questions are developed, they are distributed to trained raters for beta testing in prHD and early HD individuals. This is an iterative process, in which changes or deletions (as appropriate) are made based on empirical evidence obtained during field testing; the modified questions are then tested during subsequent iterations so that the list can ultimately be winnowed to select optimal items for scale inclusion.

We report here the development and assessment of the first iteration of interview questions aimed to assess day-to-day function in prHD and early HD individual.

Methods

Two-day Working Group meetings were held held in Princeton, NJ (May 4-5, 2010) and New York, NY (August 31-September 1, 2010). The working group charge was to review available evidence and develop interview questions to assess day-to-day functioning prHD and early HD.

Data Analysis

The distribution of the composite score for each individual item was compiled, and summary statistics associated with each item score were computed. Distributions of item scores for HD and prHD subgroups were statistically compared using the non-parametric Mann-Whitney U test.

Non-parametric item response analyses were performed to determine the relationship between scores on the individual interview questions and total score. Item Response Theory has been demonstrated to be useful in evaluating the performance of individual items (symptoms) on rating scales, by assessing the relationship between a score assigned to an item and the overall severity of the disease. [17] [3]IRT software ( TESTGRAF) was used to generate Option Characteristic Curves (OCCs) that display the probability of a particular option score (i.e., a score of 0, 1, 2, 3, 4) on each Interview Question as a function of overall level of severity. In the present analyses, total motor item score was used as a measure of severity. To illustrate this, Figure 1 depicts a hypothetically ‘‘ideal’’ item from an item response perspective, which is characterized by a clear identification of the range of severity scores over which an option is most likely to be endorsed, rapid changes in the curves that correspond to changes in severity, and an orderly relationship between the weight assigned to the option and the region of severity over which an item is likely to be endorsed. As such, OCCs provide a graphical representation of how informative a particular item (or symptom) is as a measure of illness. Frequency distribution of option scoring within each interview question were also generated.

Interview questions which were found to produce scoring and discrimination across ranges of overall severity were selected for further testing. For the subsets of well-endorsed items from each of the two batches of interview questions, total subscale scores for prHD and HD subjects were computed and compared statistically using the Mann-Whitney U test. The measure of internal consistency of each subscale was estimated using Cronbach’s alpha, and the corrected item-total correlation (between each individual item score and the total of the other selected items in the subscale) was computed. Correlations between the total subscale score and scores of individual questions from the same batch that were not selected for further testing were also computed.

Results

A total of 192 CRFs were completed (8-item CRF, N=102; 15-item CRF, N=90). The participant demographic characteristics are shown in Table 2.

Table 2. Demographic Characteristics

Follow-up Working Group meetings were held (via webinar) to review data and make recommendations in moving forward, including item deletion and modification/refinement. The FDA PRO Guidance was used to guide the decision making process. [7]

OCCs and scoring frequency distributions were generated for each of the interview questions. Of the 23 tested, 11 interview questions were found to produce scoring and discrimination across ranges of overall severity:

✓ Work-Impatience

✓ Work-Productivity

✓ Work-Self-Assessment

✓ Work-Initiation

✓ Work-Completion

✓ Household-Productivity

✓ Household-Self-Assessment

✓ Household-Initiation

✓ Household-Completion

✓ Eating/Swallowing

✓ Mobility

With respect to the six well-endorsed items from the first batch of interview questions, Cronbach’s alpha was 0.86 with respect to the entire subset of subjects, 0.91 with respect to the prHD subgroup and 0.81 with respect to the HD subgroup. Among these six items, all corrected item-total correlations were 0.50 or higher with respect to the HD subgroup (with the exception of that for the Mobility item), and all corrected item-total correlations were 0.58 or higher with respect to the prHD subgroup (Table 3). The mean total composite score with respect to the six well-endorsed items from the first batch of interview questions was 3.16 in prHD subjects and 4.36 in HD subjects; the difference in mean scores was not statistically significant ( p = 0.10, Mann-Whitney U test). The only item from the first batch of interview questions for which the mean score was significantly higher in HD than in prHD subjects was the Mobility item ( p = 0.005, Mann-Whitney U test).

With respect to the five well-endorsed items from the second batch of interview questions, Cronbach’s alpha was 0.83 with respect to the entire subset of subjects, 0.82 with respect to the prHD subgroup and 0.84 with respect to the HD subgroup. Among these five items, all corrected item-total correlations were 0.55 or higher with respect to the HD subgroup (with the exception of that for the Eating/Swallowing item), and all corrected item-total correlations were 0.61 or higher with respect to the prHD subgroup (with the exception of that for the Eating/Swallowing item) (Table 4). The mean total composite score with respect to the five well-endorsed items from the second batch of interview questions was 3.41 in prHD subjects and 4.41 in HD subjects; the difference in mean scores approached statistical significance ( p = 0.08, Mann-Whitney U test). The two items from the second batch of interview questions for which the the mean score was significantly higher in HD than in prHD subjects were the Eating/Swallowing item ( p < 0.001, Mann-Whitney U test) and the Household – Completion item ( p = 0.03, Mann-Whitney U test).

Table 3. Correlations between Interview Question Scores (Batch I)

Table 4. Correlations between Interview Question Scores (Batch II)

It was agreed that these 11 interview questions would be modified accordingly and tested in subsequent iterations; examination of the OCCs provided data on which modifications should be made to improve item performance, including changes in wording and scoring options. For example, Figure 2 shows that for “Initiation of Work-related Tasks,” the options with the highest probability of being scored for symptom intensity increased from “0” to “2” (moderate: Can begin tasks but with some difficulty, feel that you had to exert mental effort to get going); however, scores of 3 (severe: Can initiate tasks but only with prompting or cueing) and 4 (Can initiate tasks only with significant external persuasion) were never endorsed as the option with the highest probability, suggesting that assessment of severity is not captured based on comments/prompts received from others. With respect to symptom frequency, these data showed that when problems with initiation of tasks at the workplace do occur they were “much of the time/almost all of the time” and tended not to occur “occasionally” (see Figure 2).

The remaining 12 questions were very rarely endorsed and a score of zero had the highest probability of being scored across most of the range of severity (Figure 3, as example):

✗ Work-Comments

✗ Work-Quality

✗ Household-Quality

✗ Household-Comments

✗ Family Relations

✗ Non-Family Relations

✗ Driving-Limitations

✗ Driving-Confidence

✗ Driving-Comments

✗ Driving-Negative emotion

✗ Finances

✗ Answering phone/door

In general, the low frequency of response and poor discriminative properties limit the usefulness of these interview questions for assessment in prHD and early HD. Mean scores were significantly higher in HD subjects than in prHD subjects for four of the above 12 items (Household-Quality, Driving-Limitations, Driving-Confidence and Driving-Comments); none-the-less, these four items were all poorly endorsed in prHD and HD subjects.

Figure 3 shows the OCCs and frequency distributions for three interview questions that assessed three areas related to driving: “Degree to which the individual limits their driving,” “Degree to which the individual is confident about their driving,” and “Emotional responses to situations or people while driving.” For all of these areas of driving a score of zero had the highest probability of being endorsed across a broad range of severity.

With respect to reporting of negative responses to driving, although scores of zero had the highest probability of being endorsed across the entire range of severity, options 1 (Mild emotional response but no overt expression) and 2 (Some outward expression; brief) were endorsed with some frequency in both prHD and HD (see Frequency distributions, Figure 3). Indeed, this finding is consistent with previous results for interview questions aimed to assess experiences of anger and irritability. [12] However, the correlation between the score for this item and the total composite score from the 5 highly-endorsed items in the second batch of interview questions was low (Table 4), and Cronbach’s alpha would have decreased upon addition of this item to the set of well-endorsed interview questions.

It was agreed that these 12 interview questions should be removed from subsequent iterations on the basis of Relevance (Reported as not relevant by a large segment of the population of interest) and Response Range (A high percentage of patients respond at the floor) as outlined in Table 1 of the FDA PRO Guidance . [7]

Discussion

As part of FuRST-pHD, interview questions are being developed to assess functional ability in prodromal and early HD gene expansion carriers. We report here the development and beta testing of first iteration interview questions designed to assess changes in day-to-day functioning.

Of interest was that higher scores were reported in HD than prHD, including assessments of mobility, eating/swallowing and completion of household chores. These results suggest that these changes are sensitive to disease progression in HD, and may reflect the presence of motor- and/or cognitive-related deficits that can interfere with the ability to perform these tasks. Indeed, ability to detect change and differencesamong groups hypothesized a priori to be different (i.e., known groups validity) are critical measurement properties that will need to be considered for validation and use of the final instrument in clinical trials. [7]

In the development of these interview questions, participants were given the choice to indicate if the the activity was not actually engaged in the preceding two-week period; that is whether the interview question was “not-applicable.” For example, with respect to driving, 10% of prHD and 32% of HD participants reported either “never driving” or “did not drive in the last two weeks.” Similarly, 14% of prHD and 10% of HD participants reported that they did not do their finances. These cases were treated as ‘missing data’ and omitted from the analyses. It is important to recognize, however, that these cases do not represent ‘missing data’ in the usual sense, and are certainly not missing at random. Thus, the data may not represent an unbiased estimate of functional ability. It is also possible that family members or companions may contribute to the decreased activity within these domains because of the companion’s recognition of difficulties, thus modifying participant’s behavior. Although it is not possible to determine precisely the reasons for the lack of the behaviour, it is clear that the percentage participants that reported either “never driving” or “did not drive in the last two weeks” was higher in HD than prHD (32% vs 10%, respectively). This difference may reflect an increase in disability related to progression in HD.

How to incorporate ‘missing data’ for functional items not applicable to all participants in a composite scale score will need to be addressed by the FuRST-pHD. The ‘missing data’ clearly cannot be coded as a score of ‘0’ as this represents lack of disability or low symptom burden in the current scale. Neither is casewise deletion a suitable option. It may be that the scale will be divided into sub-scales, and then the average score of each sub-scale combined to derived a composite index. It is also possible to impute the ‘missing’ value, and there are various methods for doing this. [18] The decision on how to handle the cases with ‘missing data’ will need to be taken prior to finalizing the scale for validation.

Further to the reduced applicability of the driving items to the prHD and particularly early HD group, the low frequency of scoring of these items may reflect a general lack of insight or impaired self-awareness (i.e., anosognosia), [19] [20] or under-reporting due potential implications over licensing. Other factors such as the complexity of the driving task and potential compensation by patients or other drivers around them for any behavioural changes make this less amenable to self-report. Nevertheless, driving changes did arise in the more selective focus group where individuals may have been more willing and open to discussing negative aspects of driving, [11] and there was indeed more signal for the HD group relative to prHD on three of the four items here. In this respect driving remains a potentially important aspect of function but may be better examined through direct assessment of actual or simulated performance of the driving task, or possibly proxy report, rather than self-report.

Based on the results, eleven interview questions have been selected for further testing, have been modified accordingly by the working group, and are currently undergoing a second iteration of field testing. The results of the second iteration will be reported once completed.

Acknowledgments

CHDI Foundation, Inc. – a not-for-profit research organisation whose mission is to rapidly and collaboratively discover and develop therapies that slow the progression of Huntington’s disease – initiated and sponsored the development of the FuRST-pHD. We thank Jamie Levey for her help coordinating the European focus groups, LaVonne Goodman for her help coordinating the USA focus groups, and Stacie M Vik and Barbara McQuaid for administrative assistance.

Competing Interests

The authors have declared that no competing interests exist.