Frequently Asked Questions about the ORCA Measure

The Observer-Reported Communication Ability (ORCA) measure assesses the communication ability of an individual from the perspective of their primary caregiver. The measure is applicable for individuals with a neurodevelopmental disorder associated with a significant impact on communication ability and typically non-verbal. It was designed for use in research studies to assess an individual's communication ability and their changes over time if repeated assessments are collected.

The ORCA measure captures communication ability observed by the caregiver over the past 30 days. Items cover important communication behaviors in numerous areas within expressive, receptive, and pragmatic (social) communication.

The measure consists of 81 total questions with 70 behavioral items within communication concepts that cover expressive, receptive and pragmatic areas of communication (see Table 1), alongside a set of 11 descriptive items that capture important information about the individual’s unique ways of communicating (e.g. modalities the individual uses, their current vocabulary, and language complexity). The ORCA measure is modality agnostic, meaning that a child could score high or low on the ORCA measure regardless of whether they use gestures/signs, vocalizations, sounds, and/or assistive technology to communicate.

_{Table 1: Communication concepts included in the ORCA measure, as informed by concept elicitation interviews, cognitive testing interviews, and psychometric testing with caregivers}

^{*The concept “telling about the past” is currently included in the ORCA measure but not a part of the scoring metric at this time.}

The ORCA measure was initially designed to assess the communication ability of an individual with Angelman syndrome (AS, of any genotype), aged 2 years old or older (including adults). The content included in the ORCA measure was informed by interviews with family caregivers of individuals with AS and by communication specialists (e.g., speech-language pathologists). Subsequently, the ORCA measure has been validated with individuals with Rett syndrome. There is ongoing work with the U.S. Food and Drug Administration (FDA) to expand the ORCA measure to other neurodevelopmental disorders (more information here).

There are currently two versions available for use. The original version was designed specifically for Angelman syndrome and is labeled “ORCA-1.0”. A slightly modified version is available for Rett syndrome and is labeled “ORCA-2.0”. The ORCA-2.0  measure includes the same questions and uses the same scoring algorithms. However, some instructions were modified to accommodate more gesture use for individuals with Rett syndrome.

Although the ORCA measure has not been currently validated in populations other than Angelman syndrome (AS) and Rett syndrome (RTT), the investigators are excited about the possibility of applying it to other similar syndromes or disorders. The ORCA measure is likely most appropriate for neurodevelopmental disorders that have similar communication barriers to AS: most individuals are non-verbal or communicate primarily through modalities other than spoken words, they often have stronger pragmatic and receptive communication skills compared to expressive skills, communication develops heterogeneously, and ‘mastery’ of communication behaviors is shown when they are observed frequently and consistently.

If you are interested in using the ORCA measure in a population other than AS or RTT, please visit our guidelines for more information about using ORCA in other populations. If you’d like to discuss a potential partnership with Duke University to collect additional validity evidence, please contact Nicole Lucas (nicole.lucas@duke.edu).

It is recommended that the individual’s primary caregiver complete the ORCA measure, as this person will have the most familiarity with the individual’s typical communication across a number of different situations and contexts. If you are measuring changes in communication ability over time, it is recommended that the same caregiver complete the ORCA measure at all time points. Caregivers can complete the ORCA measure independently in their home or a clinic setting.

The ORCA measure can be used cross-sectionally (at one time point) or at multiple time points (e.g., baseline and post-intervention) to assess score change in communication ability. Note that the reference/recall period is the past 30 days.

The ORCA Measure is copyrighted by Duke University. Pattern Health is the exclusive authorized distributor of the ORCA Measure. If you are interested in obtaining a license to use the ORCA measure, please fill out a license request form. There is no cost for noncommercial use. There are fees associated with commercial use. The investigators ask that in any publication or presentation, you cite the original manuscripts

The ORCA measure was developed using best practice guidelines from the FDA, ISPOR, ISOQOL, PROMIS, and other organizations. The process centered on input and guidance from caregivers of individuals with AS at all stages and communication experts, including speech-language pathologists. Although validation studies are always ongoing, the following steps have been taken and provide support for the validity and reliability of the ORCA measure:

1. Concept elicitation interviews
2. Item generation
3. Cognitive testing
4. Language Translatability Review
5. Psychometric testing (cross-sectional and longitudinal)

On average, an individual can complete the ORCA measure in about 15 minutes.

No. The ORCA measure was designed for caregivers to complete the measure independently (electronically or on paper), without the need for a trained administrator. In other words, caregivers should be encouraged to read the instructions independently and complete the ORCA measure on their own. Assistance should only be given if the caregiver cannot read or has difficulty using an electronic survey system.

The ORCA measure was specifically designed for clinical trials and research settings to identify differences between two or more groups or evaluate score changes in communication ability over time within the same group(s). Although clinicians may be interested in the ORCA measure to direct therapy, its appropriateness for use in clinical settings has not yet been examined.

The ORCA measure was designed to be administered either electronically (through a web-based survey platform) or by paper & pencil. Note that the original psychometric validation study used only electronic versions of the ORCA measure. At this time, equivalence has not been established between paper-and-pencil and electronic assessment of the ORCA measure. However, there have been numerous studies that have shown negligible differences between paper and electronic clinical outcome assessments (Gwaltney, Shields, & Shiffman, 2008)¹.

The developers recommend that the ORCA measure be completed on a large-screen device, such as a laptop, tablet, or iPad, to have a full view of the questions.

_{¹ Gwaltney, C.J., Shields, A.L., & Shiffman, S. (2008). Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: A meta-analytic review. Value in Health; 11(2), 322-333. DOI: 10.1111/j.1524-4733.2007.00231.x}

The ORCA T-scores were standardized based on the original calibration sample of individuals with AS with a mean of 50 and a standard deviation of 10. Higher ORCA Tscores reflect greater communication ability. There are different scoring algorithms available based on the needs of the investigator/researcher. The ORCA scoring algorithms provide both an overall   communication score and scores for domains of expressive, receptive, and pragmatic communication. In addition, researchers have the choice to only score behaviors that are consistently observed by caregivers over the past 30 days (“Mastery” scoring algorithms) or to score behaviors that are both emerging (sometimes occur) or consistent over the past 30 days (“Emerging & Mastery” scoring algorithms).

There is an online, automated scoring service available at no additional cost to licensed ORCA users on the PatternHealth website.

Alternatively, you may use the SAS macros or R scoring program files provided with the measure to obtain scores. These macros and files are also available at no additional cost to licensed ORCA users on the PatternHealth website.

Scoring the ORCA measure is complex due to the double hierarchy (items are leveled within communication concepts, and concepts are leveled within the larger concept of “communication ability”). The scoring algorithms will score response patterns using an IRT-based model to reflect the caregiver’s observations of an individual’s communication ability over the past 30 days via reported mastery or mastery-emerging of specific behaviors.

Currently, the ORCA measure is available in English with language translations in:

The development of the ORCA-1.0 measure was financially supported by the Foundation for Angelman Syndrome Therapeutics (FAST). The modified ORCA-2.0 measure version was supported by a contract from the Rett Syndrome  Research Trust.

The following individuals developed ORCA measure version 1.0 and version 2.0: Bryce Reeve, Nicole Lucas, Molly McFatrich, Li Lin, Christina Zigler, Kelly Gordon, and Harrison Jones . If the technology (the ORCA measure, V1.0 and V2.0) is commercially successful in the future, the developers and Duke University may benefit financially.

The ORCA measure is copyright Duke University.

Pattern Health is the exclusive authorized distributor of the ORCA measure. A license request form will need to be completed and submitted to Pattern Health. Within the license request form, you can indicate which translations of the ORCA measure you are interested in using.

Yes, you can review a sample copy of either the ORCA-1.0 or ORCA-2.0 measure by first submitting a license request form through Pattern Health.

No, you may not alter, modify, or otherwise create derivative works (including translations) without permission from the developers.

The developers ask that in any publication or presentation, you cite the original manuscripts:

For ORCA-1.0 (for Angelman Syndrome): 

1. Zigler CK et al. Exploring communication ability in individuals with Angelman syndrome: findings from qualitative interviews with caregivers. American Journal on Intellectual and Developmental Disabilities 2023;128(3):185–203, https://doi.org/10.1352/1944-7558-128.3.185
2. Zigler CK et al. Validation of the Observer-Reported Communication Ability (ORCA) Measure for individuals with Angelman syndrome. American Journal on Intellectual and Developmental Disabilities 2023;128(3):204-218.
   https://doi:10.1352/1944-7558-128.3.204

For ORCA-2.0 (for Rett Syndrome):

1. Reeve BB, et al. Validation of the Observer-Reported Communication Ability (ORCA) measure for individuals with Rett syndrome. European Journal of Paediatric Neurology. 2023;46:74-81.
   https://doi.org/10.1016/j.ejpn.2023.07.007