What is the ORCA measure?
The Observer-Reported Communication Ability (ORCA) measure assesses the communication ability of an individual from the perspective of their primary caregiver. The measure is applicable for individuals with a neurodevelopmental disorder associated with a significant impact on communication ability and typically non-verbal. It was designed for use in research studies to assess an individual's communication ability and their changes over time if repeated assessments are collected.
What content is included in the ORCA measure?
The ORCA measure captures communication ability observed by the caregiver over the past 30 days. Items cover important communication behaviors in numerous areas within expressive, receptive, and pragmatic (social) communication.
The measure consists of 81 total questions with 70 behavioral items within communication concepts that cover expressive, receptive and pragmatic areas of communication (see Table 1), alongside a set of 11 descriptive items that capture important information about the individual’s unique ways of communicating (e.g. modalities the individual uses, their current vocabulary, and language complexity). The ORCA measure is modality agnostic, meaning that a child could score high or low on the ORCA measure regardless of whether they use gestures/signs, vocalizations, sounds, and/or assistive technology to communicate.
Table 1: Communication concepts included in the ORCA measure, as informed by concept elicitation interviews, cognitive testing interviews, and psychometric testing with caregivers
| Expressive Communication | Receptive Communication | Pragmatic Communication | Vocabulary |
|---|---|---|---|
| Seek Attention | Respond to Name | Greeting | Number of gestures/signs |
| Direct Attention | Understand Mood | Comfort Others | Number of sounds |
| Refuse Object | Understand Isolated Words | Play Games | Number of verbal words |
| Request Object | Turns in Conversation | Use Names | Number of symbols on an assistive device |
| Request Object Out of View | Make Choices | ||
| Request “More” | Respond to Familiar Directions | ||
| Communicate Understanding | Respond to New Directions | ||
| Asking Questions | Answer Questions | ||
| Communicate with Others | |||
| Telling About the Past* |
*The concept “telling about the past” is currently included in the ORCA measure but not a part of the scoring metric at this time.
What populations were recruited to design and evaluate the ORCA measure?
The ORCA measure was initially designed to assess the communication ability of an individual with Angelman syndrome (AS, of any genotype), aged 2 years old or older (including adults). The content included in the ORCA measure was informed by interviews with family caregivers of individuals with AS and by communication specialists (e.g., speech-language pathologists). Subsequently, the ORCA measure has been validated with individuals with Rett syndrome. There is ongoing work with the U.S. Food and Drug Administration (FDA) to expand the ORCA measure to other neurodevelopmental disorders (more information here).
What versions of the ORCA measure are available?
There are currently two versions available for use. The original version was designed specifically for Angelman syndrome and is labeled “ORCA-1.0”. A slightly modified version is available for Rett syndrome and is labeled “ORCA-2.0”. The ORCA-2.0 measure includes the same questions and uses the same scoring algorithms. However, some instructions were modified to accommodate more gesture use for individuals with Rett syndrome.
I work with individuals who have neurodevelopmental disorders that impact communication ability, but not Angelman syndrome or Rett syndrome. Can I use the ORCA measure with my population?
Although the ORCA measure has not been currently validated in populations other than Angelman syndrome (AS) and Rett syndrome (RTT), the investigators are excited about the possibility of applying it to other similar syndromes or disorders. The ORCA measure is likely most appropriate for neurodevelopmental disorders that have similar communication barriers to AS: most individuals are non-verbal or communicate primarily through modalities other than spoken words, they often have stronger pragmatic and receptive communication skills compared to expressive skills, communication develops heterogeneously, and ‘mastery’ of communication behaviors is shown when they are observed frequently and consistently.
If you are interested in using the ORCA measure in a population other than AS or RTT, please click here for more information and recommendations. If you’d like to discuss a potential partnership with Duke University to collect additional validity evidence, please contact Nicole Lucas (nicole.lucas@duke.edu).
Who should complete the ORCA measure?
It is recommended that the individual’s primary caregiver complete the ORCA measure, as this person will have the most familiarity with the individual’s typical communication across a number of different situations and contexts. If you are measuring changes in communication ability over time, it is recommended that the same caregiver complete the ORCA measure at all time points. Caregivers can complete the ORCA measure independently in their home or a clinic setting.
How should I use the ORCA measure?
The ORCA measure can be used cross-sectionally (at one time point) or at multiple time points (e.g., baseline and post-intervention) to assess score change in communication ability. Note that the reference/recall period is the past 30 days.
Do I need to pay for the ORCA measure?
The ORCA Measure is copyrighted by Duke University. Pattern Health is the exclusive authorized distributor of the ORCA Measure. If you are interested in obtaining a license to use the ORCA measure, please fill out a license request form. There is no cost for noncommercial use. There are fees associated with commercial use. You may not alter, modify, or otherwise create derivative works (including translations) without permission. The investigators ask that in any publication or presentation, you cite the original manuscripts:
- Zigler CK et al. Exploring communication ability in individuals with Angelman syndrome: findings from qualitative interviews with caregivers. American Journal on Intellectual and Developmental Disabilities 2023;128(3):185–203, https://doi.org/10.1352/1944-7558-128.3.185
- Zigler CK et al. Validation of the Observer-Reported Communication Ability (ORCA) Measure for individuals with Angelman syndrome. American Journal on Intellectual and Developmental Disabilities 2023;128(3):204-218.
https://doi:10.1352/1944-7558-128.3.204 - Reeve BB, et al. Validation of the Observer-Reported Communication Ability (ORCA) measure for individuals with Rett syndrome. European Journal of Paediatric Neurology. 2023;46:74-81.
https://doi.org/10.1016/j.ejpn.2023.07.007
How was the ORCA measure developed?
The ORCA measure was developed using best practice guidelines from the FDA, ISPOR, ISOQOL, PROMIS, and other organizations. The process centered on input and guidance from caregivers of individuals with AS at all stages and communication experts, including speech-language pathologists. Although validation studies are always ongoing, the following steps have been taken and provide support for the validity and reliability of the ORCA measure:
- Concept elicitation interviews
- Item generation
- Cognitive testing
- Language Translatability Review
- Psychometric testing (cross-sectional and longitudinal)
How long does it take someone to complete the ORCA measure?
On average, an individual can complete the ORCA measure in about 15 minutes.
Do I need to train anyone to administer the ORCA measure to caregivers?
No. The ORCA measure was designed for caregivers to complete the measure independently (electronically or on paper), without the need for a trained administrator. In other words, caregivers should be encouraged to read the instructions independently and complete the ORCA measure on their own. Assistance should only be given if the caregiver cannot read or has difficulty using an electronic survey system.
In what settings can I use the ORCA measure?
The ORCA measure was specifically designed for clinical trials and research settings to identify differences between two or more groups or evaluate score changes in communication ability over time within the same group(s). Although clinicians may be interested in the ORCA measure to direct therapy, its appropriateness for use in clinical settings has not yet been examined.
Do I need to print out paper copies of the ORCA measure to administer it?
The ORCA measure was designed to be administered either electronically (through a web-based survey platform) or by paper & pencil. Note that the original psychometric validation study used only electronic versions of the ORCA measure. At this time, equivalence has not been established between paper-and-pencil and electronic assessment of the ORCA measure. However, there have been numerous studies that have shown negligible differences between paper and electronic clinical outcome assessments (Gwaltney, Shields, & Shiffman, 2008)1.
The developers recommend that the ORCA measure be completed on a large-screen device, such as a laptop, tablet, or iPad, to have a full view of the questions.
1 Gwaltney, C.J., Shields, A.L., & Shiffman, S. (2008). Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: A meta-analytic review. Value in Health; 11(2), 322-333. DOI: 10.1111/j.1524-4733.2007.00231.x
What do scores on the ORCA measure mean?
The ORCA T-scores were standardized based on the original calibration sample of individuals with AS with a mean of 50 and a standard deviation of 10. Higher ORCA Tscores reflect greater communication ability. There are different scoring algorithms available based on the needs of the investigator/researcher. The ORCA scoring algorithms provide both an overall communication score and scores for domains of expressive, receptive, and pragmatic communication. In addition, researchers have the choice to only score behaviors that are consistently observed by caregivers over the past 30 days (“Mastery” scoring algorithms) or to score behaviors that are both emerging (sometimes occur) or consistent over the past 30 days (“Emerging & Mastery” scoring algorithms).
How do I score the ORCA measure?
There is an online, automated scoring service available at no additional cost to licensed ORCA users on the PatternHealth website (see here).
Alternatively, you may use the SAS macros or R scoring program files provided with the measure to obtain scores. These macros and files are also available at no additional cost to licensed ORCA users on the PatternHealth website (see here).
Scoring the ORCA measure is complex due to the double hierarchy (items are leveled within communication concepts, and concepts are leveled within the larger concept of “communication ability”). The scoring algorithms will score response patterns using an IRT-based model to reflect the caregiver’s observations of an individual’s communication ability over the past 30 days via reported mastery or mastery-emerging of specific behaviors.
Is the ORCA measure available in other languages?
Currently, the ORCA measure is available in English with language translations in:
| Arabic Israel | French France | Portuguese Portugal |
| Dutch Netherlands | German Germany | Russian |
| English Australia | Hebrew Israel | Simplified Chinese China |
| English Canadian | Italian | Spanish Spain |
| English UK | Japanese Japan | Spanish USA |
| French Canada | Polish Poland | Turkish Turkey |
The investigators are open to collaborating with other interested parties to culturally validate and translate the ORCA measure into other languages.
If you are interested in translating the ORCA measure, please contact the investigators (see contact information below). If you would like to obtain a copy of one of the translations of the ORCA measure, please fill out a license request form.
Who funded the development of the ORCA measure?
The development of the ORCA-1.0 measure was financially supported by the Foundation for Angelman Syndrome Therapeutics (FAST). The modified ORCA-2.0 measure version was supported by a contract from the Rett Syndrome Research Trust.