The ORCA measure (V1.0) was designed and evaluated in a sample of individuals with Angelman syndrome (AS). Subsequently, the ORCA measure was also refined and evaluated for Rett syndrome (RTT), V2.0. Thus, for populations other than AS and RTT, the ORCA measure would be considered an exploratory measure and could not serve as the primary outcome measure until additional validity evidence is gathered in your population and context of use.
Ideally, validity data would be compiled via detailed, prospective qualitative and quantitative studies. However, integrating the ORCA measure into existing studies can provide initial quantitative data on its reliability and validity (even in the absence of qualitative data) *.
*Note that positive findings from a quantitative study do not substitute for completing a qualitative-type study to evaluate the content validity of the ORCA measure. Please reach out to the investigators at Duke University if you would like to add a qualitative component (concept elicitation and/or cognitive testing).
There are a few considerations for including the ORCA measure (V1.0 or V2.0) in an ongoing or upcoming research study:
- Prior to integrating the measure into your study, the investigators recommend you and parents/caregivers within your population review the ORCA measure to get a sense of how well the ORCA measure items capture communication behaviors observed in your population.
- If you need a copy of the ORCA measure, please fill out a license request form.
- Note that there is no cost for noncommercial use of the ORCA measure. There are fees associated with commercial use.
- Consider additional variables to collect & specify your data collection schedule. The investigators can provide guidance on ways to evaluate/validate the ORCA measure for your population within the bounds of your existing study. For research studies collecting quantitative data, there are several psychometric properties one can evaluate (depending on your sample size), including but not limited to:
- Internal properties of the ORCA measure:
- Structural Validity: this involves a confirmatory factor analysis (CFA) to confirm the factor structure.
- Internal Consistency Reliability: estimating Cronbach’s coefficient alpha or ordinal alpha.
- Test-Retest Reliability: administering the ORCA measure twice within a short timeframe (typically 5-14 days) with a group of “stable” individuals (when changes in communication ability would not be expected) to examine reproducibility of scores over time.
- Ceiling/Floor Effects: this gives us a sense if the ORCA measure is capturing the entire range of communication ability seen in your target population.
- External properties of the ORCA measure:
- Convergent Validity: how does the ORCA measure relate to other established measures of communication (e.g., an SLP-administered communication measure)?
- Discriminant Validity: how does the ORCA measure relate to established measures of non-communication concepts (e.g., cognitive or physical function measures)?
- Known Groups Validity: how does the ORCA measure differentiate between sub-groups of individuals “known” to have different levels of communication abilities
- Responsiveness Over Time: are there statistically significant differences in scores over time when there are “true” changes in communication abilities?
- Internal properties of the ORCA measure:
- Consider the e-platform for administering the ORCA measure:
- Consider using the Pattern Health platform which includes the ORCA measure (V1.0 and V2.0) in its library. Please note there is a fee associated with using the Pattern Health platform.
- If you are using REDCap, the investigators at Duke University can provide a REDCap data dictionary and you can plug it into your REDCap database.
- If you are using a different electronic data collection system, you will need to program the ORCA measure into your system. Multiple people should test programming to make sure there are no mistakes (e.g., spelling errors, missing questions), the skip logic works, etc. In addition, those migrating the ORCA measure for eCOA administration should consult the ORCA measure eCOA Implementation Guide for best practices. To request a copy of the eCOA Implementation Guide, please contact Pattern Health.
- Please note that Duke University owns all IP for the ORCA measure and its derivatives, including any e-platform version of the ORCA measure. Thus, the ORCA measure can only be used for the intended study and cannot be made available for other studies (without permission from Duke). Only Duke University or Pattern Health may serve as a distributor of the ORCA measure.
- Set up scoring for the ORCA measure:
- Scoring of the ORCA measure is automatic in the Pattern Health platform.
- Alternatively, you may use the SAS macros or R scoring program files provided with the measure to obtain scores. These macros and files are also available at no additional cost to licensed ORCA users on the Pattern Health website.
- There is an online, automated scoring service available at no additional cost to licensed ORCA users on the Pattern Health website.
- Make a Data Sharing plan:
- The investigators at Duke University are very open to receiving/sharing data. All data should be cleaned, formatted, and de-identified (e.g., all identifying information must be removed) before being sent to the investigators.
- Please consider adding language in your consent form about how deidentified data will be shared with external partners for research purposes. Investigators at Duke University will need to get IRB approval if they conduct any analyses with the data or participate in any resulting manuscripts.
- The investigators at Duke University are very open to receiving/sharing data. All data should be cleaned, formatted, and de-identified (e.g., all identifying information must be removed) before being sent to the investigators.
- Consider supporting Duke’s involvement financially:
- The investigators at Duke University are experts at developing outcome measures and designed the ORCA measure using a rigorous process. The investigators are excited and willing to help other groups ensure the ORCA measure is reliable and valid in their population(s). Currently, the investigators do not have any external funding mechanisms to support their effort in assisting other groups gather, analyze, and process their data. The investigators are open to contracts or other funding mechanisms that would support a collaborative partnership with your organization.