> For the complete documentation index, see [llms.txt](https://dnanexus.gitbook.io/ofh/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://dnanexus.gitbook.io/ofh/airlock/safe-outputs-data-guidelines-for-export.md).

# Safe Outputs: Data Guidelines for Export

Learn about Safe Outputs and how you can make your results safe for release.

## How to use this document

This guide is to help you:

* learn about safe outputs&#x20;
* make your results safe for release

## What Does Safe Output Mean?

We review files before they are exported from the Trusted Research Environment (TRE) to help protect participant confidentiality.

Releasing statistical information always carries some risk that individuals could be identified. Statistical Disclosure Control (SDC) is the process used to assess and reduce these risks before outputs are released from the TRE.

You should consider disclosure risks and apply appropriate SDC measures while designing analyses and preparing outputs for export.

## Principles

We use a combination of SDC rules and principles when reviewing outputs for export. The rules provide practical guidance, while the principles help us assess more complex or unusual cases.

Our SDC principles are below.

* SDC should be applied to all outputs intended for release through the Airlock.
* SDC should consider both direct (primary) disclosure risks and risks that could arise when multiple outputs are combined (secondary disclosure).
* Researchers should use SDC methods recommended by Our Future Health or other trusted scientific or research organisations.
* Disclosure risks should be considered throughout the design, analysis, and preparation of outputs for export.
* Researchers are responsible for ensuring that exported outputs do not contain disclosive information and that appropriate safeguards have been applied.
* Descriptions of SDC methods should not include enough detail to allow the controls to be reversed.

If you believe standard SDC rules cannot reasonably be applied to your outputs, please contact the Airlock team before submitting your request.

## Data guidelines (SDC rules)

### Participant or event level data

| **Concerns and what not to export**                           | **Potential solutions (can export)**                                                                                                        | **Minimum requirement for release**   |
| ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- |
| Almost always disclosive. May reveal identifiable information | Not permitted except in exceptional cases with strong justification, additional safeguards and where they can be shown to be non-disclosive | Not applicable (aggregation required) |

### Counts or frequency tables

| **Concerns and what not to export**                                                                                                                                                                                                                                                                                                                                                                                                                                                  | **Potential solutions (can export)**                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | **Minimum requirement for release**                                                                                                                                                                                                           |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| <p>Counts or frequencies based on small groups or rare combinations of attributes (<10 observations) cannot be exported. </p><p>Frequencies or counts relating to all individuals in the study population (group disclosure) or all but one individual (within group disclosure) should be avoided by restructuring the table.</p><p>Dominance, where a single observation contributes more than 50% of the cell value, should be avoided by restructuring the table.</p><p><br></p> | <p>Counts of 0 can be exported unless they relate to zero or all observations (see next point on group disclosure). </p><p>Suppression: replace cell values with “<10”, “redacted” or a similar label. Care should be taken to make sure that redacted values cannot be inferred from row or column totals or based on other variables. As a rule, if the row or column total is released, the second smallest group should also be redacted. </p><p>Modifying attribute values or combining categories: rather than give exact values of statistical results, values can be placed in sufficiently wide bands or groups (e.g., 0-10%, 11-20%, etc.).</p> | <p>Number of observations</p><p>Frequencies can only be exported if the associated number of observations are provided alongside. </p><p>Each reported group is ≥10</p><p>Totals must not allow inference of suppressed values</p><p><br></p> |

### Descriptive or summary statistics

These include, for example:

* mean
* median
* mode
* standard deviation
* quartiles or percentiles
* confidence intervals

| **Concerns and what not to export**                                                                                                                                                                                                                                                                                                                                                                                                         | **Potential solutions (can export)**                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | **Minimum requirement for release**                                                                                                                                 |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| <p>Summary statistics calculated from fewer than 10 observations cannot be exported as they are potentially disclosive.</p><p>Medians, quartiles or percentiles, which are often a single observation, or the mode, which can represent a small number of observations, can present higher risks of attribution to single individuals. These statistics cannot be exported if they correspond to observations from a small group (<10).</p> | <p>Suppression: replace cell values with “redacted” or a similar label. </p><p>Combining groups or categories: calculate summary statistics from combined groups. </p><p>Averaging values, e.g. for 10 values within a bracket </p><p>Rounding values: reducing precision of estimates to reduce the degree to which they might relate to one or a small number of individuals. </p><p>Because descriptive and summary statistics are very commonly reported, care should be taken to ensure that SDC can be maintained in future releases of statistical results based on the same data, including avoiding risks of secondary disclosure by combining information from separate exports.</p> | <p>Number of observations for all statistics. </p><p>For medians, quartiles, and percentiles: the number of participants or events with this value. </p><p><br></p> |

### Extreme values, outliers, unusually high or low observations

These include, for example, minimum and maximum values and outliers.

| **Concerns and what not to export**                                                    | **Potential solutions (can export)**                                                                                                                        | **Minimum requirement for release**       |
| -------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |
| Unsafe as these values often represent individual observations and cannot be exported. | <p>Suppression: replace cell values with “redacted” or a similar label. </p><p>Averaging: for example, summarising the top 10 or bottom 10 observations</p> | Number of observations for all statistics |

### Figures or graphs

| **Concerns and what not to export**                                                                                                                                                               | **Potential solutions (can export)**                   | **Minimum requirement for release**                                                                                                                                                               |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| <p>Underlying data may be hidden and unsafe.</p><p>Restrictions regarding the type of information shown apply, please see the rows corresponding to the type of statistic shown in the graph.</p> | See rows corresponding to the type of statistic shown. | Underlying statistics or results and corresponding group sizes must be submitted in a separate table. Exceptions include density curves, ROC curves, spline functions based on predictive models. |

### Correlation coefficients

| **Concerns and what not to export**                                                                                                                                                                                                                                                | **Potential solutions (can export)**                                                                                                          | **Minimum requirement for release**                            |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- |
| <p>Correlations based on small groups are potentially disclosive and cannot be exported.</p><p>Values close to +/- 1 may have higher risks of disclosure, particularly secondary disclosure when other descriptive statistics (means and standard deviations) may be exported.</p> | <p>Suppression: replace values with “redacted” or a similar label. Suppress extremes if risky.</p><p>Rounding values to 2 decimal places.</p> | Number of observations, overall and within relevant subgroups. |

### Regression coefficients

| **Concerns and what not to export**                                                                                                                                                                                                                                                            | **Potential solutions (can export)**                                                                                                                                                                                | **Minimum requirement for release**                                                              |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| <p>Models with <5 degrees of freedom or small subgroups potentially disclosive.</p><p>Intercepts may expose small categories.</p><p>Residuals are unsafe.</p><p>Regressions on small groups cannot be exported. </p><p>Regressions with less than 5 degrees of freedom cannot be exported.</p> | <p>Suppression: replace cell values with “<10” or “redacted”, or a similar label. </p><p>Combining groups or categories:</p><p>Remove intercept (especially if the model is only using categorical variables). </p> | <p>Number of observations, overall and within relevant subgroups. </p><p>Degrees of freedom.</p> |

### Regression residuals

| **Concerns and what not to export**                                                                                          | **Potential solutions (can export)** | **Minimum requirement for release** |
| ---------------------------------------------------------------------------------------------------------------------------- | ------------------------------------ | ----------------------------------- |
| Unsafe as these as individual data points can be re-derived in combination with the regression model and cannot be exported. | None – always unsafe                 | Not applicable                      |

### Test statistics&#x20;

These include, for example, t-test, chi-squared, and p values.

| **Concerns and what not to export**                                                                                                                                                                                                                                 | **Potential solutions (can export)**                                                                                                                                                                                                                                                                                                                                                                | **Minimum requirement for release** |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------- |
| <p>Statistics based on small groups are potentially disclosive and cannot be exported. </p><p>Repeated exports risk inference.</p><p>Test statistics should be treated the same way as other descriptive or summary statistics (mean, standard deviation, etc).</p> | <p>As described above for descriptive or summary statistics: suppression, combining groups or categories, averaging values, and rounding values. </p><p>Similarly, care should be taken to ensure that SDC can be maintained in future releases of statistical results based on the same data, including avoiding risks of secondary disclosure by combining information from separate exports.</p> | Number of observations.             |

### Goodness of fit statistics&#x20;

These include, for example:

* likelihood ratios
* chi-squared statistics
* R-squared
* adjusted R-squared and pseudo-R-squared
* deviance values
* information criteria

| **Concerns and what not to export**                                                                                                                                                                                                                                                                                                                                                            | **Potential solutions (can export)**                                                                                                                                                                                                                                                                        | **Minimum requirement for release** |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------- |
| <p>Statistics based on small groups cannot be exported. </p><p>R-squared statistics of various types can correspond to correlation coefficients (and regression coefficients). Very high R-squared values (close to 1) may therefore have higher risks of disclosure, particularly secondary disclosure when other descriptive statistics (means and standard deviations) may be exported.</p> | <p>Rounding values is recommended to limit the risks of secondary disclosure (and because high precision is typically uninformative in goodness-of-fit statistics). </p><p>Suppression: replace extremely high R-squared values with e.g., “<-0.9” or “>0.9”, “redacted” or a similar label.</p><p><br></p> | Number of observations.             |

### Odds ratios

| **Concerns and what not to export**                                                                                                                                                                                                                                                                                                                                          | **Potential solutions (can export)**                                                                                                                                                                                                                                                                             | **Minimum requirement for release**                                                                                                                                                                                                                      |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| <p>Odds ratios based on small groups or small cell counts are potentially disclosive and cannot be exported.</p><p>Extremely large or small odds ratios, including infinite odds ratios, may increase disclosure risk, particularly where rare outcomes or categories are involved.</p><p>Excessive precision may increase disclosure risk and is generally unnecessary.</p> | <p>Suppression: replace values with “redacted”, “not estimable” or a similar label where disclosure risk remains high.</p><p>Combining groups or categories to increase group sizes.</p><p>Rounding odds ratios and confidence intervals to 2–3 significant figures (or 2 decimal places where appropriate).</p> | <p>Number of observations for each group.</p><p>Confidence intervals should be provided alongside odds ratios.</p><p>Models should generally have at least 5 degrees of freedom.</p><p>Infinite odds ratios should be replaced with “not estimable”.</p> |

### Genomic research outputs

#### Individual level SNP / genotype data

| **Concerns and what not to export**                                                                                                                                                                          | **Potential solutions (can export)**                             | **Minimum requirement for release** |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------- | ----------------------------------- |
| <p>Individual-level SNP or genotype data is directly identifiable and cannot be exported. </p><p>Even where direct identifiers have been removed, genomic data carries a high risk of re-identification.</p> | None – individual-level SNP or genotype data cannot be released. | Not applicable.                     |

#### Allele / genotype counts

| **Concerns and what not to export**                                                                                                                                                                   | **Potential solutions (can export)**                                                                                                                                                                                                                                                         | **Minimum requirement for release**                        |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- |
| <p>Rare alleles or genotype counts based on small groups may be potentially disclosive and cannot be exported.</p><p>Variants observed in only one participant (singletons) must not be released.</p> | <p>Suppression: replace values based on groups with fewer than 10 observations with “<10”, “redacted”, or a similar label.</p><p>Apply complementary suppression where needed to prevent suppressed values being inferred.</p><p>Combining groups or categories to increase group sizes.</p> | Each reported group must contain at least 10 participants. |

#### Allele / genotype frequencies

| **Concerns and what not to export**                                                                                                                                                                                             | **Potential solutions (can export)**                                                                                                                                                                           | **Minimum requirement for release**                                                                                                  |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| <p>Rare allele or genotype frequencies may present a high risk of disclosure and cannot be exported where based on small groups.</p><p>Very low minor allele frequencies (MAFs) may increase the risk of re-identification.</p> | <p>Suppression: replace values based on groups with fewer than 10 observations with “redacted” or a similar label.</p><p>Avoid releasing very low minor allele frequencies unless sufficiently aggregated.</p> | <p>Frequencies must be based on at least 10 participants.</p><p>Number of observations should be provided alongside frequencies.</p> |

#### GWAS summary statistics (betas, Odds Ratios, Standard Errors, p-values)

| **Concerns and what not to export**                                                                                                                                                                                                | **Potential solutions (can export)**                                                                                                                                                                                                                                                                                  | **Minimum requirement for release**                                                                                                          |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| <p>Rare variants may increase the risk of participant re-identification and cannot be exported where based on very small groups.</p><p>Extremely large odds ratios or very small standard errors may increase disclosure risk.</p> | <p>Suppression: replace values where the Minor Allele Count (MAC) is fewer than 10 with “redacted” or a similar label.</p><p>Suppress tests with fewer than 5 degrees of freedom.</p><p>Round betas and odds ratios to 3 significant figures.</p><p>Very small p-values may be truncated (for example, p < 1e-8).</p> | <p>Minor Allele Count (MAC) must be at least 10 per SNP.</p><p>Number of observations should be provided for each SNP where appropriate.</p> |
