Content types on the ONS website Quality and methods guides

Overview

Quality and methods guides are the main methodology content type on the ONS website. They provide detailed supporting information for official statistics.

Every regular set of statistics or statistical article series must have a corresponding quality and methods guide that includes information on:

• any recent, past, and upcoming changes to quality and methods
• what the statistics cover
• how we get the data and produce the statistics
• the quality of the statistics
• the comparability and coherence of the statistics
• who uses the statistics, and what for
• understanding complex terms

There can only ever be one version of a quality and methods guide. This is to avoid inconsistent and duplicated information, and to ensure users have one source of truth.  The content lives on a single page with no previous editions.

Users need to feel confident they are accessing the most up-to-date information when using a quality and methods guide. You do not necessarily need to update the guide every time you release new statistics or publish a new edition of a statistical article. However, you must review the guide regularly and update it whenever there is an important change to the quality or methods.

Example

View an example of a quality and methods guide (opens in a new tab) 

Title and summary

Title

A quality and methods guide title must:

• include the same topic and geography as the corresponding statistical article or dataset page title
• not include a time period – there should only ever be one version of a quality and methods guide that is kept up to date
• state that the page is a quality and methods guide – this is so search engines can tell the difference between statistical articles and quality and methods guides for the same topic

Read more about general best practice in our Titles and headings guidance.

Summary

The standard summary all quality and methods guides must use is: 

"What the [topic] statistics cover, how we produce them, and their quality and comparability. Includes definitions and latest, past and upcoming changes."

Overview section

All quality and methods guides must start with an Overview section that summarises the most important takeaways. These will be different for each topic, but might include:

• what surveys or data sources we create the statistics from
• the coverage of the statistics
• how reliable and complete the statistics are
• any essential quality considerations

This helps users to get the main messages and receive reassurance quickly, and know if the content is relevant to their needs.

View an example (opens in a new tab) 

If all of the statistics are accredited official statistics and/or official statistics in development, use the following standard text at the end of the Overview section:

• "These are accredited official statistics."
• "These are official statistics in development."
• "These are a combination of accredited official statistics and official statistics in development."

 Link to the Quality of the statistics section for more detail on the designation.

If only some of the statistics are accredited official statistics and/or official statistics in development, use the following standard text:

"For information about the statistical designation of these statistics, go to Quality of the statistics."

The Overview section should be a few paragraphs in length. Aim for fewer than 300 words.

Latest changes to quality and methods section

All quality and methods guides must include a Latest changes to quality and methods section.

This helps users to:

• quickly understand if the quality or methods have changed
• feel confident they are accessing the most up-to-date information
• access links to more detailed sections further down the page on any new changes

If the quality or methods have changed

If there has been at least one important change since you previously updated the guide, start the section with the following standard lead-in line: 

"We updated this guide on [DD MM YYYY]. Important changes to quality and methods include:"

Briefly summarise each change in a separate bullet point – more detail can be included in the Changes and their effects on comparability over time section.

Examples of important changes include changes to:

• a definition
• classifications and groupings
• geographic boundaries
• when we extract the data
• what datasets we publish
• response rates
• survey questions

View an example (opens in a new tab) 

Each time you update the guide, remove any changes that were added in the previous update.

At the end of the Latest changes to quality and methods section, use the following standard text:

"For more information on these latest changes, as well as any past and upcoming changes, go to Changes and their effects on comparability over time."

If the quality or methods have not changed

You might need to make minor updates to the guide that would not be considered important changes to quality and methods. In these cases, use the following standard text:

"We made minor updates to this guide on [DD MM YYYY]. We made no important changes to quality and methods.

"For information on past and upcoming changes, go to Changes and their effects on comparability over time."

What the statistics cover section

All quality and methods guides must include a What the statistics cover section. 

This helps users to understand what is and is not included in the statistics, so they can see if they are relevant to them. 

Include information on the statistical coverage, such as: 

• geographic areas
• time periods
• demographics
• categories and breakdowns

If the statistics do not cover something that users might reasonably expect, explain why it is excluded. 

If the statistics are intended to cover something, but that coverage is limited in practice, explain this clearly.

View an example (opens in a new tab) 

Where the data come from and how we produce the statistics section

All quality and methods guides must include a How we get the data and produce the statistics section with two subsections: 

• Where the data come from
• How we produce the statistics

This helps users to have a detailed understanding of our data collection and statistical production methods.

View an example (opens in a new tab) 

Where the data come from subsection

Include information on how you collect the data through surveys or administrative data sources.

If multiple sources are used, describe how each contributes to the statistics. For example:

• different groups may be covered by different sources
• certain sources may be used to weight the data

How we produce the statistics subsection

Provide a description of the main steps taken to produce the statistics from the data using either:

• a numbered list (read our Numbered lists guidance)
• subheadings

Provide reassurance to users without going into detailed, reproducible steps.

Quality of the statistics section

All quality and methods guides must include a Quality of the statistics section.

This helps users to understand the quality and trustworthiness of the statistics, and how to use them appropriately.

The section must include four subsections: 

• Statistical designation
• How we quality assure the data and statistics
• Strengths and limitations
• European Statistical System Quality Dimensions

View an example (opens in a new tab) 

Statistical designation subsection

This subsection provides users with a short statement about the official designation of the statistics. 

Use the standard wording agreed with the Office for Statistics Regulation (OSR) and the Statistics Head of Profession Office, available in our Statistical designation guidance.

Read more about statistical designations on the OSR’s website:

• accredited official statistics (opens in a new tab) 
• official statistics in development (opens in a new tab) 

How we quality assure the data and statistics subsection

Provide a description of the main steps taken to quality assure the data and statistics using either:

• a numbered list (read our Numbered lists guidance)
• subheadings

Provide reassurance to users without going into detailed, reproducible steps.

Strengths and limitations subsection

This subsection briefly summarises the main strengths and limitations of the statistics in relation to their quality.

Include a bullet pointed list of the main strengths under a Strengths subheading, and the main limitations under a Limitations subheading. 

Each bullet point should provide a brief summary of a strength or limitation. Link to another section for more detail if needed.

Bullet points must start with a capital letter, end with a full stop, and only include one short sentence. Read more in our Bullet points guidance.

If more detail on a strength or limitation is needed, but is not appropriate to include in another section, use subsections in the Strengths and limitations subsection.

European Statistical System Quality Dimensions

To help users measure the quality of the statistics against European standards, use the following standard text: 

“The Office for National Statistics (ONS) has developed Guidelines for measuring statistical quality (opens in a new tab)  based on the five European Statistical System (ESS) Quality Dimensions. These are:

• relevance
• accuracy and reliability
• timeliness and punctuality
• comparability and coherence
• accessibility and clarity

"We have integrated these considerations into the guide.”

Changes and their effects on comparability over time section

All quality and methods guides must include a Changes and their effects on comparability over time section.

This helps users to:

• know which changes to quality and methods affect comparability of the statistics over time
• recognise real trends rather than those caused by changes to methods

If appropriate, the section can start with a short summary of why we sometimes need to make changes.

The section must include three subsections: 

• Latest changes
• Past changes
• Upcoming changes

If there are multiple changes to cover in a subsection, use a dedicated accordion for each change. This helps users to easily find the change they need. 

If there is more than one change in a subsection, use the following standard text at the start of the subsection:

"These changes are ordered by date, with the most recent first."

You do not need to use an accordion if there is only one change in a subsection – just use plain text.

Clearly explain whether each change affects comparability over time or not.

View an example (opens in a new tab) 

Latest changes subsection

Use this section to provide additional detail on the latest changes you briefly covered in the Latest changes to quality and methods section.

Latest changes to quality and methods are important changes that occurred since you previously updated the guide.

Each time you update the guide, move any changes that were added to the Latest changes subsection in the previous update to the Past changes subsection. These are no longer considered latest changes. 

If there are no latest changes, use the following standard text: 

"We made minor updates to this guide on [DD MM YYYY]. We made no important changes to quality and methods."

Past changes subsection

If there are no past changes, use the following standard text: 

“We have not made any important changes in the past.”

Upcoming changes subsection

Upcoming changes are changes that we know will take place in the near future.

If there are no upcoming changes, use the following standard text: 

“We currently have no plans to change the methods in the near future.”

Comparability and coherence with other statistics producers section

All quality and methods guides must include a Comparability and coherence with other statistics producers section.

This helps users to know if other organisations produce statistics on the same or related topics, and if so, any similarities and differences.

A statistical article can include a Data from other statistics producers section  (opens in a new tab) that provides a rating of “highly”, “broadly” or “not” comparable for each comparative source. This section signposts to the Comparability and coherence with other statistics producers section in the corresponding quality and methods guide to give users more information on why we have given these ratings. 

In the guide, each comparative source must have its own subsection within the Comparability and coherence with other statistics producers section. The subsection heading must be the name of the statistics producer. 

If the statistics are not comparable

Use the following sentence at the start of the subsection:

“[Statistics producer]’s [topic] statistics are not comparable with ours because of the following differences.”

Use a clear and consistent table format to show the differences. 

View an example (opens in a new tab) 

If the statistics are broadly comparable

Use the following sentence at the start of the subsection:

“[Statistics producer]’s [topic] statistics are broadly comparable with ours, but be aware of the following differences.”

Use a clear and consistent table format to show the differences. 

If the statistics are highly comparable

Use the following sentence at the start of the subsection:

"[Statistics producer]’s [topic] statistics are highly comparable with ours, and there are no differences to be aware of."

Users and uses of these statistics section

All quality and methods guides must include a Users and uses of these statistics section.

This helps users to understand the relevance and importance of the statistics.

Use bullet points to briefly set out which key stakeholders use the statistics, and what they use them for. If you need to go into more detail, use subsections instead.

View an example (opens in a new tab) 

Definitions section

All quality and methods guides must include a Definitions section. 

Clear definitions help users to understand complex methodological terms. Use a dedicated section with an accordion for each term to help users find the definition they need.

View an example (opens in a new tab) 

Definitions are added through a central ONS glossary in the content management system (CMS) and reused across multiple publications. This is to ensure consistency across our content. 

If you need to add a new term to the glossary, or update an existing term, email content.design@ons.gov.uk (opens in a new tab) 

Related links section

All quality and methods guides must include a Related links section. 

Include links to relevant, timely and useful pages. Three to six is the optimum number for good engagement, but you can include additional links if there is a clear user need to do so.

The format of this section is automated within the CMS to display the page title and link, release date, content type and page summary.

View an example (opens in a new tab) 

Cite this page section

All quality and methods guides must include a Cite this page section. 

The format of this section is automated within the CMS to display the source, last updated date, page title and link, location, content type, and page title and link. 

It helps users to cite our publications accurately and consistently.

View an example (opens in a new tab)