 |
| Australian Q Fever Register |
| Making the workplace safer for meat and livestock workers |
Quality Assurance
Register Data Quality
Data on the Q fever Register is entered by users via the Internet. Many aspects of the data entry process are controlled by the system to prevent problems such as missing or inconsistent data, which are automatically detected. Further checks are performed when original documentation is received by the Register office and reconciled against the data entered by users on the system.
Despite these measures, it is still possible for errors to find their way into the Register. Some errors are of little consequence, such as a small typographical error in the street name, while others are potentially very important, such as an incorrect test result. The Register therefore contains a mechanism which allows managers and users to estimate the proportion and nature of errors contained within the system.
It is worth noting that virtually all databases, especially those generated through the input of many different people working in different circumstances, contain errors. The presence of small, relatively unimportant errors are unlikely to significantly impact the value of the system, while major errors may indicate that improved data entry systems are required. The purpose of the Quality Assurance component of the Register is to alert managers to problems if and when they occur, and to provide users with an objective measure of data quality, in order to ensure continuing confidence in the system.
The database managers are not aware of any other similar databases that implement a routine, automated self evaluation mechanism with publication of the findings.
Methodology
A random sample of data items (single pieces of data, such as first name, or date of test) is selected from all the data in the Register. These data items are identified only by a Register reference number. The Register manager is asked to re-enter specifically referenced data items from the original hard-copy documentation, without being shown the values stored in the Register. The differences between the data stored in the Register and that obtained from the original documentation are statistically analysed to estimate the rate of errors in the database. After the error rate is estimated, any errors detected are corrected.
This analysis is repeated every 6 months to continuously monitor changes in data quality.
Results
July 2002
Summary
Sample size: 871
Errors detected: 13
Estimated error rate: 1.49%
95% confidence interval: 0.83% - 2.61%
Analysis of errors
| Data Type | Errors | Comments |
| Personal Details | ||
| Middle Name | 2 | Minor typographical errors |
| Family Name | 2 | Minor typographical error |
| Address | 3 | 2 Minor typographical errors, 1 Street instead of Avenue |
| Consent to be contacted for research | 2 | 2 indicated that consent had not been given whereas in fact it had |
| Town | 1 | Incorrect town |
| Post Code | 1 | Incorrect post code |
| Job Type | 2 | Incorrect classification |
| Test Results | 0 | |
| Vaccination | 0 | |
Conclusion
The most important finding from this study is that no errors were detected in the critical data maintained by the Register - test results and vaccination records. Those errors that were detected were at a very low level (1.5%) and, as they were mainly minor typographical errors, were unlikely to have any impact on the operation of the Register.
January 2003
Summary
Sample size: 500
Errors detected: 5
Estimated error rate: 1.0%
95% confidence interval: 0.37% - 2.46%
Analysis of errors
| Data Type | Errors | Comments |
| Personal Details | ||
| Middle Name | 1 | Middle name omitted |
| Address | 2 | Minor typographical errors |
| Town | 1 | Incorrect town |
| Job Type | 1 | Incorrect classification |
| Test Results | 0 | |
| Vaccination | 0 | |
Conclusion
The findings of the second quality assurance survey indicate that there has been no increase in the Register error rate compared to the first survey, despite an increase in the number of users entering data. All errors detected were relatively minor, and, most importantly, there were no errors detected in the critical data maintained by the Register - test results and vaccination records.
July 2003
Summary
Sample size: 500
Errors detected: 6
Estimated error rate: 1.2%
95% confidence interval: 0.49% - 2.73%
Analysis of errors
| Data Type | Errors | Comments |
| Personal Details | ||
| Date of Birth | 1 | Incorrect day of month |
| Town | 1 | Incorrect town |
| Test Results | ||
| Date of Test | 3 | Incorrect day of month (error of less than 3 days) |
| Vaccination | ||
| Date of Vaccination | 1 | Incorrect day of month (error of 1 day) |
Conclusion
The findings of the third quality assurance survey indicate that the error rate is continuing at the same very low level of previous surveys. For the first time, errors were detected in data on vaccination and tests, however these errors all related to the date of the vaccination or test and the incorrect dates were all within a few days of the correct date. These errors therefore had no impact on the integrity of the sysem.
December 2003
Summary
Sample size: 500
Errors detected: 8
Estimated error rate: 1.6%
95% confidence interval: 0.75% - 3.26%
Analysis of errors
| Data Type | Errors | Comments |
| Personal Details | ||
| Job Type | 3 | Incorrect classification |
| Town | 1 | Specified suburb name instead of town name |
| Given Name | 1 | Minor typographical error |
| Street Address | 1 | Minor typographical error |
| Consent to be contacted | 1 | Mistakenly indicates consent was given |
| Test Results | 0 | |
| Vaccination | ||
| Date of Vaccination | 1 | Incorrect year (error of 1 year) |
Conclusion
The high level of data quality continues, as demonstrated by the fourth quality assurance survey. Only two errors of any significance were detected, one relating to consent to be contacted for future research, and one on the date of vaccination (where the incorrect year was provided, but the date was otherwise correct). As with all previous surveys, no errors affecting the Q fever status of any individual were detected. Based on the analysis of all surveys, this indicates that we may be more than 99% confident that the maximum possible status error rate in the Register is less than 0.2%.
July 2004
Summary
Sample size: 500
Errors detected: 6
Estimated error rate: 1.2%
95% confidence interval: 0.49% - 2.73%
Analysis of errors
| Data Type | Errors | Comments |
| Personal Details | ||
| Street Address | 4 | Minor typographical error |
| Test Results | ||
| Date of Test | 2 | Incorrect date (error of 1 week) |
| Vaccination | 0 | |
Conclusion
The only non-typographical errors detected in this round of the quality assurance survey related to the date of testing. In both cases, the date in the Register was one week later than the date on form, which is likely to be due to confusion between the date the skin test was administered and the date on which the results were read. These errors have no significant impact on the operation of the Register.
January 2005
Summary
Sample size: 500
Errors detected: 5
Estimated error rate: 1.00%
95% confidence interval: 0.37% - 2.46%
Analysis of errors
| Data Type | Errors | Comments |
| Personal Details | ||
| Job Type | 1 | Incorrect category chosen |
| Secret Question | 1 | Incorrect question chosen |
| Test Results | ||
| Date of Test | 3 | Incorrect date (error of 1 week) |
| Vaccination | 0 | |
Conclusion
Again, a number of small errors related to the date of testing were discovered which are likely to be due to confusion between the date the skin test was administered and the date on which the results were read. These errors have no significant impact on the operation of the Register.
July 2005
Summary
Sample size: 500
Errors detected: 4
Estimated error rate: 0.80%
95% confidence interval: 0.26% - 2.18%
Analysis of errors
| Data Type | Errors | Comments |
| Personal Details | ||
| Job Type | 2 | Incorrect category chosen |
| Address | 1 | Typographic error in street name |
| Test Results | ||
| Date of Test | 1 | Incorrect date (error of 1 week) |
| Vaccination | 0 | |
Conclusion
The error relating to the date of test is, has previously been noted, probably due to the difference between the date the test done (blood taken or skin test injection) and the date of the results (normally about a week later).
Since the last QA survey, the Register record management system has moved from paper files to electronic storate of scanned files. This continuing low error rate demonstrates that this new system has not resulted in any change in data quality stored in the Register.
January 2006
Summary
Sample size: 500
Errors detected: 0
Estimated error rate: 0.00%
95% confidence interval: 0.0% - 0.95%
No Errors Detected
Conclusion
The absence of errors detected indicates a continuing high level of accuracy in the operation of the register.
July 2006
Summary
Sample size: 500
Errors detected: 13
Estimated error rate: 2.60%
95% confidence interval: 1.45% -4.52%
Analysis of errors
| Data Type | Errors | Comments |
| Personal Details | ||
| Job Type | 1 | Incorrect category chosen |
| Job Type | 1 | No category selected on form |
| Consent to be contacted | 2 | Incorrect consent data |
| Name | 1 | Missing middle name |
| Test Results | ||
| Date of Test | 3 | Incorrect date (error of < 1 week) |
| Date of Test | 1 | Incorrect date (error of > 1 week) |
| Vaccination | ||
| Vaccination Date | 2 | Missing date on form |
| Vaccination Date | 1 | Typo in date |
| Batch Number | 1 | Typo in batch number |
Conclusion
This is the highest number of errors found in any of the quality control surveys since the establishment of the register and the first time the estimated error rate has gone over 2%.
Of the errors a number may be considered important, namely incorrect informaiton on consent to be contacted for further research (2 records), incorrect test date (by more than 1 week - 1 record), missing vaccination dates (2 records) and incorrect batch number (1 record). Further analysis is being undertaken to determine if these errors are associated with any particular data entry operator, in order to assess if further training is required.
January 2007
Summary
Sample size: 500
Errors detected: 8
Estimated error rate: 1.63%
95% confidence interval: 0.76% - 3.31%
Analysis of errors
| Data Type | Errors | Comments |
| Personal Details | ||
| Job Type | 3 | Incorrect category chosen |
| Name | 1 | Missing middle name |
| Date of Birth | 1 | Typographical error |
| Post Code | 1 | Typographical error |
| Secret Question | 1 | Incorrect question |
| Test Results | ||
| Type of Test | 1 | Incorrect test type |
Conclusion
The error rate for this period has returned to the normal range. None of the errors detected were critical, although some could hamper identification of individuals.
July 2007
Summary
Sample size: 500
Errors detected: 10
Estimated error rate: 2.00%
95% confidence interval: 1.02% - 3.77%
Analysis of errors
| Data Type | Errors | Comments |
| Personal Details | ||
| Job Type | 1 | Incorrect category chosen |
| Consent to be contacted | 2 | In one case, consent given but recorded as refused, in the other, consent refused but recorded as given |
| Date of Birth | 1 | Typographical error |
| Test Results | ||
| Date of Test | 2 | Error of 2 days and 1 week |
| Vaccination | ||
| Batch Number | 1 | Typographical error |
| Date of Vaccination | 3 | Errors of 2 days, 1 month and 3 months |
Conclusion
The error rate is within the normal range. The only critical error is the one case of consent not being give to be contacted for research purposes, but being recorded as given. The error rate for consent has been very low in the past. In practice, such errors have had no importance as there have been no requests during the life of the register to use the register for any research that would involve contacting individuals.
January 2008
Summary
Sample size: 500
Errors detected: 4
Estimated error rate: 0.80%
95% confidence interval: 0.26% - 2.18%
Analysis of errors
| Data Type | Errors | Comments |
| Personal Details | ||
| Surname | 1 | Incorrect surname |
| Date of Birth | 1 | Typographical error |
| Test Results | ||
| Date of Test | 2 | Error of 1 week |
Conclusion
The error rate is very low. The incorrect surname is important, and this is the first time this has been detected.
July 2008
Summary
Sample size: 500
Errors detected: 8
Estimated error rate: 1.64%
95% confidence interval: 0.76% - 3.33%
Analysis of errors
| Data Type | Errors | Comments |
| Personal Details | ||
| Town | 1 | Incorrect town |
| Surname | 1 | Typographical error |
| Address | 1 | Typographical error |
| Gender | 1 | Incorrect gender selected |
| Secret question | 2 | Typographical error and incorrect answer |
| Test Results | ||
| Date of Test | 1 | Error of 1 week |
| Vaccination | ||
| Date of Vaccination | 1 | Error of 5 days |
Conclusion
The error rate continues to be in the acceptable range. As previously noted, errors of up to a week for the date of testing or of vaccination are relatively common and do not affect the operation of the register. This is the first time an error has been detected with the gender.
January 2009
Summary
Sample size: 500
Errors detected: 2
Estimated error rate: 0.41%
95% confidence interval: 0.07% - 1.64%
Missing data items: 4
Analysis of errors
| Data Type | Errors | Comments |
| Personal Details | ||
| Missing data | 4 | Missing details (both on submitted form and in database) including postcode, job type and batch number |
| Test Results | ||
| Date of Test | 1 | Error of 4 days |
| Vaccination | ||
| Date of Vaccination | 1 | Error of 1 year |
Conclusion
The error rate is low. A number of items of missing data were detected in this round, indicating a low incidence of incomplete forms. The missing post code is unlikely to be important. The missing job type information will have a very minor impact on system statistics. Batch numbers are included for possible future follow-up in case of research or investigation. Thus far, in the life of the register, batch numbers have not been used.
June 2009
Summary
Sample size: 500
Errors detected: 9
Estimated error rate: 1.8%
95% confidence interval: 0.88% - 3.51%
Missing data items: 3
Analysis of errors
| Data Type | Errors | Comments |
| Personal Details | ||
| Names | 4 | Typographical errors in middle names |
| Address | 2 | Typographical errors in postcode |
| Consent | 1 | Incorrectly recorded as consent given to be contacted for research purposes |
| Job type | 1 | Incorrect classification |
| Vaccination | ||
| Batch number | 1 | Incorrect number |
Conclusion
The error rate is once again in the expected range. No critical errors relating to the immune status of individuals were detected. It is worth noting that the register is now completely electronic, with paper forms immediately scanned for long-term storage and the original shredded. During this survey two items were unreadable due to poor quality scans. Scanning was originally contracted commercially, but is now being done by the register managers to ensure consistent high quality.
January 2010
Summary
Sample size: 500
Errors detected: 7
Estimated error rate: 1.4%
95% confidence interval: 0.61 - 2.99%
Missing data items: 3
Analysis of errors
| Data Type | Errors | Comments |
| Personal Details | ||
| Gender | 1 | Incorrect gender |
| Address | 4 | Two incorrect towns and incorrect postcodes (both for neighbouring towns) |
| Job type | 1 | Incorrect classification |
| Tests | ||
| Test date | 1 | Incorrect date (10 day difference) |
Conclusion
No critical errors were detect and the error rate continues to be low.
| Home | Contacts | Help | Access Register | Download Area |