| Index | Last Week | Contact DQ News | Latest Issue |
| Featured Articles | Data & Information | Science & Technology | Education |
| Economics | Business | Law | Medicine |
Data Quality Paper Published in JASA
A paper in the December, 1997 issue of the Journal of the American Statistical Association explores missing data as an issue of longstanding concern among researchers interested in wealth accumulation and portfolios of wealth. Written by Statisticans F. Thomas Juster (University of Michigan) and James P. Smith (The RAND Corporation) the paper, "Improving the Quality of Economic Data: Lessons from the HRS and AHEAD," explores relatively simple new survey methods - such as follow-up brackets - which (according to the authors) appear to appreciably improve the quality of household economic data.
Assets are notoriously poorly reported on American surveys. Non-response is pervasive. Other evidence suggests that asset values may include many errors. Three cognitive problems may explain why missing data rates are so high for so many forms of household wealth. The respondent may not know the answer to the question. The respondent may have a rough idea of the amount but assumes the interviewer wants a precise figure. Third, the respondent may refuse to disclose the value of assets.
The research relies on data from two surveys fielded by the Institute for Social Research at the University of Michigan. The HRS is a national sample of about 7,600 households (12,654 individuals) which was taken during 1992 and 1993. The sample included at least one person in the household between 51 and 61 years old at the interview date. The HRS (Health and Retirement Survey) baseline interview was conducted in-home, face-to-face. The companion survey to the HRS - AHEAD - includes 6,052 households (8,204 individuals) from the birth cohorts of 1932 or before, thus with at least one person age 70 or older in 1993. A supplemental sample of persons age 80 and above enrolled in Medicare was obtained from the Health Care Finance Administration HCFA) to guard against underrepresentation of the extremely disabled elderly. The baseline AHEAD survey was conducted using CATI techniques for respondents age 70-79 and CAPI for person age 80 and older.
The HRS and AHEAD (Asset and Health Dynamics Among the Oldest Old) methodology involved two main features. First, unfolding brackets taking the form of simple questions that follow immediately after a "don't know" or "refuse" response placed the respondent's asset into one of a set of categories. Second, interviewers were told not to extensively probe "don't know" or "refuse" responses, but rather to proceed to the first question in the unfolding bracket sequence. The design philosophy was that dropping the usual practice of probing for exact answers would shorten the survey and minimize the probability of annoying respondents. The paper's authors felt that the loss of data quality as the result from losing some exact answers would shorten the survey and minimize chances of annoying respondents.
The authors conclude that using brackets to estimate total asset values significantly increased the value of mean nonhousing assets (especially business assets) compared to exact answer responses. There was a significant difference in the value of assets between the younger age cohorts (HRS) and the older age cohorts (AHEAD). It appears that ISR did not check survey responses against actual financial records. The paper appears on page 1268.
Designing for Data Quality
The business and design journal @issue: recently featured an article about Morningstar, Inc., a financial services company that has carved a strong brand identity for itself by emphasizing design in the creation of database products. The article, "Designing for Data," emphasizes Morningstar's use of excellent graphic design as a vehicle for enhancing data quality. Morningstar principally publishes printed reports and database products that evaluate mutual funds.
According to the @issue: article, Morningstar designers spend countless hours analyzing ways to compress voluminous amounts of information onto a single page in a clear and accessible manner. The Morningstar Page integrates numerous graphic devices to give investors at-a-glance information about the performance of a mutual fund. To keep the layout clean and open, Morningstar designs every graphic element to contribute valuable reader information. But subjective and objective information are often juxtaposed. Morningstar provides an array of financial products and services related to mutual funds. These include CD-ROMs, financial analysis software, user manuals, newsletters, and printed mutual fund evaluations. The Morningstar Web page can be found at: (www.morningstar.net). The corporate sponsor of @issue: is Potlatch Corporation. The Corporate Design Foundation Web site can be found at (www.cdf.com).
Data Quality Problems Plague Drug Use Surveys
According to a front page Washington Post report on January 2nd, differing surveys and analyses have yielded different, and often unreliable, data about drug use in the United States. The Post reports that drug use in the United States is badly measured. Researchers from the RAND Corporation, GAO Auditors, and Congressional staffers who have studied drug statistics found that drug statistics are "often incomplete, erratic, and contradictory."
One problem that plagues drug use statistics is the sheer number of measurements. There are more than 50 federal drug-related "data systems" with hundreds of "drug variables" produced by an array of federal agencies. The Post article examines four major national drug surveys, and finds problems with all of them - problems that should prevent their data from being used as a measure of drug use in the United States. According to the Post, the surveys are marked by serious flaws in statistical methodology, survey data is used for purposes not intended by a survey's designers, small yearly changes that are not statistically significant are claimed to be very significant by law enforcement officials and politicians, data obtained from respondents are sometimes wrong (it appears high school students often lie about drug use), large trends are often extrapolated from a small number of responses, and the sheer profusion of data can baffle those looking for simple answers.
According to the Post, the Clinton Administration plans to implement a comprehensive "Performance Measurement System" intended, for the first time, to standardize measurement of progress in the "drug war." The article was written by Post staff writer Jeff Leen.