Index Last Week Contact DQ News Latest Issue
Discrepancies Between Meta-Analysis and
Subsequent Large Randomized, Controlled
Trials
A special article in the August 21st issue of The New England Journal of Medicine compares the results of large randomized, controlled trials (involving 1000 patients or more) that were published in four medical journals (the New England Journal of Medicine, the Lancet, the Annals of Internal Medicine, and the Journal of the American Medical Association) with the results of meta-analyses published earlier on the same topics.
The authors identified 12 large randomized, controlled trials and 19 meta-analyses addressing the same questions. The randomized controlled trials were published between 1991 and 1994. The authors then searched for meta-analyses of similar topics that had been published before the large randomized, control trial. Searches included the references listed in the randomized trials and computrerized searches of Medline. The authors carefully compared each trial with the set of meta-analyses corresponding to it and selected only those meta-analyses that coincided with the trial with regard to population similarity, theraputic intervention, and the principal and secondary outcomes. The authors concluded that the outcomes of the large randomized, controlled trials that they studied were not predicted accurately 35 percent of the time by the meta-analyses published previously on the same topics. Written by Jacques LeLorier, Genevieve Gregoire, Abdeltif Benhaddad, Julie LaPierre, and Francoise Derderian of the University of Montreal Faculty of Medicine and Montreal Hospital Research Center, the article appears on page 536.
An editorial by Dr. John Bailar III in the same issue of NEJM discusses the LeLorier research and issues and problems that arise when meta-analysis is used. Dr. Bailar's editorial appears on page 559.
Money Magazine's Hyperactive 'Best Places' List
According to an article in the August 24th issue of The Washington Post, wild swings in position have been the norm in the annual Money magazine ranking of the "quality of life" in 300 American metropolitan areas. From 1996 to 1997, cities moved an average of 40 places up or down - and 23 metro areas rose or fell 100 positions or more. Money has been publishing it's "Best Places" list for 11 years. The list is widely reported and high rankings are hailed by mayors, real estate agents, and chambers of commerce. Low rankings promote dismay and frantic editorials in local newspapers.
To produce its "Best Places" listing, Money magazine surveys 250 to 500 subscribers annually to determine the relative importance of 41 "quality of life" factors. The magazine then collects crime, economic, housing and other socio-economic data about cities and economic areas. Money then weights the data to reflect what readers value most in the poll, according to the magazine. The exact weighting scheme the magazine uses to translate data into rankings is a closely guarded secret.
At least one critic has dertermined that the weighting scheme that Money uses gives high value to indicators of short-term economic strength in each city, and puts too little weight on other "quality of life" factors such as safety from crime and quality health care. Relying on short-term indicators, such as projected job growth in smaller metropolitan areas is likely to produce exaggerated changes in the ratings. Money magazine counters that changes in readers preferences and year-to-year changes in economic and social conditions sometimes produce dramatic fluctuations in a city's ranking. Written by Post columnist Richard Morin, the article appears on page C5.
2 Communities, Poles Apart, Illustrate
Debate on Census
A front page report in the August 30th issue of The New York Times discusses the problems the U.S. Census Bureau will try to avoid in 2000 by using statistical sampling to supplement traditional enumeration. According to the article, Census Bureau officials acknowledge failing to count nearly 10 million people in 1990 and admit that 4.4 million others were counted twice or were made up by Census enumerators who could not gain access to homes.
The Times article focuses on two socially and economically different neighborhoods in the Harrisburg, Pensylvania metropolitan area, and the problems the Census Bureau will face there in 2000. In one low-income neighborhood both the transient nature of many residents and their apathy about being counted in a census lead to an undercount of 5.4 percent in 1990. Persons likely to not be counted in such neighborhoods are black and Hispanic men, children, and renters.
In contrast, the Census Bureau estimated an upper-income community in the Harrisburg area was overcounted by 2 percent in 1990. This community (New Cumberland) has all the attributes of areas where everyone tends to counted and some groups tend to be overcounted. One attribute is the number of retirees who spend part of the year in warmer climates. Because the Census Bureau sends a questionnaire to every residential address in the country, those with two residences receive two questionnaires. Although only the questionnaire mailed to the primary residence is supposed to be mailed back, many retirees send back both. Accoding to the article, the Census Bureau often did not catch the duplication in 1990.
The Census Bureau is planning to make it easier for people to respond to the Census by providing special census forms in libraries, post offices, and convenience stores. The Bureau will also provide an 800 toll-free telephone number for people to provide information. In 2000, Census enumerators will not visit all the homes that do not return the mailed census questionnaire. The percentage of "follow up" visits to such households would depend on the number of non-responding households in each census tract. The Bureau's goal is to have a completed questionnaire from 90 percent of American households. The Bureau plans three large surveys for coverage improvement and to determine who was missed in the 2000 enumeration. The article was written by Times staff journalist Steven Holmes.