Take the State of StatsCan Survey

  • 主题发起人 主题发起人 guest
  • 开始时间 开始时间

guest

Moderator
管理成员
注册
2002-10-07
消息
402,176
荣誉分数
76
声望点数
0
Statistics Canada was once a source of pride and wonder, but embarrassing errors, budget cuts and policy changes have diminished the institution.

Is this what Prime Minister Stephen Harper wanted all along?





For almost a century now, Statistics Canada and its forerunner, the Dominion Bureau of Statistics, have been asking Canadians questions, tabulating answers and charting trends.

The national statistics agency today produces bundles of data on everything from hog numbers to new car sales to jobs. It publishes about 350 statistical products each year — reports scoured by academics, industry analysts and government officials for social and economic insights.

But the agency, once a source of pride and wonder for its seemingly omniscient reporting on everything from coal to cattle, has proven itself seriously flawed during the past year.

Its most embarrassing gaffe occurred earlier this month when StatsCan had to retract a key economic report: the July 2014 Labour Force Survey.

In early August, the jobs report said that only 200 full-time positions had been created in the previous month. The news came as an unhappy shock to analysts — it signalled underlying weakness in the economy — and led to a drop in the value of the Canadian dollar.

But the numbers were wrong. Statistics Canada issued a corrected labour survey report one week later, adding a whopping 42,000 jobs to its original estimate. An agency report said the mistake was due to human error that occurred when a new data processing system was introduced for the Labour Force Survey (ironically) as part of a $5-million investment in the program.

“It is absolutely not the case that there was any budget cutting that caused or contributed to this problem,” chief statistician Wayne Smith said in an interview this week. “It was strictly an error and a problem with our protocols for testing data that we’re going to fix.”

A similar problem occurred last summer when an error was discovered two days before the scheduled publication of data from the National Household Survey (NHS). StatsCan has never revealed the source of that error, which delayed release of NHS data by one month.

The miscues form part of an unhappy narrative at StatsCan that began in 2008 with a Conservative government initiative to cut costs by returning the agency to its “core” mandate; built with the elimination of the long form census; and continued through the 2012 budget, which imposed more budget cuts.

All of it has raised fundamental questions about the state of Statistics Canada, a national statistics agency once regarded as among the world’s best. What has been the collective impact of six years of cutbacks? Of making fundamental changes to the census? Can the agency still reflect the country to Canadians? And can it withstand still more change?


Munir Sheikh, former chief statistician who resigned to protest the decision to eliminate the mandatory long-form census, speaks at a statistics conference in Vancouver in 2010.

A StatsCan survey, 2008-2014


First, a little history. In February 2008, following a strategic review, Statistics Canada was ordered to find $21.5 million in savings from its budget through internal efficiencies and reduced services. No sooner had the agency completed those cutbacks in 2011, however, than a new round of belt-tightening was ordered. The March 2012 federal budget required the statistics agency to cut a further $33.9 million by 2015.

At the same time, the agency also had to manage a $20-million revenue drop since many cash-strapped federal departments could no longer afford specialized StatsCan surveys.

Some of StatsCan’s staff and responsibilities were transferred to another department, and the government has since injected about $25 million back into the agency for specific programs.

Since 2008, Statistics Canada estimates, it has cut a net total of $34 million from its budget while also absorbing the cost of salary increases. It has meant that 825 jobs — about 16 per cent of full-time positions — have been lost since 2008, mostly through early retirements.

In the middle of that turmoil, Statistics Canada also suffered an unprecedented political crisis. Former chief statistician Munir Sheikh quit in 2010 to protest the Conservative government’s controversial decision to eliminate the mandatory long-form census, valued by academics and social planners for its insights into shifting immigration patterns, living conditions and families. Many consider the long-form census a critical piece of research infrastructure.

It was replaced by the voluntary National Household Survey, which then Industry minister Tony Clement described as a less intrusive, less coercive survey instrument.

Earlier this year, federal Auditor-General Michael Ferguson reported that Canada is paying more to conduct a census that has less value. The new voluntary NHS added $22 million to the cost of administering the census in 2011 yet the shift resulted in a significant drop in the overall response rate: from 94 per cent in 2006 to 69 per cent in 2011.


No caption, leave this text or replace image


The response rate is a key measure of data quality. In fact, the statistics agency will not release data for those census areas that do not achieve a minimum standard. As a result, StatsCan has released data for only 75 per cent of the country’s 4,567 census subdivisions — down from 97 per cent in 2006.

Smith notes that areas for which there’s no published data represent only three per cent of the Canadian population.

Impact assessment


During StatsCan’s downsizing, many smaller surveys have been shelved or scaled back, including those that examined cultural spending, youth, pensions, residential care and trucking. But the budget cuts also eliminated a number of more comprehensive surveys, including:

  • The Workplace and Employee Survey, eliminated in 2009, offered detailed, annual information about pensions, benefits, technology use and training in the private sector
  • The National Longitudinal Survey of Children and Youth, launched in 1994, was a long-term study of Canadian children that followed their development and well-being from birth to early adulthood
  • The Survey of Labour and Income Dynamics was a longitudinal survey — it allowed analysts to chart changes over time — that examined the economic well-being of Canadians and offered insights into the lives of low-income families

University of Ottawa economics professor Miles Corak, a former StatsCan senior manager, calls the changes “a major loss” to our understanding of Canadians.

“Some of the things that helped give Canadians a full portrait of the way they lead their lives have been cut back,” says Corak. “Statistics Canada took pride in calling itself a statistics agency for the nation. But I think it has moved back somewhat to a more traditional role as a service provider for the federal government.”

Laval University economics professor Stephen Gordon says the cuts have limited his ability to compare long data sets over time since so many longitudinal surveys have been interrupted or changed. “The lack of comparability makes it very hard: so many things have become opaque,” he says.

Others, however, including Philip Cross, the former chief economic analyst at Statistics Canada, contend the agency has come through its downsizing as a “better, leaner, more focused organization.”

“I think this is something that every organization should go through every once in a while: it forces you out of your comfort zone,” says Cross, now a senior fellow at the Macdonald-Laurier Institute, a public policy think-tank in Ottawa.

It must be remembered, he says, that the agency has for decades gone through a boom-and-bust cycle of funding: its budget contracted in the 1980s then ballooned in the 1990s when the government needed better provincial data to distribute GST revenue.


No caption, leave this text or replace image


The latest cuts, he argues, have opened the door to staff turnover frustrated by a longstanding no-layoffs policy. In one case, openly discussed among senior managers, an employee put up a sign at his desk, saying, ‘I’m going to be sick every Friday the rest of the year.”

“A lot of deadwood had built up over 25 years,” says Cross, who left the statistics agency in 2012.

Smith says the agency confronted its budget challenge by launching an initiative to streamline operations and consolidate services such as research, communications and data collection. It has freed up about five per cent of the budget for new technology and strategic priorities, including an initiative to eliminate user fees for StatsCan data.

The agency, he says, also chose not to impose across-the-board cuts, but to eliminate “low priority” programs in order to preserve the quality of everything else. The longitudinal studies mourned by the academic community, he says, were simply too expensive and time-consuming.

“I think the things we’ve chosen not to do are lower priority in terms of the broader needs of Canadians, when you go beyond the interests of academics to the interests of government policymakers and businesses,” he says.

Most analysts agree that Statistics Canada continues to produce high quality numbers about the economy and national accounts, information that’s critical to monetary policy. But they’re deeply concerned that the elimination of the long-form census — and the cumulative impact of six years of budget cuts — have impaired the agency’s ability to offer meaningful insight into the lives of Canadians.

“We’re losing our ability to tell clear-eyed stories about what is happening to people,” argues Armine Yalnizyan, senior economist at the Canadian Centre for Policy Alternatives.

“We’re losing our ability to track what’s happening to young people, what’s happening to immigrants, how that compared to what happened a generation ago. We’re losing our ability to tell these stories, which are critical to the way we’ve built our nation.”

Related


Yalnizyan calls the long-form census “the spinal column” of information about people. It provides the means by which (much smaller) random samples for other studies are properly weighted to reflect the country’s demographic makeup.

But researchers know from experience that Canadians from poor families and those who live in rural areas are less likely to respond to voluntary questionnaires such as the National Household Survey — and they worry that StatsCan’s new methodology will ensure that those groups are under-represented in all studies that depend of the census for a “data frame.”

“We have effectively limited our ability to know who needs our help the most,” says Ryerson University professor Murtaza Haider.

Smith contends that data from the NHS is “very robust” given that it is based on responses from 2.5 million households with six million people. “To say that the result is invalid and should be dismissed is absurd,” he says. “I don’t dispute the fact there was an erosion of quality, I just think the issue is being oversold.”

A critical question: Why?


There’s little doubt that Statistics Canada has been diminished during the Conservative government’s decade in power. But a critical question remains: Why? Has the government simply applied good, financial rigour to the agency? Or were the changes politically motivated?

Yalnizyan is among those who believe the Conservative government wants to starve StatsCan of funding in order to limit the production of research that advances the case for social change.

“The whole point is to reduce dissonance,” she says. “And one way to do that is shutting down information sources. It is hugely problematic.”

Take Our Poll
Haider, associate dean of Ryerson University’s Ted Rogers School of Management, believes the Conservatives must be motivated by ideology because “it makes no sense to cut the budget of institutions that are fundamental to the social and economic prospects of the country.”

“What concerns me is that this is an information age: Big Data is big,” says Haider. “But the government is trying to squeeze our data people into as small an envelope as it possibly can.”

Prime Minister Stephen Harper has made no secret of the fact he wants to make federal institutions more friendly to conservatism after decades of Liberal hegemony. To that end, Harper has steadily promoted a new/old set of Canadian symbols — the monarchy, the military, the North — while gradually shrinking the size of government.

By 2015, the federal government (measured by government spending as a percentage of GDP) is projected to be smaller than it has been at anytime in the past 50 years. As Maclean’s columnist Paul Wells argues in his book, The Longer I’m Prime Minster, “He (Harper) is the first prime minster in the history of the country who has wanted to leave behind a government that is doing substantially less than when he arrived.”

There’s no doubt Statistics Canada will be doing less when Harper leaves office, but Philip Cross is convinced that’s not such a bad thing.

“StatsCan is still a much bigger organization than it was a couple of decades ago,” he argues, “and it has a very good sense of where it’s going and what its priorities are.”

Smith contends there’s no evidence to show that Statistics Canada has suffered disproportionately to other federal departments.

Conclusion


Doug Norris, Statistics Canada’s former director general of social and demographic statistics, says the agency used to be a place where people worked for 20 or 30 years and became experts in their field. Much of that experience has left the building, he says.

“I think there’s no doubt that with the errors, the budget cuts, the survey reductions, people have a different attitude, a concern about Statistics Canada,” says Norris, now chief demographer at a private firm, Environics Analytics.


Statistics Canada’s former director general Doug Norris in 2013.


Even federal officials seem to have concerns. The finance department used statistics from a private company, Wanted Analytics, in a February report that examined job vacancies, an area in which Statistics Canada is notoriously weak. (Wanted Analytics’ software scours online job boards.) Then in May, former Conservative cabinet minister David Emerson, who led a heavyweight advisory committee on the public service, told the Citizen that public servants can no longer rely on “structured” and “cleansed” data from Statistics Canada to advise government ministers who have access to massive amounts of unfiltered information.

With its reputation under assault, StatsCan faces still more challenges. First, it will have to get used to diminished circumstances: the agency’s funding is projected to remain constant at just under $400 million for the next three years. (It also expects to generate about $100 million in revenue from cost-recovery projects.)

Then, there are methodological issues. The widespread use of call display and a growing reluctance among Canadians to devote time to surveys have made it more difficult to complete traditional telephone surveys. The National Graduate Survey — it examines how post-secondary grads have fared in the job market — now suffers from an overall response rate of less than 50 per cent, down from 77 per cent three decades ago.

StatsCan is actively experimenting with digital questionnaires, online surveys and Internet data scraping to reduce costs and improve information gathering. It’s also relying, increasingly, on what’s called “administrative data:” income tax records, export permits, motor vehicle registrations and other data sets generated as part of government business.

Exploiting those huge caches of data has become a strategic priority for the agency in the past three years. Statistics Canada routinely uses 500 such data files from government and private sector sources — it now represents about 40 per cent of the agency’s work — and there are plans to tap still more data files.

Some believe the Statistics Act is also a problem. Alex Usher, president of Higher Education Strategy Associates, says the law is a straitjacket that requires the agency to conduct a census of agriculture every five years. It means, he says, that it continues to spend a lot of time and money counting crops and livestock even though the economy has changed dramatically in the three decades since the legislation was drafted.

Canada has little data on college tuition or enrolment, he says, even though the information would help the country sharpen its education system and train people for the right jobs. Others complain the agency doesn’t produce enough data about energy exports, job vacancies or business investment.

For its part, StatsCan has committed itself to improving statistics that measure the environment, energy, household wealth and the aging population, while also improving the reliability of data for small geographic areas.

Smith says the agency must strike the right balance between fiscal restraint and an unquenchable thirst for data while also ensuring that Canadians don’t grow tired of their questions.

“There’s no limit to the demand for information,” he says. “If you gave me a budget three times the size of the one I have now, I still couldn’t satisfy everybody. The number of things we could measure is huge. But we’re restricted not only by limited dollars, but also by the willingness of Canadians and Canadian businesses and organizations to respond to surveys. We can’t multiply these things without limit.”

aduffy@ottawacitizen.com

b.gif


查看原文...
 
后退
顶部