Maclean's annual rankings
We evaluated 47 Canadian universities, by the numbers and under the microscope
MARY DWYER | Nov 02, 2006
See also:
Surveying the professionals
The reputational survey seeks opinions on university performance
Now in its 16th year, the annual Maclean's rankings assess Canadian universities on a diverse range of factors from class sizes and average entering grades to spending, library volumes and faculty success in obtaining national research grants. The focus is on the undergraduate experience, and the intent is to offer an overview of the quality of instruction and services available to students at public universities across the country.
Continued Below
Maclean's places universities in one of three categories, recognizing the difference in levels of research funding, the diversity of offerings, and the range of graduate and professional programs. Primarily Undergraduate universities are largely focused on undergraduate education, with relatively few graduate programs. Those in the Comprehensive category have a significant amount of research activity and a wide range of programs at the undergraduate and graduate levels, including professional degrees. Medical-Doctoral universities offer a broad range of Ph.D. programs and research. In addition, all universities in this category have medical schools, which sets them apart in terms of the size of research grants.
In each category, Maclean's ranks the institutions on a range of factors -- or performance indicators -- in six broad areas (weightings are in parentheses). In total, Primarily Undergraduate universities are ranked on 22 performance measures, Comprehensive universities on 23 and Medical-Doctoral universities on 24. In reporting to Maclean's, universities include data from all federated and affiliated institutions. The magazine does not rank schools with fewer than 1,000 full-time students or those that are restrictive due to a religious or specialized mission.
The ranking process begins in the spring when a detailed questionnaire is circulated to the universities. At the same time, Maclean's collects information on dozens of student and faculty awards from 45 administering agencies. Thousands of reputational surveys are also sent to high school principals and guidance counsellors, university officials, heads of organizations, CEOs and corporate recruiters across the country, asking for their views on quality and innovation at Canadian universities.
This year, the rankings were marked with controversy as 26 universities refused to make public performance indicators that they had released, as a matter of course, for the past 15 years. On Aug. 14 -- two weeks before the deadline for submitting completed surveys -- 11 universities informed Maclean's that they would not do so, citing concerns about the ranking methodology and some of the measures used, and claiming that over the years these concerns had "gone largely unaddressed." Fifteen other universities shortly followed suit. The rankings have always been contentious, however, since they place universities under the microscope. During the mid-1990s, a number of universities boycotted the exercise.
From its inception, Maclean's has consulted with academic experts about the design, composition and methodology of the rankings. After the inaugural rankings were released in 1991, Maclean's responded to criticism from the university community by overhauling the survey. For several months in 1992, editors met with educational experts and representatives, including university presidents, institutional analysts and such organizations as the Association of Universities and Colleges of Canada and its regional counterparts, asking them for input on what factors to consider when assessing how well a university is doing its job. New indicators were developed and the universities were divided into the three peer-grouped categories. Over the next several years, Maclean's continued to respond to concerns and suggestions from the post-secondary sector by adding three new indicators -- total library holdings, international first-year students and retention rate -- as well as significantly revising the class size indicators. Guidelines were refined, definitions were expanded, and the reputational survey increased the range and number of groups canvassed while boosting regional representation. Maclean's continues the mandate it established 16 years ago: to provide basic, essential information in a comprehensive package to help students choose the university that best suits their needs.
In calculating the rankings this year, universities that completed a questionnaire were processed with data being collated and checked in the usual manner. For universities that refused to complete a questionnaire, Maclean's obtained public data wherever possible for the most recently available fiscal year. For the two research grant indicators, data for the fiscal year 2005-2006 was received directly from the three major federal granting agencies: the Social Sciences and Humanities Research Council of Canada, the Natural Sciences and Engineering Research Council of Canada, and the Canadian Institutes of Health Research. Statistics Canada data was used for the five financial indicators -- operating budget, spending on student services, scholarships and bursaries, library expenses and acquisitions. The Canadian Association of Research Libraries and its regional counterparts provided figures used for the library holdings indicators. All financial and library figures are for fiscal year 2004-2005. As in previous years, Maclean's conducted the reputational survey, and collected student and faculty awards for all universities. Any outstanding data for non-participating universities was taken from last year's questionnaires.
The 26 universities that did not complete a ranking questionnaire this year are: University of Alberta, Brandon University, University of British Columbia, Brock University, University of Calgary, Carleton University, Concordia University, Dalhousie University, Lakehead University, Laurentian University, University of Lethbridge, the University of Manitoba, McMaster University, Université de Moncton, Université de Montréal, University of New Brunswick, Nipissing University, University of Ottawa, Queen's University, Ryerson University, Simon Fraser University, University of Toronto, Trent University, the University of Western Ontario, University of Windsor and York University.
STUDENT BODY (22% to 23% of final score) Students are enriched by the calibre of their peers. For that reason, Maclean's collects incoming students' average high school grades (11%), and the proportion of those with averages of 75 per cent or higher (2%). This count includes only those students whose secondary school averages or CÉGEP scores served as the basis of admission. No conversion formula is applied to incoming grade averages to adjust for provincial differences or varying admission policies, although CÉGEP grades are converted from an R score to a percentage grade. As well, it should be noted that certain universities, to enhance accessibility, accept students with lower grades.
As a measure of drawing power, we count the proportion of out-of-province students in the first-year undergraduate class (1.5%). The percentage of international students in the first-year undergraduate class (0.5%) is measured as well, acknowledging the benefits such diversity brings to the classroom. The percentage of international students at the graduate level (1%) is also measured for Comprehensive and Medical-Doctoral universities.
In taking stock of retention rates (2%), Maclean's asks for the percentage of first-year students who return in second year, either full-time or part-time. While many factors can affect a student's choice not to return -- personal considerations, or a decision to transfer to a program unavailable at their home university -- student retention, on the whole, reflects a university's success in keeping its students on course. Maclean's also measures graduation rates (2%): the percentage of full-time undergraduates in their second year who go on to graduate from the institution within one year of the expected time period.
In addition, Maclean's collects data on the success of the student body at winning national academic awards (3%) over a five-year period. The list of roughly 40 fellowships and prizes includes such prestigious awards as the Rhodes Scholarships, the Fulbright Awards, as well as scholarships from professional associations and the three federal granting agencies.
CLASSES (17% to 18%) The rankings take stock of the entire distribution of class sizes at the first- and second-year levels (7.5% for Primarily Undergraduate universities, 7% for the other two categories), as well as the third- and fourth-year levels (7.5% for the Primarily Undergraduate category, 7% for the others). Maclean's measures the percentage of students in classes in each of the following class-size ranges: 1 to 25; 26 to 50; 51 to 100; 101 to 250; 251 to 500; 501 and higher.
Maclean's also ranks universities on the percentage of first-year classes taught by tenured and tenure-track professors (3%).
FACULTY (17%) In assessing the calibre of faculty, Maclean's calculates the percentage of faculty with Ph.D.'s (3%), and the number who win national awards, including the distinguished Killam, Molson and Steacie prizes, the Royal Society of Canada awards, the 3M Teaching Fellowships and almost 40 others (3%). In addition, the magazine measures the success of eligible faculty in securing research grants from each of the three major federal granting agencies: Social Sciences and Humanities Research Council of Canada, Natural Sciences and Engineering Research Council of Canada, and the Canadian Institutes of Health Research. Maclean's takes into account both the number and the dollar value received last year. Social sciences and humanities grants (5.5%) and medical/science grants (5.5%) were tallied as separate indicators.
FINANCES (12%) This section examines the amount of money available for current expenses per weighted full-time-equivalent student (3.3%), as well as the percentage of the budget spent on student services (4.3%) and scholarships and bursaries (4.3%). Funds used to pay off debt are excluded from the analysis.
LIBRARY (12%) This section assesses the breadth and currency of the collection. Universities received points for the number of volumes and volume equivalents per number of full-time-equivalent students (4% for Primarily Undergraduate and Comprehensive, 3% for Medical-Doctoral). The total holdings measurement was used in the Medical-Doctoral category (1%), acknowledging the importance of extensive on-campus collections in those universities.
As well, Maclean's measured the percentage of a university's operating budget that was allocated to library services (4%) and the percentage of the library budget spent on updating the collection (4%). In acknowledging a shift from the traditional library model -- books on shelves -- to an electronic access model, Maclean's captures spending on electronic resources in both the library expenses and acquisitions measurements.
REPUTATION (19%) This section reflects a university's reputation in the community at large, as well as with its own graduates. For the reputational survey (16%), respondents rated the universities in three categories: Highest Quality, Most Innovative and Leaders of Tomorrow. Best Overall represents the sum of the scores.
When looking at alumni support, institutions received points for the percentage of alumni giving gifts -- rather than the total value of gifts -- over the past five years (3%). M
The Maclean's ranking takes a measure of the undergraduate experience, comparing universities in three peer groupings. Those in the Medical-Doctoral category have a broad range of Ph.D. programs and research, as well as medical schools.