To what extent are men and women with the same job being compensated differently? That was the question that sparked this investigation two and a half years ago. From there, as so often happens, the reporting pulled the project in new directions.
Early analysis showed that while the wage gap was definitely still a problem, there was an even bigger issue: Compared to men, there just weren’t many women in high-paying jobs. Women – especially women of colour – were not well represented in decision-making roles. Women were under-represented not just at the helm of organizations, but on executive teams, in management positions and in six-figure jobs in general. We realized a more accurate framing of the inequities between men and women at work would be the power gap.
What is the Power Gap?
For more than 100 years, the discussion about gender discrimination in the work force has largely focused on the wage gap between men and women. According to Statistics Canada’s most recent figures, that gap currently stands at 13 per cent (and hasn’t budged in any significant way in nearly a decade.) But money, while important, is not the only measure of workplace gender imbalance. Overall representation, particularly in high-paying jobs and decision-making roles, is just as important. In The Globe’s analysis of public-sector salary data – which only includes six-figure earners – it was not uncommon for men and women to be paid similar salaries. However, there were often two, three or even five times more men.
The Power Gap is a way to assess where women stand in the workplace using multiple criteria: How much are women earning compared to men? How many women are employed at the organization? How are those women distributed across the work force’s pay scale? (For example, are they mainly found in lower salary bands?) How many women are in leadership or management positions? And how many senior women are women of colour? There isn’t one statistic that can capture all of this. These issues are complicated and are best understood when viewed through numerous lenses.
How did you determine gender?
Public-sector salary disclosure lists don’t include gender. Most – but not all – include a first and last name, sometimes a job title, and a compensation figure. For privacy reasons, public organizations won’t disclose the gender of employees. But 90 per cent of names in Canada are associated with one gender 95 per cent of the time.
The Globe and Mail commissioned Statistics Canada to complete a gender analysis for this project. The Globe sent the federal statistics agency a list of tens of thousands of first names, culled from public-salary disclosure lists. Statistics Canada then assigned a gender probability to that name in 5-point increments. For example: Michael is a male name 95 to 100 per cent of the time. (Quebec was analyzed separately from the rest of the country because of potential gender-probability differences.) Any exceptionally rare names – names that belong to 20 or fewer individuals – were excluded from the analysis. The cost of this analysis was $6,000.
To provide the most certainty possible, we set our threshold at that 95-per-cent mark.
What did you do when it was not possible to determine gender?
Only names that met that high bar are included in our analysis. There were 89,423 names in our dataset, and we could not determine the gender of 11 per cent of them. Calculations on the stated gender probabilities of those “below threshold” names suggest the gender split is close to even. What wasn’t certain was how those unknown names could affect the results of an individual organization. The Globe’s data-science team helped with this.
On our website, we have included a chart that breaks down the gender divide at different salary bands for each entity. Entities with more than 100 six-figure earners were assessed using 10 levels, and entities with fewer than that were assessed using five. Using a data-science analysis, we identified areas with the highest potential for volatility. For example, let’s say a salary band has 10 employees: four men, four women, two of unknown gender. Those two individuals are named Allison (who has a 90- to 95-per-cent chance of being a woman) and Kamal (who has a 90- to 95-per-cent chance of being male). The system would recognize this salary band as having a low chance of volatility. But let’s say there was a band with 10 employees: two women and six men, and the unknown two are named Courtney and Elisha (both of whom have a 75- to 80-per-cent chance of being a woman). This would be flagged because there is a bigger risk of the representation gap swinging from between 40 to 60 per cent and 20 to 80 per cent.
To resolve some of this volatility, The Globe researched the gender of 1,140 people whose names were in the unknown category. (We contacted hundreds of these people by email or phone but also relied on pronouns used in official biographies and mainstream media coverage. We did not rely on photographs to assign gender.) In the end, unknown names caused less than 5 per cent volatility in 81 per cent of the 1,721 salary bands. There were 21 bands that had more than 10-per-cent volatility, which we have flagged. We have also disclosed the number of unknown names in every salary band in our data search tool. Additionally, we made the decision to remove three entities – eHealth Ontario, Health Shared Services Ontario and HealthForceOntario Marketing and Recruitment Agency – because of ongoing volatility.
Why did you only collect data for the public sector and for people who earn at least $100,000?
Easy answer: It’s all that’s available.
The initial goal of this project was to dig deeper into the wage gap. The overall disparity is 13 per cent. This is the average hourly pay of all working men compared with all working women. But there hasn’t been a wide-scale study (at least that we could find) of the difference between what men and women in the same job are being paid. The problem with trying to investigate this is that salaries are secret – except for one segment of the work force. Most provinces (excluding Prince Edward Island and New Brunswick, the territories, and the federal government) have public-sector salary-disclosure legislation on the books. This means employees of government entities who earn above a certain threshold, usually $100,000, have their name and compensation published once a year. Most records also include job titles. (Experts who reviewed The Globe’s data, such as economist Armine Yalnizyan, said the situation in the private sector is likely worse than the public sector, in part, because there is less scrutiny.)
Private companies listed on the TSX are also required to disclose compensation for top executives. We also analyzed this information.
In an ideal world, it would have been possible to evaluate the entire work force of the entities included in our database, rather than just the six-figure earners. We know that women are more likely to be concentrated in lower-paying jobs. We also would have liked to assess private business, particularly large Canadian corporations with strong equity policies. Unfortunately, unlike countries such as the United Kingdom, which requires companies with more than 250 employees to publish pay-gap statistics each year, salary transparency is a black hole in Canada.
Why did you decide to focus on universities, cities, provincial governments and government-owned corporations?
The very first idea for this project came in February, 2018. At the time, our work was limited to figuring out the parameters of legislation in each jurisdiction. After gathering some preliminary information, we decided to run a pilot of what this investigation might look like based on early data from a few provinces. That was done between September and November, 2018. At that point, we realized it was not going to be possible to collect every piece of disclosure in the country. For one, provinces could not provide us with a list of all the entities that are subject to the legislation, only the types of entities that are, such as universities, municipalities and so on. But even if we could obtain such a list, there would likely be many thousands of entities that would need to be individually contacted. In the early days of this project, The Globe actually did collect information from hundreds of additional places. Much of the time, especially with smaller entities, we needed to manually enter the data into a spreadsheet. Sometimes places wrote out the information in an email. Other times, it was a PDF or an image file in a freedom-of-information request. It would have taken years.
The other problem is that different provinces and entities had different disclosure policies. Not everyone included first names. Some, especially in British Columbia and the Prairies, just used first initials. Others didn’t include job titles. This made it difficult to assess what we were looking at. (See the section on municipal police and firefighters under “Why is the data three years old?”) Health care was an area ripe for exploration, but many entities did not include job titles. What proportion of female staff were nurses, doctors or administrators? Where did they fit in the leadership hierarchy? Were female doctors and male doctors earning the same? We couldn’t answer any of these questions without job titles.
In assessing what was available and the quality of that data, four areas stuck out as being the most complete, the most comparable across jurisdictions and having the most impact on Canadians’ lives: universities, the public service in municipalities and provincial governments, plus government and Crown corporations.
In total, we have incorporated data from 244 entities in eight provinces. Detailed information was available in 171 cases. (Apart from the City of Montreal, entities in Quebec only provided data for the most senior staff. Sometimes, as was the case with most corporations, this was interpreted as only one or two people. In other cases, such as the municipal governments, each list had a few dozen. Because there was no consistency around disclosure, each organization can only be assessed on its own. As a result, they were not included in the broad salary analysis. It was possible to include them in the “power positions” assessment.)
Why is the data three years old?
The Globe launched this investigation in September, 2018. It took more than a year to gather and transfer all the different disclosure lists. Ontario and (for the most part) Alberta neatly bundle their disclosures into large online spreadsheets, but other jurisdictions do not. In many cases, the information stays with the individual organization, and some don’t even post it publicly, although they will release it upon request. The date of disclosure varies too. Some release their previous year’s numbers in the spring. Others in the late summer. Sometimes the data is from a single calendar year (January to December) and some manage their fiscal from April to the following March. When reports came back, they were often returned as PDFs that had to be manually transferred (in some but not all cases, it was possible to use software to transfer the data, but every line – and there were sometimes thousands – still needed to be checked for accuracy).
By late 2019, we could start dealing with the problems. For example, some places switched the first and last names for some employees. Others didn’t include first names to which we could assign gender. When this came up, we tried to negotiate with the entity to release the first names or provide us with an employee directory so we could match them up. In some cases, we requested this information through freedom of information. (Most of the time, it was not possible to reliably reconcile the lists.)
As we dug into the data, we noticed that many municipalities across the country had huge representation gaps between men and women. When we looked into why, we realized that some places lumped police and firefighters in with their municipal public service. These are well-paid, male-dominated jobs. In some cities, about half of their total disclosure list was made up of fire and police. We sent additional requests to municipalities to send us lists of all emergency-services employees so we could filter them out. Some cities didn’t provide job titles, so it wasn’t possible to identify these workers. After trying to obtain those job titles, we eventually concluded these cities had to be removed. (The Globe initially collected salary data from hundreds of cities and towns in Canada, but because of the police and firefighter problem, we decided to focus on the 25 largest urban centres that disclose full information. It was just not feasible to manually remove emergency services everywhere.)
After we finished transferring and cleaning the data we set about dealing with volatility from unknown names. (See “What did you do when it was not possible to determine gender?”)
Why did you not include race in all of your data analyses?
We recognize that this is a major hole in our data. Black women, Indigenous women and women of colour face significant additional barriers in the workplace. Women of colour earn 85 cents for every dollar that white women make, according to a report by Sheila Block and Grace-Edward Galabuzi for the Canadian Centre for Policy Alternatives.
Names are not a reliable indicator of race, so it was not possible to capture this element. We did contact 357 women who ran organizations or who fell in the top 1 percentile of earners to inquire about how they identify racially. If we did not receive a response, we used photos included with the person’s biography or in mainstream media coverage to make a determination on whether the individual was white or racialized. Two people assessed each photo.
What about people who don’t identify as a man or woman?
This year, Statistics Canada is going to add a third gender option to the census. Currently, there is very little statistical information available about the transgender community and gender minority groups, such as non-binary and two-spirit. It was not possible to capture this element through the data. It’s been well-documented that these groups face additional systemic barriers in the workplace and we will be writing about some of these issues in future stories.
How did you handle the fact that different provinces have different disclosure thresholds?
While some provinces have different bars for public disclosure (in Manitoba, for instance, it was $50,000, though it has since increased to $75,000), the majority of provinces have a threshold of $100,000. In provinces that disclosed below this, we only captured employees who crossed the six-figure mark. So the base threshold almost everywhere is $100,000. The only province where this wasn’t the case was Alberta, where the threshold for provincial employees was $107,071, while crowns and universities disclosed at $127,765. (No large cities disclosed in Alberta.) This has the potential to skew the results slightly male, since other data suggest the lowest salary bands have more women. To analyze the impact this disparity might have, The Globe adjusted Ontario and British Columbia’s disclosure to Alberta’s standards. The result was that the overall numbers moved very little. The swing was one to four percentage points, except when comparing Ontario’s Crown corporations. In that case, the swing was 6 per cent. It’s worth noting that the cost of living and average salaries also fluctuate between provinces. A salary of $100,000 in St. John’s is not the same as it would be for a person living in Vancouver.
How did you pick “power positions”?
One of the ways we wanted to assess the power gap was to look at representation and salaries among the most important and senior decision-makers within an institution.
Whenever possible, we identified an entity’s “power positions” using 2017 executive biographies or organizational charts that had been posted to their websites — which we found using the Internet Archive — or that were included in annual reports. With organizational charts, we focused on managers who had a direct line to the president (or equivalent). When this was not possible, we assessed each entity with a set of job titles commonly found in the highest earning bracket. Using one set of criteria for everyone didn’t work, so adjustments were made when necessary. For example, at some universities, the “associate vice-president” is part of a very small executive team, but at other schools, including this position would mean the university had more than 50 senior executives.
These were the base keywords we used:
- Municipalities: city manager, chief administrative officer, deputy city manager, commissioner, director
- Provincial governments: deputy minister, associate deputy minister, assistant deputy minister; sous‐ministre, sous‐ministre associée, sous‐ministre adjointe (plus the masculine forms)
- Publicly owned corporations: president, chief executive officer, chief financial officer, executive vice-president, vice-president; présidente, vice présidente, directrice (plus the masculine forms)
- Universities: president, vice-president, provost, vice-provost; rectrice, vice principale, vice rectrice, secrétaire générale (plus the masculine forms)
How did you choose which management job titles to compare?
There is a wide range of sectors covered off in The Globe’s dataset, from energy and culture to transportation and infrastructure, from finance and cannabis to tourism and government – and the list goes on. But in most entities, six job titles came up repeatedly: supervisor, manager, senior manager, director, executive director and vice-president.
The Globe pulled out these titles to assess the gender divide at different management levels. The goal was to examine people in the same role. In some cases, a person was a director, but also something else. For example, “executive director and chief information officer.” In this case, the person would not have been included. The exception was for vice-presidents, who very often had additional roles (such as CFO), and university deans, who were also often listed as professors. (In keeping with our methodology, professors who held additional titles, such as “department chair” or “Canada research chair,” were not lumped in with regular full professors.)
How did you choose which provincial government departments to compare?
When we initially ran the numbers for provincial governments, we were surprised at how evenly split the work force appeared, given what we had seen in other areas. But once we started to investigate further, we realized that there is significant gender segregation at the ministry level. Comparing is a bit challenging because there is no uniform process for naming ministries. Moreover, the names are constantly changing, and different departments are being grouped together. To land at our final list, we pulled out the 12 most common, although things never quite lined up perfectly. (For example, Alberta’s “Culture and Tourism,” Ontario’s “Tourism, Culture and Sport” and Nova Scotia’s “Department of Communities, Culture and Heritage” are all filed under “Culture.”)
Although Manitoba has salary-disclosure legislation and many entities within the province are included in our project, the provincial government itself could not be analyzed because the lists did not include first names. Quebec is only included in the assessment of power-position representation because only the most senior officials are disclosed in that province.
A note about the numbers
In this project, we reference various wage gaps within the different pillars (cities, universities, provincial governments and government-owned corporations). This is the average among all individuals within each of those areas. For example, female executives at corporations made 9 per cent less than male executives. In the data search tool, you can see the wage gap at individual entities and among power positions.
Also, any entity that had fewer than 10 employees was excluded from our analysis and is not included in this project.
What are the caveats?
This data is not perfect.
Public-sector disclosure rules vary across borders and between jurisdictions. It’s not always clear what the listed compensation includes. Some places include annual salary; others use any amount of money paid to that employee that year, including expense reimbursements. For this reason, the best way to assess The Globe’s data is to evaluate the situation at individual entities – since all the employees will be operating under the same disclosure rules – and to take in the broad trends.
Sunshine List data is a record of what the person was paid that year, not what that person’s annual salary is – and that’s not always the same thing. For example, The Globe noticed a woman at the executive level had earned double what her male counterparts were being paid. Upon further investigation, it turned out she had retired the previous year, and the compensation was a payout. Whenever there were significant gaps like this, The Globe dug deeper. Other times, a person’s salary was low because they took on a role part-way through the year. There are retirements, maternity leaves and sick leaves. Overtime can also be included.
Additionally, as mentioned above, the disclosure thresholds vary between provinces, as does the cost of living. An annual salary of $100,000 means different things in different parts of the country. In Alberta, where the threshold is higher, it’s likely that the numbers skew a few percentage points more male, since we know from other jurisdictions that there are a disproportionate number of women in the lowest salary bands.
There is also risk that the information provided by the entities contained errors. For example, we found instances where the first and last name had been flipped, as well as typos. To examine gender, The Globe married public-sector salary data with information on the gender probability of first names. To be included in our analysis, a name needed to be associated with men or women between 95 and 100 per cent of the time. Within that range, there is an element of uncertainty.
Ultimately, there were many moments during this process where we had to make a judgement call. In those instances, our guiding principle was to err on the side of the least controversial decision. If there was ever any doubt about which course to choose, we went with the one that showed the most gender diversity (in other words, the one that made the organization look best).
We will continue to update our dataset to make it as accurate as possible. (For example, hundreds of people contacted by The Globe about their gender or race did not respond to our inquiries, but they may after publication. Dozens of organizations also did not respond to questions. Others have responded and then updated their information later on.)
The Globe arrived at its findings by using a programming language and set of statistical tools that allowed for analyses to be tweaked and rerun continuously during the investigation. All of the code developed for the analysis was verified by a data journalist, and all of the methodology were verified by a data scientist, both of whom were not previously involved with the project.
Robyn Doolittle
Chen Wang