2013 CWILA Count Methods and Results

By Judith Scholes

Since CWILA began producing an annual Count of book reviews in Canada, our major objective has been to quantify the existing but often silent gender inequities within Canadian critical culture. Now in its third year, the Count has become an integral part of the larger critiques CWILA is engaged in, as an organization dedicated to fostering equitable literary culture in Canada. Our numbers are facts. They reveal concretely where equitable access and representation exists and where it can be improved, which can be a powerful impetus for change. The real change, however, does not rest in our numbers, but in how we move from these quantifications into the knotty questions, unfamiliar perspectives, fresh conversations, and ultimately into positive change.

It is in this spirit that we launch the Count this year. In my second year as Count Director, I wanted to ensure the Count represented critical culture in Canada as accurately as possible. To this end we made key changes to the scope of the Count, our Count process, and our gender data representation, while maintaining the metrics from the 2012 Count. What follows is a report on how the Count was conducted, what we might need to change as we move forward and, of course, the 2013 numbers. 

Scope of the 2013 Count

For the 2013 Count, CWILA added six new publications to our list, including four French-language publications (Le Devoir, Lettres Québécoises, Liberté: Art et Politique, Nuit Blanche), one from Eastern Canada (The Chronicle Herald), and one from Ontario (Toronto Star).

Literary Journals, Magazines, and Online Publications {24}

Antigonish Review, Arc Poetry Magazine, Briarpatch, Brick, Broken Pencil, Canadian Literature, Canadian Notes and Queries, Event Magazine, Fiddlehead, Geist, Lemon Hound, Lettres Québécoises, Liberté: Art et Politique, Literary Review of Canada, Maisonneuve, The Malahat Review, Nuit Blanche, Prism, Quill & Quire, rabble.ca, subTerrain, This Magazine, Walrus

Metropolitan Newspapers {5}

The Chronicle Herald, Le Devoir, Toronto Star, Vancouver Sun, Winnipeg Free Press

National Newspapers  {2}

National Post, Globe and Mail                                                                  

The shortlist of new publications was chosen by CWILA’s Executive Director, Sheila Giffen, and myself and approved by the CWILA Board of Directors. In making our selections we considered regional variability, circulation and readership of the newspaper or journal, size of reviews section, as well as breadth and diversity of books reviewed. The additions brought the total number of publications included in the Count to 31, and increased the total number of reviews counted by 82%, from 3092 in 2012 to 5613 in 2013.

The 2013 Count was completed by many dedicated people. As Count Director, I recruited and coordinated volunteers, led development of 2013 Count methodology and research questions, managed outreach and communications with English-language publications involved in the 2013 Count, as well as managed data collection, verification, and statistical analysis and representation. CWILA Executive Director Sheila Giffen established and managed Count deadlines, led the selection committee for French-language publication inclusions, managed outreach and communications with French-language publications, led development of a communications plan and media outreach, and coordinated interviews editors. CWILA Chair of the Board Erin Wunker oversaw all phases of Count production, the implementation of a communications plan and media outreach, as well as coordinated the production and launch of materials and editorial content that contextualized the Count data, including her essay introducing the 2013 Count. Evelyne Ledoux-Beaugrand produced analysis and context for the French-language numbers, with translation by Bronwyn Haslam. Tina Northrup, one of CWILA’s Interviews Editors led efforts to restructure CWILA’s gender categories for the 2013 Count in consultation with Morgan M. Page, Ivan E. Coyote, Mattilda Bernstein Sycamore, and Nathanaël (N) Stephens. CWILA also benefited from the UBC Arts Internship program, through which we were able to hire two UBC undergraduate students, Sonja Cvoric, who served as CWILA Count Assistant and reported to me, and Susan Bahaduri who served as CWILA Communications Assistant and reported to the Executive Director.

The labour required to count 5613 reviews necessitated a large team of volunteers, which included members fluent in French. In total, CWILA recruited 42 extraordinary volunteers from among its membership to meet the needs of this year’s expanded Count. The names of our volunteers are listed in the acknowledgments below.

To help us through the task of collecting review data, we also invited all publications included to provide a record of their reviews using our Count spreadsheet. This year, 12 of the publications we included (Arc, Briarpatch, Brick, Broken Pencil, Canadian Literature, Event, Geist, Malahat Review, Matrix, rabble.casubTerrain, This Magazine) offered to help us in this way.

Finally, the 2013 Count was funded by a Canada Council for the Arts Literary Arts Promotions Project Grant and via the UBC-CWILA Research Network, a Thematic Research Network Grant from the Institute for Gender, Race, Sexuality and Social Justice at the University of British Columbia, and from the generous support of our membership and private donors. 

2013 Count Process

Following the methodology established in the 2011 CWILA Count, and further developed in the 2012 Count, the 2013 CWILA Count used Excel spreadsheets to collect multiple data on book reviews in Canada, including title, publisher, and genre of books reviewed; name, Canadian identity, and gender of authors; and name and gender of reviewers.

In the 2013 Count, as in previous years, each book review was recorded as a single entry in our data sheets, with its associated book, author, and reviewer information. Each entry refers to a single book reviewed. This means that when a single newspaper article reviewed multiple books, each book was counted as a separate review in our data sheets. While we recorded reviews of any length, from one paragraph to multiple pages, we also restricted our definition of book review to critical appraisals of works already published and released for sale. Editorial opinion about anticipated releases, unsigned roundups of related books annexed to reviews, news about book releases or awards, and author interviews were not counted as reviews.

The 2013 CWILA Count proceeded in five phases:

  1. Count Setup

Count preparations included the creation of the 2013 Count data collection spreadsheet and directions, integrating new objectives, methodology, and research questions. We contacted each publication involved in our Count, inviting them to provide an in-house count using our data collection sheet. We also recruited 42 volunteers from among the CWILA membership and established a master count assignment sheet for managing volunteer activity. Counting assignments were split up into blocks of 5 hours of labour or less, or around an estimated 100-150 reviews. In some cases an entire publication fit into one count block, while other publications with reviews in excess of 150 reviews, such as the Globe and Mail, filled up to 6 count blocks.

  1. The Count

Each count block was assigned to volunteers working independently. Volunteers and participating publications were given a spreadsheet with instructions for entering book review data from either print or online publications. The use of digital versions of newspapers in the Count is discussed in a separate section below.

Volunteers were instructed to record each review separately, and to count national identity once per author, and count gender once per author and per reviewer. Thus, for co-authored books or reviews, each author and reviewer was counted. Translators and illustrators were counted as authors. Author and reviewer gender was determined by checking the pronoun use in the review, or by checking biographies or interviews recently published online. Volunteers were instructed never to assume gender based on names or photos.

  1. Count Verification Step 1: Reviews & Book Data

Verification of the count was split up into two steps for efficiency. In the first step, count blocks were collected and checked by me, then compiled into separate spreadsheets organized by publication. New volunteers were assigned to verify reviews by checking that each book review in the given publication had been correctly accounted for in the spreadsheet.

  1. Count Verification Step 2: Authors & Reviewers Data

Verified publication sheets were collected, standardized, and compiled into a master data sheet. As Count Director, I sorted the master data sheet by author and reviewer name and split up the names among a final verification team, which included me, the Executive Director, CWILA Count and Communications Assistants, and graduate students from the department of English and the Institute for Gender, Race, Sexuality, and Social Justice at the University of British Columbia. To minimize the possibility of misidentification, this team verified the gender and Canadian identity of each author counted, and the gender of each reviewer counted, by checking pronoun use in author biographies and interviews. Any confusion was flagged and sent to the Count Director to investigate further. Each writer who was possibly positioned outside of the gender binary was double-checked. The entire count was further checked for names of known trans, genderqueer, and non-binary authors.

  1. Statistical Analysis

From the available verified data, statistics were generated on gender, language, Canadian authors, genre, and publishers for book reviews counted. Data charts in percentages were produced representing reviews and books by gender, language and by publication. Finally, pie charts and bar graphs representing the data were created in Excel. 

Counting Reviews in Newspapers

Four of the publications included in our Count are online-based or have online reviews sections, and thus we count and verify these entirely online. The remaining 27 publications are print-based, and while many of these have online components, we use the print version to verify our count, whenever possible. Given our current method of counting reviews published in the previous year, however, counting print versions of daily newspapers is not yet possible.

Because we do not yet have the resources to consult print copies of the newspaper and count reviews as they are published (a real-time print Count will be implemented for the 2015 Count), we have had to rely on imperfect digital means to find and record reviews for all Counts thus far. Though all of the newspapers included in the 2013 Count publish reviews online simultaneously with the print version, only two (National Post and Le Devoir) keep an easily accessible blog-style archive of all reviews published that goes back several years and poses few challenges for our counting purposes. In contrast, reviews published online by five of the seven newpapers in our Count (Globe and Mail, Toronto Star, Chronicle Herald, Winnipeg Free Press, and the Vancouver Sun) are partly inaccessible or unavailable (e.g. only reviews from the previous 6 months are listed), do not have a devoted space in the website, or can only be found using a global site search based on keywords that return only partial results, or by reviewer name. The best solution, overall, has been to use a third-party news archive database to collect all reviews published. Unfortunately, these databases often rely on newspapers to furnish appropriate keywords and tags for their articles, for ease of searching. In most cases, however, this work is incomplete and cannot give us the full picture of reviews published in a given year. We therefore must use a combination of methods to ensure we are getting the most complete picture possible, including third party databases, newspaper website archives, and editor consultation, when possible. Our process was as follows:

  1. Search relevant newspaper archives via third-party database(s) using multiple, broad keywords to collect and record book reviews published in a given year. This furnished on average, about half to two-thirds of our count.
  1. Sort all reviews counted in Step 1 by reviewer name and search archives on newspaper website for list of all reviews published by reviewer. This filled in several reviews missing from our initial keyword search.
  1. Consult directly with books editors, if available to verify the number of reviews published in the paper. In some cases editors were able to assess our total number of counted reviews (Globe and Mail; Vancouver Sun) and indicate any glaring omissions, but due to time constraints, no editor was available to check that all reviews were accounted for. In future Counts we are hoping to have editors confirm numbers on a monthly basis.

While it is certainly possible that we missed reviews published in 2013 in a given newspaper, we have done our best to collect them all. In any case, based on the number of reviews we are collecting, we are well within the range of making statistically significant claims. In fact, given the high number of reviews counted in these papers, a random handful of missing reviews would have no significant impact on the overall percentages we are drawing from the data. In other words, we are still presenting an empirically sound and accurate picture of the reviewing practices in the paper. For this reason, among others, we opt to express our claims with as much transparency as possible to avoid any misrepresentation —e.g. “[x%] of the total reviews counted in [x publication] were written by women”

Gender Data Representation

CWILA faced two pressing questions during the 2013 Count: What has CWILA done to make sure that writers’ gender identities have been identified correctly? How is CWILA making space for trans writers in the Count?

Building on our objective for the 2012 Count to develop a more nuanced method for collecting gender data, this year we have modified the way we collect and represent gender data for authors and reviewers who identify as trans or who identify outside the gender binary.

In the 2013 Count we counted 1 reviewer and 3 authors who do not identify as either male or female, each of whom we are including in a “non-binary” category. We also counted 1 reviewer who identifies as a trans woman, whom we are including within the “female” gender category, while signalling that the category includes a trans woman.

We have taken all possible steps to minimize any misidentification of writers who identify as trans or who identify outside the gender binary, though this possibility still remains. Before, during, and after verification of gender for the 2013 Count, the verification team was acutely aware of the importance of collecting gender data as sensitively as possible, and we debriefed after the verification to discuss problems and possible solutions arising from our methods. As a rule, the writers we included in the female and male category were identified as “female” or “male” explicitly and repeatedly online and no trans or non-binary identity was associated with them. We do not currently possess the resources to ask all writers counted to self-identify, but we will continue to work toward this.

In addition to amending our gender categories, we have chosen to expand our gender classification by adding multi-gender “Co-Authored” categories where relevant. As the only multi-gender co-authors collected in the 2013 Count were groups with at least one male and one female, and no co-authors included writers who identify as trans or who identify outside the gender binary, we only included “Male and Female Co-Authored” as a category in this instance. To be clear, “Female-Authored” includes only books or reviews by women or trans women authors and reviewers either singly or in collaboration; “Male-Authored” includes only books or reviews by men or trans men authors and reviewers either singly or in collaboration; and “Non-Binary-Authored” includes only books or reviews by authors and reviewers who identify outside the gender binary. Finally, books counted as “Unknown Collective” include only those books that have no author(s) associated with them, as in the case of a book published by a corporate collective.

Further Considerations

Counting for Race

The question of how to account for race in our annual Count has been a site of ongoing conversation. Though methodological challenges associated with collecting data on the racial or ethnic backgrounds of writers has not yet been resolved, CWILA is addressing the question of race discursively. Thus we are producing essays, interviews, and talks that address issues of race and race-based discrimination in Canadian critical culture. In March of 2014, the CWILA-UBC Research Network hosted a panel discussion, “CWILA and the Challenge of Counting for Race,” which was filmed and posted online. The panel, which included author Madeleine Thien, along with Associate Professors of English Laura Moss and Mary Chapman, discussed the benefits and risks of calculating the distribution of reviews based on ethnic or racial backgrounds of authors.

Establishing a Baseline of Total Published Books

One of the most frequently asked questions following the launch of our Count data, is whether or not we have been able to establish a baseline number of total books published during the Count year. Since the book reviews we count consider both Canadian and international titles, a comprehensive baseline would involve statistics on total books published both in Canada and internationally. Gathering such statistics on the international side would prove enormously challenging; however, even gathering Canadian numbers has been problematic and remains unresolved due to inconsistent or nonexistent archives, as well as limited access and resources.

Firstly, there is no single, comprehensive source for this information in Canada. Some provinces collect this information, others do not, and books statistics that are available are often sales-based. While it is possible to gather information on some sectors of the book market, this would take an extensive amount of labour and arguably never be comprehensive. CWILA has looked at BookNet, Library and Archives Canada, and provincial government records and has come up short thus far. We have cursorily investigated ISBNs granted to books during the Count year, but so far a list of those books is not accessible.

Were we to gather a list of total books published in Canada for a given year, we would then have to gender code each author, and our numbers would be limited to the Canadian portion of our book review Count. Though we will arguably never have a baseline that may be used for the entire count (international + Canadian titles), we are still actively pursuing a solution to collecting data on the total books published in Canada, and we welcome help on this matter.

SUMMARY OF Count Results

The 2013 CWILA Count was our biggest yet. As we worked to add French-language publications and more metropolitan newspapers to our Count list, we increased our scope by 82% over last year. In total, we recorded 5613 book reviews (4354 in English and 1259 in French) published during 2013 in 31 Canadian publications, including monthly and quarterly literary journals, online magazines, blogs and independent newspapers, as well as several major metropolitan and national newspapers. The results of the 2013 Count, which are summarized below, reveal why we must continue to raise the question of gender equity in Canadian literary culture. A gender gap remains and our numbers talk; we can spark the conversations that close the gap.

Books Reviewed in 2013

Though many publications in the 2013 CWILA Count are approaching or have reached gender parity, overall, the gap remains wide – women occupy less review space than men. This is particularly the case when we look at the books reviewed across publications included in our Count. Of reviews we counted, 57% feature books written by men, while only 37% feature books written by women.

CWILA_2013_BooksOverall

CWILA_2013_BooksCanCWILA_2013_BooksNonCan

CWILA’s count of women’s books reviewed in 2013 is dismal; indeed, the gender gap appears to be widening. There are a few reasons for this, however, which bear explanation. Firstly, the new publications we added to the Count this year have poorer numbers, and thus have brought down our overall totals. On most metrics, each of these new publications show numbers in line with those from our first Count, that is, before the CWILA numbers brought gender inequities to light and got people talking. Secondly, we have opted this year to not subsume co-authored books into the male and female gender categories (by percentage of contribution). Instead we have chosen to expand our gender classification for Books and Reviews by adding multi-gender “Co-Authored” categories where relevant (see Methods Report for more details). Though the number of female and male co-authored books reviewed is relatively low, adding this category has limited and thus reduced overall numbers in the “female-authored books” category to books written exclusively by women or trans women; either singly or in collaboration with other women. Thirdly, while some of the publications are publishing more reviews of women’s books, it remains the case that many have not heeded CWILA’s call.

Although we see mostly Canadian male authors and Canadian male-authored books among those books with 7 reviews or less in 2013, several books by women are getting a lot of nationwide attention. Of books with 8 or more reviews, 59% were written by women (mostly Canadian), and of the top reviewed books (10+ reviews), 50% were written by women (all Canadian).

Most Reviewed Books in the 2013 CWILA Count (10+ Reviews)

  • Life After Life, Kate Atkinson (10)
  • My Life Among the Apes, Cary Fagan (10)
  • David and Goliath: Underdogs, Misfits, and the Art of Battling Giants, Malcolm Gladwell (10)
  • The Water Rat of Wanchai, Ian Hamilton (10)
  • How The Light Gets In, Louise Penny (10)
  • The Luminaries, Eleanor Catton (11)
  • Blood: The Stuff of Life, Lawrence Hill (11)
  • Joyland, Stephen King (11)
  • The Moonshine War, Elmore Leonard (11)
  • Caught, Lisa Moore (11)
  • MaddAddam, Margaret Atwood (12)
  • Hellgoing, Lynn Coady (15)
  • The Orenda, Joseph Boyden (21)

Most publications counted by CWILA in the past show a growing trend toward gender parity for books reviewed. Of all reviews counted in the National Post in 2012, 42% were of books written by women. In 2013, this number has risen to 45%. Last year, then National Post books editor Mark Medley, attributed the 2011 CWILA Count as the reason for changing in his editorial practices. Notably, publications newly counted in 2013 show numbers similar to overall findings from the 2011 Count; in other words, much room for improvement. Of reviews counted in Le Devoir, only 25% are of women’s books, while of reviews counted in The Chronicle Herald, 31% are of women’s books. The Toronto Star looks the best in this group with 39% of reviews featuring women’s books, whereas Nuit Blanche looks the worst, with only 21% of reviews featuring women’s books. By including these publications in our annual Count we have begun the important task of opening conversations on equitable editorial practice.

 

Reviewing Practice in 2013

According to our Count, women are publishing reviews 46.59% of the time, while men are publishing reviews 51.59% of the time, which is a slight improvement from last year, toward gender parity. Improvements in reviewing practice are also apparent when we look to the publications themselves. Within half of the publications we counted, the number of reviews written by women is at parity or above.

CWILA_2013_ReviewsOverall CWILA_2013_ReviewsEnglish CWILA_2013_ReviewsFrench

Unfortunately, it is also the case that our top publishing reviewers tend to be men. Of the top 20 reviewers in the country (reviewers with 50+ reviews), only 38% are women. This statistic is especially concerning because as the 2012 Count last year revealed, and as the 2013 Count reconfirms, men are reviewing women’s books only a third of the time: women’s books comprise only 25% of reviews written by men, while men’s books comprise 69% of reviews by men. This gender dynamic does not happen among women reviewers, who feature women’s books at about the same rate as men’s books: of reviews written by women, 51% are of women’s books, while 43% are of men’s books.

r_female r_male

Genre Matters

The 2013 CWILA Count tracked the genre of books reviewed. Largely replicating last year’s genre numbers, fiction and non-fiction books comprise the majority of books reviewed, by our count. Also in line with last year, poetry comprises only 8% of books reviewed, while drama comprises less than 1%.

results chart 7

Gender trends are clearly evident as we look closer at the data, particularly in the case of non-fiction books. As our numbers suggest, there is a glaring omission of women’s non-fiction books in Canadian review culture. Overall, only 28% of non-fiction books reviewed were written by women, and in nine publications, including The Antigonish Review, Literary Review of Canada, National Post, Chronicle Herald, and Le Devoir, reviews of women’s non-fiction books are outnumbered by reviews of men’s non-fiction books 3 to 1. Put another way, men’s non-fiction books are reviewed twice and often three times more often than women’s non-fiction books in a majority of publications. The conjectural argument that women’s non-fiction is reviewed less than men’s because women publish fewer non-fiction books is rendered mute by the fact that four of the publications we counted have numbers approaching or reaching parity, even favouring women’s non-fiction. These include Geist (6% gap), Canadian Literature (2% gap), This Magazine (parity), and Lemon Hound, where 7 out of 12 non-fiction books reviewed were written by women. If Canadian Literature can fill nearly half of its review space in every issue with women’s non-fiction – that’s 124 new books in 2013 alone – then why must the Literary Review of Canada, for instance, devote only a third of its review space to women’s non-fiction – a mere 32 books in 2013?

The 2013 CWILA Count numbers do more than quantify a gender gap. They serve as a starting point for discussing the barriers to equitable representation in Canada’s review culture, and they invite us to redress inequities in our literary communities. As you use them to think through your own critical practices and perspectives, we invite your feedback, commentary and critical responses.

A Special Thanks to our Count Volunteers

As a grassroots non-profit feminist collective, CWILA relies heavily on the indefatigable labour and eye strain of many volunteers to help us produce the annual CWILA Count. The community of volunteers that came together during this Count season was as generous as it was uplifting. For making the CWILA Count happen, endless thanks to these volunteers: Kelsey Attard, Susan Bahaduri, Emily Ballantyne, Kristy Bell, andrea bennett, Laura Bidwell, Alice Burdick, Andrea Cabajsky, Mandy Catron, Sonja Cvoric, Sarah Deretic, Sonia Di Placido, Tatrina Finlay, Cynthia Flood, Janet Fretter, Elee Kraljii Gardiner, Sheila Giffen, Neta Gordon, Katia Grubisic, Warren Heiti, Bethany Hindmarsh, Bara Hladikova, Kyla Jamieson, Katherine Leyton, Sylvia McKelvie, Kaarina Mikalson, Monica Miller, Dominique Millette, Jane Munro, Carmel Purkis, Maddie Reddon, Clélie Rich, Alexandra Saginur, Guldana Salimjan, Natalie Simpson, J.C. Sutcliffe, Taryn Thomson, Leslie Timmins, Christina Turner, Rachel Wallace, Daniel Wai Hon Yip, and Jan Zwicky.

 

Judith-photo-newJudith Scholes has served as Count Director for the 2012 and 2013 CWILA Counts. She is a Vancouver-based writer, teacher, and PhD Candidate in English at the University of British Columbia, where she is completing a dissertation on Emily Dickinson, material rhetoric and the ethos of women’s poetry in nineteenth-century American periodicals. Her work has been published in The Emily Dickinson Journal and American Periodicals.

View the 2013 numbers. See the 2013 CWILA Count Infographic.

Published on September 25, 2014

This entry was posted in The CWILA Count, Uncategorized, Writings. Bookmark the permalink.
  • andreakb

    Hey Joe. Are you volunteering? I notice that that is a non-beret hat there in your avatar.