In the Summer 2018 issue of International Debate, Miriam Kohn (LearningLeaders, Rochester ’17) and Chris Perkins (Rochester ’17) published an article titled “Fair Play: Proposals for Reform to the Break System in British Parliamentary Debate.” In said article, they discussed the possibility to replace team breaks on speaks with a point-based break and additional bye-based elimination round.

AchteMinute (AM): First off, so lucky to be chatting with you, Miriam & Chris. Can you tell us briefly about your background in debate?

Miriam Kohn (M): Thanks for taking the time to chat with us! My high school didn’t have a debate team, so I started debate my freshman year of college (fall 2013). Chris and I debated together in BP format (as Rochester KP) when we were students at the University of Rochester, where our awards included winning the Pan-American Championships. I’ve since stayed involved with debate as an IA – in North America and Asia, where I’ve been coaching debate at LearningLeaders, a debate education company in Shanghai, since August 2018.

Chris Perkins (C): I debated for three years in high school and four years at the University of Rochester, including winning the Pan-American Championships my senior year with Miriam. I have since been an independent adjudicator throughout the Northeast and will be part of the tab team at the 2019 United States Universities Debating Championships.

I. The Proposal – Debate Without Speaks

We’ll talk about your research more in-depth in a moment, but for now could you briefly summarize what you set out to fix with your proposal on altering how BP advances teams to elimination rounds?

M: Speaker points are a subjective system on face and particularly disadvantage speakers who already face substantial barriers to entry. Personally, I find that particularly concerning in an activity that sets people up for success after graduation, because these effects carry over. Our proposal seeks to reduce arbitrariness and bias in BP scoring by scrapping speaker points. This won’t entirely negate speaker bias, but it will, in my opinion, improve our ability to meaningfully reduce it.

Comments on chair feedback submitted on a female panelist at USUDC Alaska 2015

C: The biggest problem I see with the existing system is the complete arbitrariness between, say, the 16th and 17th seeds at a tournament advancing to quarterfinals. Often, the difference between these teams is 1-5 team speaker points, which likely depends more on who their judges were than the quality of their performances. In addition, many other formats of debate give a bye round to the top teams as a benefit to having the best preliminary round showing, which currently doesn’t exist in BP debate, largely because it is harder to draw brackets when you have 4 teams per round. I wanted a system that would fix both issues.

If it is implemented, what would your proposal look like for the average tournament participant?

M: For the average speaker, preliminary rounds wouldn’t change but their speaker points would no longer be restricted by the rule banning low point wins and elimination brackets would be seeded slightly differently. Judges would have more freedom to award the speaker points that they think a given speech deserves

As I read it, your proposal seems situated within a US context of growing BP debate, increased participation in international tournaments, and shifts in who participates in intercollegiate debate. Do you think that your proposal accounts for the experiences of speakers and judges within other regions?

M: While I can’t speak directly for people from other circuits, I think a lot of other circuits are experiencing the same kinds of shifts as the American circuit. While a proposal can’t perfectly encapsulate all aspects of all people’s experiences across all circuits, we draw on widely applicable experiences. Inclusivity and experiences of bias or arbitrariness in judging (whether real or perceived) are near-universal areas of concern across debate circuits.

C: Our examples and framework are simply from a US context, but that’s due to personal background. The proposal would work at any tournament in any country.

II. The Harms – Do Speaks Specifically Enforce & Reflect Bias?

You’ve so far alluded to the experience of bias extensively – I’d like to go a bit more in depth on the types of bias you seek to address. How do identity and style based biases interact with regional norms preferencing, and why do you deem existing measures for correcting regional and identity bias (balancing judging panels, including chairs that reflect the composition of the room) insufficient to at least contain bias?

M: It’s difficult to separate identity or style-based biases and biases based on regional norms because identities don’t exist in separable components and home regions often inform one’s identity. That’s compounded by regional power dynamics. For example, in the US, most debate programs were historically run by white men. (Check out Pamela Steppe’s work on this.) That’s changing but US debating norms are informed by that history.
On the second point, those who are promoted to chair judges or rewarded as good judges are people who have likely internalized the current system’s norms. That’s why they are regarded as competent enough to be promoted. If community norms are biased, successful judges who are female, ESL, etc. still are likely to have internalized biased norms. There is no particular reason that female judges are immune from internalizing and replicating sexist standards. The same reasoning holds for ESL status, race, and other identity categories. (A colleague and I have a long-term project testing this on the back burner right now, so hopefully we will be able to offer data-based commentary about speaker point allocation trends in the future.) While I consider diverse judging panels a good thing for their own sake, I don’t believe they’re an adequate solution for scoring disparities. Attempting to weed out conscious biases is insufficient to prevent these biases from affecting snap decisions such as awarding speaker points.

Further, the pool of judges at a given tournament is not sufficiently diverse to make judging panels truly diverse, rather than sprinkling a handful of diverse judges in certain rooms. Given the scope of that problem, even if this were a viable solution, it would take quite a long time for most debate tournaments to offer meaningfully diverse judging pools. I’ll be curious to see how the new WUDC affirmative action policy affects retention and training for non-male judges, which could alter this balance.

C: Judge paradigms lead to a wide range of speaker scores, as everyone has seen a different set of debate rounds in their history and has a different view of an ‘average’ speech, or a 75. Some tournaments address what an ‘average’ speech should be at that tournament, while others leave it out of the judge briefings. Even if all judges were un-biased and had the same metric for a 75 speech (both of which are a near-impossibility), some judges would have a wider range than others (i.e. a 65-85 range will lead a below-average speech to a 71, while a more conservative 68-83 range might lead the same speech to a 73). Thus even under perfect conditions judge range can be the difference between the team breaking as one of the last seeds or being left out of the break entirely. Simply put, there is no reasonable mechanism to avoid these types of issues completely, and they routinely factor into which are the last few teams to reach elimination rounds.

“Even under perfect conditions judge range can be the difference between the team breaking as one of the last seeds or being left out of the break entirely.”

Miriam Kohn in the Final of the 2017 Pan-American Championships – © University of La Verne

Inasmuch as you find subjectivity in speaker point standards instrumental in the insidious creep of bias, why is this distinct from the also subjective team ranks? You distinguish between the two on a basis of accountability (publicity and feedback) and clarity of standards, both of which seem to characterize team ranks.  

M: There’s more subjectivity in debate than any of us would like to admit. Having said that, speaks are more subjective than team points because they’re nominally pegged to concrete standards which, in practice, are not concrete and vary widely from person to person. Rankings, on the other hand, are definitionally based on the round that happened. Even though there is room for bias to infiltrate ranking discussions, at least judges are starting from the round that actually happened, rather than semi-arbitrary personal interpretations of a speaker scale. Additionally, because rankings are concrete and specific to that round, they are definitionally more comparative. That means judges have to provide some comparative justification, even if that justification isn’t robust. Speaker points, on the other hand, hardly require any justification at all. Rankings aren’t perfect but I’m very comfortable saying they’re comparatively better.

The most important reason why there are fewer checks in speaks than team points is that, if you dispute a judge’s ranking, you can point to a concrete reason for why it was wrong (i.e. they didn’t credit an argument). That makes it easier to assess the reasonableness of feedback, which makes reasonable feedback more meaningful. This also applies during panel discussions. It is much easier for a panelist to point to a specific argument or rebuttal that the chair missed than to dispute speaker point allocations based on vague criteria. I also think judges put a lot more thought into team rankings than speaker point allocations because they have a fixed amount of deliberation time and they know they will have to publicly justify them. Therefore, chairs spend deliberation time persuading the panel to agree with their call, rather than discussing speaker points in great detail. Given that tournaments are unlikely to add more deliberation time for each round into the schedule, that balance is unlikely to change.

This solution can’t offer perfect objectivity in judging and I won’t pretend that it will. However, it will increase judging objectivity and make it much more feasible to implement additional checks in the future.

C: The biggest difference is that most judges would likely arrive at a similar round ranking, as it is easier to quantify that (which teams weighed their arguments the best, explained their impacts the clearest, etc.). When it comes to speaker points, however, those same judges may well differ (even a minor difference of 1 or 2 speaker points can affect whether the team breaks or not in the current system). Another major difference between team rankings and speaker points is that chairs are tasked with leading a discussion around the team rankings and coming to a consensus, while speaker points are often decided by the chair saying ‘I think that speech was a 78’ and the panelists agreeing without much thought.

 “It is much easier for a panelist to point to a specific argument or rebuttal that the chair missed than to dispute speaker point allocations based on vague criteria.” 

III. The Alternatives 

In the same vein, you allude to how wing judges serve as checks on chair judges in awarding speaker points – how does this operate in the context of asymmetrical power balance on judging panels and (usually) shared background / norms?

M: Shared norms and power imbalances are critical concerns for both allocating speaker points and deciding team rankings. To reference back to my last answer, the key distinction is that being able to contest calls based on specific criteria and concrete instances from the round is much more feasible than going toe to toe comparing vague judgments about speaker point allocations. This proposal isn’t perfect. That’s impossible, given that debate is an activity practiced by people with human failings. However, I absolutely believe Chris’s and my solution is comparatively better and sets the stage for more improvements down the road.

Chris Perkins runs tab at the 2015 Brad Smith Debate Tournament

C: The answer to this question changes substantially if we are talking about a regional tournament or an international competition. For regional tournaments, the biggest issue is that wings are generally less experienced (often students in their first year or two of collegiate debate), and so they rely almost entirely on the chair’s input for speaker points. For international tournaments, different regions tend to have a different speaker scales, which leads to wildly different ranges (often as much as a 5-point difference in evaluation). This means the team that gets a judging panel from a more speaker-point ‘generous’ region is more likely to advance to elimination rounds than the team that does not. Some tournaments (WUDC, most notably) have done work to minimize this gap, but minimization is all they can achieve – you simply cannot completely remove this gap when judges bring wildly different backgrounds to the tournament.

How do you think current work being done by ESL/EFL debaters and Womxn’s Debate groups to combat bias within judging panels and debate relates to your research? What does your proposal offer those groups?

M: Those groups do the heavy lifting on advocacy for their members, which is critically important work. Ideally, this proposal would bring discussions that they started into the mainstream and lend more support for those groups’ advocacy. Addressing these concerns requires mainstream support. The only way to get there is to put this discussion front and center.

Reducing the amount of play given to these biases also means that biases that can’t be curbed with training and advocacy have less effect on adjudication. While trainings to reduce conscious bias are excellent, it’s a long road to reducing the kinds of subconscious bias that directly impact debate scoring.

C: While our proposal can’t fix underlying biases, it can reduce how biases affect the break. My perception is that most work being done to combat bias is lowering the bar of entry to debate. This work is largely separate from our proposal, although we would help level the field by eliminating the effect of bias in speaker points in breaks.

Thanks for your time. Really appreciate your efforts in critically evaluating how we structure debate. What would a long-term implementation strategy look like, and are you taking steps now to incorporate your proposal?

M: Thank you for the interview! The first step is to start a community discussion about this question. That’s the goal of this interview. From there, the long-term game plan is to persuade enough people to agree with our proposal that it gets normalized. Ideally, people who agreed with us would choose to use this proposal at the tournaments they host, which would hasten the normalization process.The steps I’m currently taking are all geared towards raising awareness – this interview, chatting about the proposal with other members of the debate community, etc.

C: The first step would be for a few tournaments to implement this new system and see how it goes (i.e. gather feedback from the CA team and the participants of the tournament to see how they perceived it). The most useful feedback would be around how the teams getting a bye perceived the bye itself. If teams receive this new system well, the next steps would be to expand to more tournaments, perhaps an entire circuit if someone wants to adopt this as their new break philosophy. I’m hopeful that some innovative tournaments will want to give this system a shot once they have read about it. Thanks for speaking with us, always a pleasure.

Disclosure: the author has coached Chris & Miriam

ae./jm.

Miriam Kohn is a Pan-American Champion, USUDC & NAUDC Quarterfinalist, NAW*DC & China BP IA, and former President of the Rochester Debate Union. As a Fulbright Scholar, she worked on debate programs and women’s empowerment initiatives in Laos before joining LearningLeaders as a debate educator. Miriam began debating in university.

Chris Perkins, also a Pan-American Champion and USUDC & NAUDC Quarterfinalist, Yale IV & Brandeis IV IA as well as the former Vice President of the Rochester Debate Union, has tabbed extensively in the US region and will be part of the tab team at what is historically the second largest BP tournament in the world behind WUDC, the 2019 US Universities Debate Championship. Chris has been debating since his sophomore year of high school.

Mittwochs-Feature: Every Wednesday at 10.00 a.m. the Mittwochs-Feature features an idea, interview or book regarding debate – usually in German, sometimes in English, sometimes both. If you would like to start a debate please mail us your idea to team@ achteminute.de .