Kialo is an online structured debate platform with argument maps in the form of debate trees. It is a collaborative reasoning tool for thoughtful discussion, understanding different points of view, and collaborative decision-making, showing arguments for and against claims underneath user-submitted theses or questions.[2][3][4][5]
Kialo debate tree schema with an example path through it.[6]Example partial argument tree with claims and corresponding impact votes for arguments within the given line of reasoning, one form of collective determination of argument weights that is used on the platform[7]
The deliberative discourse platform is designed to present hundreds of supporting or opposing arguments in a dynamic argument tree[8] and is streamlined for rational civil debate on topics such as philosophical questions, policy deliberations, entertainment, ethics, science questions, and unsolved problems or subjects of disagreement in general.[3][2][9][10]
Argument-boxes are structured into hierarchical branches where the root is the main thesis (or theses) of the debate, enabling deliberation and navigable debates between opposing perspectives. A debate is divided into Pro (supporting) and Con (refuting or devaluing) columns where registered users can add arguments and rate the impact on the weight or validity of the parent claim. The arguments are sorted according to the rating average.[15]
Its argument tree structure enables detailed scrutiny of claims at all levels of the tree[16] and allows users to for example quickly understand why a decision was made or which of the aggregated arguments swayed it this way.[3] Newcomers can join a debate at any time and look back at the structured discussion history, and then weigh in at the right place with their new argument or their comment on a specific argument.[2][11][17] The design presets a structure on debates "that allows participants to easily see, process, and ultimately assess the many facets of competing claims".[16]
The word Kialo is Esperanto for "reason".[3][2] The platform is the world's largest argument mapping and structured debate site.[18][19]
Overview
Users can comment on every Pro or Con, for example for requesting sources or expansions.[9] Recent activities of a debate are shown in a panel on the right side of the respective debate.[9] Debates can be found through the search or on the Explore page through their descriptions and topic-tags.[5]
Mere comments that do not make a constructive point (a self-contained argument backed by reasoning) are not allowed and are picked up by other users and moderators.[3][5][20] "Civil language and sensible observations from opposing perspectives" can be seen also in debates about controversial topics.[21] The site by-design incentivizes fair, rigorous, open-minded dialogue.[22] Contributors making claims often also write counterpoints to their own contribution.[3] Claims need to be shorter than 500 characters and can link to external sources.[23]
Debate trees can also start off with multiple theses – such as different policy options or hypotheses. Claims can link to related debates or include segments of them.[24] In the discussion tab of each claim, users can make edit proposals (e.g. for accuracy, improving sources, or changing scope), decide if the argument should be moved or copied to another branch, call for archiving a claim, and ask for extra evidence or clarification.[25]
Debates can grow large and complex for which a sunburst diagramvisualization of the topology of the debate[5][26][16][27][28][29] and the search functionality can be useful. Each debate also has a chat-box.[30][31] In cases where e.g. a "Con" is a point against multiple in the "Pros", users – through moderators – can link these arguments at the respective places to avoid duplication of content and allowing a clean chain for people to understand which points are arguments against each other.[9] Contributions of users are tracked, enabling a board of thought-leaders for every debate.[27] Other gamification elements include a feature to thank users for their contributions.[32][23]
The "Perspectives" feature allows users to see 'Impact' ratings of supporters and opposers of a thesis as well as of the debate's moderators and individual contributors.[33] It thereby enables participants to see a debate from other participants' perspectives and to sort by them.[33] In Kialo Edu, this feature lets teachers view votes for a whole class, individuals, or supporters/opponents of a specific thesis.[34] Users in both versions of Kialo can vote on the overall debate topic as well as on individual claims to express their perspectives or conclusions, with the rationale (i.e. the main causal arguments) why they voted on the veracity of the thesis as they did not being captured.[35] Voting can be done by any registered user while navigating through any debate that has voting enabled or via using the Guided Voting wizard user interface that automatically walks through branches.[36]
Diagram roughly showing the shares of site content per top debate category; each argument can be revised and the diagram shows the numbers of revisions in each category which roughly reflect which categories are most actively and/or extensively debated.[10]
A 2018 report stated the collaborative argument platform hosts more than 10,000 debates in various languages.[23] It also hosts private debates. The website claims that it has over 18,000 public debates as of July 2023, as well as over 1 million votes and over 720,000 claims.[38] Debates can be found via the site's internal search and up to six tags per debate.
Preprint studies have scraped public debates on over 1.4K issues with over 130K statements as of October 2019[39] and 1628 debates, related to over 1120 categories, with 124,312 unique claims as of June 26, 2020.[10]
Kialo Inc.
The site is run by Kialo Inc. It was founded by German-born entrepreneur and London School of Economics and Political Science graduate Errikos Pitsos in August 2017 and is based in Brooklyn and Berlin.[3][2] According to a 2018 report, the site does not show advertisements and does not sell user's data.[3] The for-profit company was founded in 2011,[40][additional citation(s) needed] Pitsos began to develop the concept in 2012[23] and described various specifics of the system in 2014.[41] In 2018, he stated that they intend to make money by selling the platform to companies as a deliberation and decision-making tool.[23] The site is free to use for the public and in education.[11] According to the site, as of 2023 Kialo.com is a non-revenue generating site with no ads and no reselling of user data.[1]
Applications of its content or the platform in society include:
Teachers and professors, especially in high schools – including the universities Harvard and Princeton, are using Kialo for class discussions and exercises in critical thinking and reasoning,[3][37][23][17][18][35] as consolidating understanding of materials covered in recent classes,[18] more useful and engaging learning experiences,[35] for remote/e-learning,[42] for clearing up misconceptions,[12] teaching logical fallacies and rational argumentation,[43][44] for academic dialogue,[45] teaching media literacy, and for teaching to sufficiently reflect or research before posting online.[44] Like for debaters of the main site, access for schools and universities is free. Kialo Edu is the custom version of Kialo specifically designed for classroom use where debates are private and locked to invited students.[46][11][47][48][18][49]
Kialo allows teachers to provide feedback to students on their ideas, argument structure, and research quality while it is left to other students to rate the impacts of their peers' arguments.[11]
Students can be allowed to contribute anonymously which may be useful for controversial issues as well as for safeguarding privacy in education.[50]
Students are or can be encouraged to back up their claims with evidence which can foster digital literacy and research skills.[11]
Students and teachers can use it to arrange their thoughts when structuring an essay or project.[11]
The site's name was decided on internally using the software.[2]
Structured debates from the site could be used for intelligent assistants and similar AI software[28] such as in computational reasoning; such bots have been called "artificial deliberative agents" (ADAs).[51]Example of an ADA contributing missing information to a debate via crawled Kialo data and selected based on the prior conversation and crawled argument weight ratings.[51]
Potential, theoretical, prototypical or little-used applications include:
Pitsos claimed that "we're training students to be very good test-takers instead of critical thinkers", suggesting teaching people to think things through may be more important or neglected compared to essay writing skills.[2]
Many young people and adults are "submerged into a sea of dispersed information", "[b]rowsing and engaging in superficial thinking activities". Kialo could counteract this issue and help people develop good sane reasoning.[26]
Three scholars from three prestigious U.S. universities outlined possible benefits in this domain, including applications beyond higher education such as for academic communication. They suggest the debate platform could be used for structuring the communication of open peer-review by helping those giving feedback to "hone in on[sic] core arguments and pieces of evidence in an even more direct way" than annotated commenting.[26][16]
It could be used to evaluate extracted argument structures and sequences from raw texts,[16] as in a Semantic Web for arguments. Such "argument mining", to which Kialo is the largest structured source so far, could e.g. be used to assess the completeness and effectiveness of an argumentative discussion[14] or to augment it (with additional arguments, contextual information, assessments, refuting evidence or supporting data).[6]
A security studies paper suggested it could be used for "managing arguments more effectively than traditional paragraph/bullet-point approaches". It claims that "complexity demands adaptation" but also notes that "Kialo's simplicity does pose some weaknesses and limitations, and in general current systems cannot reliably automate analysis or synthesis of arguments in the same way that statistical packages can automate analysis of data".[52]
With a platform like Kialo, users provide "both data on what they see is in the landscape of relevant arguments but also some indication of what they think is important [or has priority] in determining their policy preferences" and "also shows which arguments the individual did not find persuasive, and possibly which rebuttals to a particular argument [was] used to discard it."[16] Current functionality of the site may still be insufficient for the latter outside of experiments.
Policy-makers and scientists could use platforms and debates like these to engage with each other as well as the public[24] if they were aware of it and used it. Considering only argument trees beneath theses, its arguments-crowdsourcing and revision principles are not or less vulnerable to framing-issues, intentionally placed attackable segments, weak or missing arguments, straw man points, oversimplification, agenda-setting and other issues that may be common in contemporary public political debates.
The debate trees can be used to identify arguments that are seen as most credible, as well as reveal which areas of argumentation lack support, precedent, or evidence,[16] which may be useful for subsequent work or more efficient and useful science (as in identifying little-supported assumptions, providing key missing data, or researching key open questions).
General
Writers in general, as well as possibly major other opinion leaders, could populate a Kialo debate with their arguments and release it alongside the traditional linear written format,[16] albeit such would mean the arguments would be open to scrutiny, with such being more accessible than large and fans-dominated unstructured comment sections or may already be part of an existing debate tree. They could also use the site in other ways such as for selecting questions to pose to interviewees or for selecting unexplored questions to investigate and report on.
Websites could embed read-only argument trees (or branches) from the site.[23]
More broadly, the site's content could be used for reflective brainstorming, and as a crowdsourcing resource for points to use in other media (e.g. long-form text). It enables detailed exploration of some theses or topics as the visual reasoning through tree-based structure allows for many levels of depth and for follow-up questions in the discussion tab of each claim.[25] The founder stated that "The public debates are basically supposed to become a site where people can go and inform themselves. If a debate has over 2,000 unique arguments, it's going to be hard to find an argument that's not in there already. You can go there, similar to Wikipedia, and read."[2]
Kialo is a subject of research studies and its data has been used in research as there are datasets of its contents[5][13][54][10][7] and the site allows exporting CSV files[16] as well as crawling and filtering debates.[6][51]
Computational research on argumentation
The platform has gained attention in computational research on argumentation because of its high-quality arguments and elaborate argument trees.[14][56] Its data has been used to train and to evaluate natural language processing AI systems such as, most commonly, BERT and its variants.[61] This includes argument extraction, conclusion generation,[58][additional citation(s) needed] argument form quality assessment,[10] machine argumentative debate generation or participation,[6][7][56] surfacing most relevant previously overlooked viewpoints or arguments,[6][7] argumentative writing support[54] (incl. sentence attackability scores),[39] automatic real-time evaluation of how truthful or convincing a sentence is (similar to fact-checking),[39]language model fine tuning[62][56] (incl. chatbots),[63][64] argument impact prediction, argument classification and polarity prediction.[20][65]
Content analysis in social science and belief studies
The contents can also be analyzed to e.g. show the most common Con rationale-types and factors in general,[39] or reveal the most contested arguments where ratings diverge the most for a given topic.
The site's founder proposed the types of arguments and ways people reason could be investigated as well as the "performance of Kialo versus long-form text in making people change their minds".[2] One study suggests arguers seem to change their viewpoints more readily when a fact they believe has evidence and is undermined when compared to prior beliefs without any specified supporting data.[39]
A study showed that when evaluating policies via Kialo debates, "reading comments from most to least liked, on average, displays more [winning arguments] than reading comments earliest first".[66][67] Kialo has a set of different permissions that participants can have in a given debate. A preprint study makes suggestions regarding "interface design as a scalable solution to conflict management" to prevent adversarial beliefs and values of moderators to have negative impacts on the site.[33]
Reception, motivation and distinction from alternatives
In 2022, MakeUseOf named the site as one of the five best "debate sites to civilly and logically argue online about opinions"[9] and in 2019 as one of the "100+ best websites on the Internet".[68]
Online discourse quality
The site aims to be a hub for civilized debate where shouting, rudeness or irrationality aren't allowed.[3][23] This has been described as remarkable in an "age of Trumpian tweeting".[3] The site's founder stated that he noticed early on that the Web became "ideal for bad conversations, with prominence given to the most outrageous conversations" and that he "wondered if there wasn't a better method of online discourse", claiming the site's mission is to "empower reason and to make the world more thoughtful",[3][4][69][46] describing it as a "platform where people with opposing views can meet and understand each other's thinking".[70] As of 2023, there are major concerns about online irrational or misinformation-fueled debate – for example, a researcher affirmed[21] that "Twitter was not designed or intended to be a digital town square" as part of a "functioning democracy", addressing Elon Musk's comments about the site in 2022. Instead, she claims it to be a "space for millions of town criers, but not a town square for people to come together and debate".[71] Reports suggest the site may present a more complete and complex view of reality than some other sites where "it's easy to get trapped in echo-chambers of like-minded people where your beliefs are never [meaningfully] challenged" as it shows you "the best arguments on both sides of a debate".[72][24]
Communication formats
Standard digital formats e.g. "tend to only allow a linear progression of arguments in a stream-of-discussion format".[16] On many websites, "circuitous comment threads [often] render meaningful discussion impossible" and "formats that we use to communicate shape the way we communicate".[2] On the site, users contribute to a debate tree rather than engaging in argumentative back-and-forth commenting.[73]
Kialo may be more appropriate especially for discussions that are relatively complex and hard to visualize or oversee otherwise and allows for public ideation and structured interaction among different types of stakeholders.[24] Linking to supporting evidence is encouraged,[21] but not as strictly required as for example on Wikipedia. Kialo has advantages over structured knowledge bases and Wikipedia in "that it includes many debatable statements; many attacked sentences are subjective judgments, so fact-based knowledge sources may have limited utility".[39] Chains of reasoning can be followed "from beginning to end" with relatively little text to read, nearly no repetition or unexplained statements and without having it derailed by for example "name-calling and directionless ranting".[21] Online debates "have grown so large and acrimonious that no one realistically has the time to read everything and hence get a sense of the actually winning arguments (winners) after all points have been considered" and there is research into how to efficiently calculate the winning arguments or arguments weights and the overall conclusions.[67] Moreover, argumentations on the site are less fleeting and repetitive than debates on social media sites – they are commonly read and actively contributed to over the span of years.[11][2]
One preprint study stated that "[t]hough kialo is designed for scale, and therefore has to be not only robust but also both easy and appealing to use, it has simplified its notion of argument structure so much that there is very little flexibility left. As a commercial entity, its data [not reusable] and platform [not open source] are also closed, making wide-scale application at the science-policy interface more challenging."[74]
One study found that "Kialo's simplicity does pose some weaknesses and limitations" and found the functionality of current systems including Kialo for "synthesis of arguments" to be insufficient.[52] One study suggests the platform is structured in a way that gives insufficient capacity for users to do anything else other than to either agree or disagree with a side,[75] with there e.g. only being options to rate the veracity of the main thesis but not for proposing concrete alternatives and middle-grounds such as more nuanced policies or specifying conditional critical considerations (e.g. exceptions, applicable scopes and limitations) of one's veracity rating of the main thesis, which tend to be very brief and rarely revised.
One study points out that without 'Writer' permissions in a debate, the arguments have "to get past the gatekeepers" of it, which can in some cases be problematic as moderators' beliefs and values may play a role.[33] For instance, such can lead to some users feeling like certain perspectives (or arguments) are being excluded from a debate[33] or getting positioned inappropriately (such as not being visible at the level most relevant). There may be issues relating to framing and argument positioning, whereby for example a false claim (with or without a source) can be added as supporting a thesis which is then only addressed by a later countering claim stating the opposite beneath it – which may reduce the former's 'Impact' rating but is not shown directly at the tree level above as an 'countering' argument. Instead, only the false or weak supporting argument can be seen at the level above in such a case. Impact rating votes do not require reading the arguments beneath but voting can be turned off until the argument map has had time to sufficiently develop.[76]
Complementarity
The founder clarified key distinctions and complementarity of the site saying "We're going to just be an added place. We're not competing with anybody out there with regards to thoughtful discourse. There are a couple of sites that are question-and-answer sites, or commenting sites, or sharing sites, but there's not a single [major] site for collaborative reasoning — a repository of the why".[2] He states that Wikipedia – another peer production site to which Kialo is sometimes compared with due to argumentative discussions on Talk pages[14] and its public collaborative knowledge integration[33][35] – "tells you the what and we tell you the why".[2]
^ ab"About". Kialo. Archived from the original on 7 June 2023. Retrieved 9 June 2023.
^ abcdefgAgarwal, Vibhor; Joglekar, Sagar; Young, Anthony P.; Sastry, Nishanth (25 April 2022). "GraphNLI: A Graph-based Natural Language Inference Model for Polarity Prediction in Online Debates". Proceedings of the ACM Web Conference 2022. pp. 2729–2737. arXiv:2202.08175. doi:10.1145/3485447.3512144. ISBN9781450390965. S2CID246867079.
^ abcdeDurmus, Esin; Ladhak, Faisal; Cardie, Claire (2019). "The Role of Pragmatic and Discourse Context in Determining Argument Impact". Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). pp. 5667–5677. arXiv:2004.03034. doi:10.18653/v1/D19-1568. S2CID202768765.
^ abcDe Liddo, Anna; Strube, Rosa (21 June 2021). "Understanding Failures and Potentials of Argumentation Tools for Public Deliberation". C&T '21: Proceedings of the 10th International Conference on Communities & Technologies - Wicked Problems in the Age of Tech. Association for Computing Machinery. pp. 75–88. doi:10.1145/3461564.3461584. ISBN9781450390569. S2CID235494842.
^ abAgarwal, Vibhor; P. Young, Anthony; Joglekar, Sagar; Sastry, Nishanth (2024). "A Graph-Based Context-Aware Model to Understand Online Conversations". ACM Transactions on the Web. 18: 1–27. arXiv:2211.09207. doi:10.1145/3624579.
^ abAnastasiou, Lucas; De Liddo, Anna (8 May 2021). "Making Sense of Online Discussions: Can Automated Reports help?". Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery. pp. 1–7. doi:10.1145/3411763.3451815. ISBN9781450380959. S2CID233987842.
^Kiesel, Johannes; Spina, Damiano; Wachsmuth, Henning; Stein, Benno (27 July 2021). "The Meant, the Said, and the Understood: Conversational Argument Search and Cognitive Biases". CUI 2021 - 3rd Conference on Conversational User Interfaces. Association for Computing Machinery. pp. 1–5. doi:10.1145/3469595.3469615. ISBN9781450389983. S2CID236203094.
^Woodward, Heather; Padfield, Laura (2021). "A Blended Approach to Flipped Learning for Teaching Debate – Using Kialo Edu for EFL Debate Preparation". Journal of Multilingual Pedagogy and Practice. 1. doi:10.14992/00020487.
^ abcBetz, Gregor (2022). "Natural-Language Multi-Agent Simulations of Argumentative Opinion Dynamics". Journal of Artificial Societies and Social Simulation. 25: 2. arXiv:2104.06737. doi:10.18564/jasss.4725. S2CID233231231.
^ abcdSkitalinskaya, Gabriella; Wachsmuth, Henning (2023). "To Revise or Not to Revise: Learning to Detect Improvable Claims for Argumentative Writing Support". arXiv:2305.16799 [cs.CL].
^ abDurmus, Esin; Ladhak, Faisal; Cardie, Claire (2019). "Determining Relative Argument Specificity and Stance for Complex Argumentative Structures". Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. pp. 4630–4641. arXiv:1906.11313. doi:10.18653/v1/P19-1456. S2CID195699602.
^ abcdAl Khatib, Khalid; Trautner, Lukas; Wachsmuth, Henning; Hou, Yufang; Stein, Benno (August 2021). "Employing Argumentation Knowledge Graphs for Neural Argument Generation"(PDF). Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics. pp. 4744–4754. doi:10.18653/v1/2021.acl-long.366. S2CID236460348. Archived(PDF) from the original on 2023-04-08. Retrieved 2023-06-11.
^Fanton, Margherita; Bonaldi, Helena; Tekiroglu, Serra Sinem; Guerini, Marco (2021). "Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speech". Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). pp. 3226–3240. arXiv:2107.08720. doi:10.18653/v1/2021.acl-long.250. S2CID236087808.
^Hahn, Ulrike; Jens Koed Madsen; Reed, Chris (2022). "Managing Expert Disagreement for the Policy Process and Beyond". arXiv:2212.14714 [cs.CY].
^Althuniyan, Najla; Sirrianni, Joseph W.; Rahman, Md Mahfuzer; Liu, Xiaoqing "Frank" (2019). "Design of Mobile Service of Intelligent Large-Scale Cyber Argumentation for Analysis and Prediction of Collective Opinions". Artificial Intelligence and Mobile Services – AIMS 2019. Lecture Notes in Computer Science. Vol. 11516. Springer International Publishing. pp. 135–149. doi:10.1007/978-3-030-23367-9_10. ISBN978-3-030-23366-2. S2CID195353310.
_for_argumentation.png)
.png)
