Students in the CDSC are recruiting for two interview studies.
The AI Slop study seeks to interview people involved in open source software who have experience dealing with AI-authored contributions. We hope to learn more about how communities are responding to this challenge, with a particular focus on ‘AI Slop’. You can read more about the study and fill out our screening survey here!
The Software Handoffs study seeks to interview people about how responsibility for a given piece of code gets handed off from one person to the next. We are wondering how this task is handled in industry and how this might differ from what is taught in a computer science curriculum. You can read more about the study and fill out our screening survey here!
How does software engineering change when AI is part of the product we're creating?
Current practice suggests a need for structured experimentation, tolerance for uncertainty, deep qualitative research and evaluation, wrangling with questions of meaning and knowledge, advanced statistics and mathematics, and more.
How might we expand our efforts to build these skills?
CDSC faculty member Kaylea Champion recently received the very welcome news that, together with her Co-PI Jeffrey Kim, she was awarded a grant from the UW’s internal AI-SEED program to explore this key topic in empirical software engineering and computer-supported cooperative work!
The official press release from UW — including many other exciting awards supporting innovation in teaching with / about AI — is here.
Grant Title: “Rethinking engineering workflows: Building AI-powered software.”
Grant abstract: “This project focuses on teaching students the impacts to a software engineering workflow — including planning and estimation, requirements analysis, design, development, testing, deployment and maintenance — when AI-powered features are part of the product. Although substantial attention has been paid to AI as a code generating tool and to the technical skills involved in building AI models, impacts of AI features on overall engineering workflow is relatively less explored. All computing students should have an awareness of these process impacts, regardless of their specialties and goals. We propose to develop and evaluate two interventions: a no-code hands-on simulation of evaluating and fine-tuning a model, and a series of enhancements and extensions for learning modules focused on different stages of a development project. We propose to develop a preliminary version of these interventions, then to evaluate and refine them through a series of co-design workshops, and finally to publish our results.”
We advocate for consideration of harms to communities (including online communities) as they respond to, are used for, and incorporate generative AI algorithms. One area of risk is one we call ‘Social Model Collapse,’ the notion that changes to community dynamics — social and technical — can disrupt the fundamental processes they rely on to sustain themselves and flourish. We see this as a clear point of shared concern among STS, HCI, Sociology, and Communication scholars, and are hosting an open panel at the upcoming meeting of the Society for Social Studies of Science (4S), September 3-7 in Seattle. This panel is being coordinated by CDSC members Kaylea Champion (University of Washington) and Sohyeon Hwang (Princeton University) along with our colleague Hanlin Li (University of Texas).
From our call for submissions:
Model collapse in machine learning refers to the deterioration such a model faces if it is re-fed with its own output, removing variation and generating poor output; in this panel, we extend this notion to ask in what ways the use of algorithmic output in place of human participation in social systems places those social systems at risk. Recent research findings in the generation of synthetic text using large language models have fed and been fed by a rush to extract value from, and engage with, online communities. Such communities include the discussion forum Reddit, the software development communities producing open source, the participants in the question and answer forum StackExchange, and the contributors to the online knowledge base Wikipedia.
The success of these communities depends on a range of social phenomena threatened by adoption of synthetic text generation as a modality replacing human authors. Newcomers who ask naive questions are a source of members and leaders, but may shift their inquiries to LLMs and never join the community as contributors. Software communities are to some extent reliant on a sense of generalized reciprocity to turn users into contributors; such appreciation may falter if their apparent benefactor is a tireless bot. Knowledge communities are dependent on human curation, inquiry, and effort to create new knowledge, which may be systemically diluted by the presence of purported participants who are only algorithms echoing back reconstructions of the others. Meanwhile, extractive technology firms profit from anyone still engaging in a genuine manner or following their own inquiries.
In this panel, we invite consideration of current forms of social model collapse driven by a rush of scientific-industrial activity, as well as reflection on past examples of social model collapse to better contextualize and understand our present moment.
Submissions are 250-word abstracts due February 2nd; our panel is #223, “Risks of ‘Social Model Collapse’ in the Face of Scientific and Technological Advances” [Submission site link].
Kaylea Champion will be presenting a practitioner-focused talk to the Puget Sound Python user group (PuPPY) on December 18th at 5:30 p.m. PST at the GitHub offices in Bellevue, Washington. Details on how to attend this free event are available here. This practitioner-oriented talk will draw from her own research as well findings from across the academy.
If you are attending the ACM conference on Computer-supported Cooperative Work and Social Computing (CSCW) this year CSCW in San José, Costa Rica. You are warmly invited to join CDSC members during our talks and other scheduled events. Please come say hi!
This CDSC has four papers at CSCW, which we will be presenting over the next three days:
Monday: At 11:00 am in Talamanca, Kaylea Champion will be presenting “Life Histories of Taboo Knowledge Artifacts” (full paper)
Tuesday: At 11:00 am in Central 3, Zarine Kharazian will be presenting “Governance Capture in a Self-Governing Community: A Qualitative Comparison of the Croatian, Serbian, Bosnian, and Serbo-Croatian Wikipedias” (full paper, blog post), followed by Sohyeon Hwang presenting “Adopting third-party bots for managing online communities” (full paper, blog post)
Wednesday: At 2:30 pm in Guanacaste 3, Kaylea Champion will be co-presenting “Challenges in Restructuring Community-based Moderation” (full paper, preprint)
If you’re at CSCW, feel free to get in touch in person or via Discord!
In the seventh talk of the Science of Community track we organized for FOSSY, Google FOSS researcher Sophia Vargas offered an overview of different strategies for measuring the value of open source (particularly in the context of a company thinking about how to engage with FOSS).
Some of Sophia’s key insights are: models for measuring one-time cost are relatively widespread (but depend on outcome metrics like lines of code rather than difficulty); understanding the cost of maintenance and community is still in formative stages; and that business leaders can make use of research-grounded models developed to measure value and risk in an academic context into decisionmaking tools within a business context.
This is part 7 of an 8-part series sharing highlights from the Science of Community track at FOSSY. Visit the FOSSY site for more bio details and an abstract of the talk.
In the sixth talk of the Science of Community track we organized for FOSSY, independent FOSS researcher Darius Kazemi described the results of an interview study to learn from the moderation teams of decentralized social network servers. One of Darius’ key observations is the extensive compliance and legally-required work that running such a server requires.
This is part 6 of an 8-part series sharing highlights from the Science of Community track at FOSSY. Visit the FOSSY site for more bio details and an abstract of the talk.
In the final talk of the Science of Community track we organized for FOSSY, Computer Science professor and FOSS researcher Dr. Bogdan Vasilescu described his team’s work to understand how developers think about abandoned dependencies. One of the key insights from this work is that abandonment of dependencies is quite common, but that updating a package to remove the abandoned dependencies is very slow — and that one of the factors that drives faster replacement is when projects explicitly announce that they are ending maintenance.
This is part 8 of an 8-part series sharing highlights from the Science of Community track at FOSSY. Visit the FOSSY site for more bio details and an abstract of the talk.
In the fifth talk of the Science of Community track we organized for FOSSY, Dr. Kaylea Champion describes a series of research results on both how to build high-quality FOSS and how to sustain a community alongside it. One of her key insights is that a great community is no guarantee of a high-quality project — and to serve the public, we need both.
This is part 5 of an 8-part series sharing highlights from the Science of Community track at FOSSY. Visit the FOSSY site for more bio details and an abstract of the talk.
In the fourth talk of the Science of Community track we organized for FOSSY, principal developer advocate Paige Cruz shared the results of her investigation into the subject of how we can all do a better job of asking questions of one another in FOSS communities. One of her key insights is to invite us to engage with the perspective of those who might answer our question, and to think critically about what details we include and whether they really help others understand and respond — for example, a screenshot of our code can’t be copy pasted and might be unreadable, but a screenshot of a UI bug might replace wordy description.
This is part 4 of an 8-part series sharing highlights from the Science of Community track at FOSSY. Visit the FOSSY site for more bio details and an abstract of the talk.
