Salon with Caroline Sinders #

Hi everyone, welcome to this fourth AI and Society Salon. I’m very happy that we have Caroline Sinders today with us for this for this Salon. In this Salon, we discuss how we can rethink Artificial Intelligence from the perspective of artists and creatives who work with this technology. The goal is to offer different ways of looking at Artificial Intelligence, especially when it comes to art and culture. We want to illustrate diverse narratives and imagine how to regain power over this technology.

My name is Baptiste Caramiaux, I’m a researcher at CNRS, the French National Centre for Scientific Research. I’m also a researcher at Sorbonne Université and I’m based in Paris.

This series of Salons is supported by the AI and Society Programme of Microsoft Research. It is co-organised with Jenny Williams, Gonzalo Ramos, Kate Crawford and Vera Liao.

If you have missed the previous Salons that we organised in the past months, you can find them online. I will be putting the URL in the chat for those interested. The chat should be available to everyone. As I mentioned for this first Salon, we are very happy to welcome Caroline Sinders. Before introducing Caroline, I want to let you know that this episode is recorded. We usually use a shared document for you to put notes, questions and comments. That will be very helpful and used during the Q&A after Caroline’s presentation. I’m also putting the URL to this document in the chat. Feel free to add all the notes, questions and comments that come to mind during Caroline’s presentation.

Let me present Caroline now. Caroline Sinders is an award-winning critical designer, researcher and artist. They are the founder of Human Rights and Design Lab, Convocation Research + Design and a current Braid Fellow with the University of Arts London. For the past few years, they have been examining the intersection of artificial intelligence, intersectional justice, harmful design, systems and politics in digital conversational spaces and technology platforms. They have worked with the Tate Exchange at the Tate Modern, the United Nations, the UK’s Information Commissioner’s Office, the European Commission, Arts Electronica, the Harvard Kennedy School and others. Caroline is currently based between London in the UK and New Orleans where they are today. I give them the floor.

Thank you so much. I’m really honoured and excited to be here. I guess I’ll just get started with the presentation so everyone can see my screen. Great, wonderful. I’m going to be toggling between my phone to read my presenter notes as I very, very much need them at times.

Hi, I’m Caroline. You already have heard a little bit about my bio and background. One of the things I sort of want to point out is I do a lot of work, both in my art practice and in my human rights practice, at looking at how technology impacts society through many different lenses, but one sort of looking at how it disproportionately harms vulnerable and marginalized populations. I also look at the sort of design layer of the Internet. So if we’re thinking of the Internet as an onion, we’re often on the content layer. Design is an integral part of that content layer of how users or consumers, we can think of them perhaps, are interacting with all different kinds of technologies.

So design is, I think, a very specific and I would argue very sort of politicized space, including one of activism, because how users understand or misunderstand how technology works, design is very much involved in that understanding or misunderstanding. So if we’re thinking of how to create safer or more equitable forms of software for all different kinds of users and consumers, well, design will be and is very much a part of that. And one of the reasons I bring that up is that’s also what I worked on at the ICO, the Information Commissioner’s Office, the UK’s data protection and privacy regulator. The ICO and actually also the CNIL in France, the privacy regulator there, are some of the few privacy regulators that regularly hire designers and sort of understand the power of design.

What I was working on there was how harmful design patterns, often called dark patterns, how they harm consumers, how they can impact regulation of making it much more difficult to access. If you’ve ever interacted with a cookie banner and found it difficult to reject, oh, then you know exactly what I’m talking about and how this does have impacts on privacy regulation and our own personal privacy.

A lot of my work looks at how technology is also designed on this graphical user interface level and the design choices and, again, how that impacts digital rights, which is a subset of human rights. I’m here to talk about another project of mine, putting on my artist hat, though all of my artwork does usually sort of come out of my human rights work. But we’re here to talk about Feminist Dataset, which is a critical design and art project and also a research project started in 2017 as a response to the many sort of burgeoning documented cases of problems related to technology and bias in machine learning.

So why machine learning? Well, I used to work in machine learning as a design researcher at IBM Watson at the same time, and this is something I was very, very interested in. But then also, as I was seeing as machine learning algorithms and AI sort of all became one thing in regards to the general public, also how so much, pardon my language if you will, so much snake oil regarding AI was starting to bleed into consumer products and how this also sort of led, at least from the standpoint of human rights work, to a lot of confusion over as to what that tech product is doing. How is it using AI? What does that mean for me as a user and a consumer? What does that mean for the safety of the consumer?

And so Feminist Dataset is also interested in these many different sections. Feminist Dataset has held workshops at the University of Indiana Bloomington within the high school and Librarians, Space Art and Technology, Z27, which was a series of workshops with actually German high schoolers, which was really amazing, Ars Electronica, Blackwood Art Gallery, the Victoria and Albert Museum, and Soho 20, which is one of the oldest feminist art spaces in New York, as well as many others. So again, it’s this sort of critical research and art project that aims to examine bias in machine learning through all the different steps or parts of the machine learning pipeline, such as data collection, data training, generating neural networks, and eventually sort of placing this in some kind of interactive interface for users.

This project is inspired by the work of the maker movement, critical design, Arte Util, the Critical Engineering Manifesto, and APC’s Feminist Principles of the Internet. APC is a really wonderful NGO based out of the majority world, and they’ve been doing a lot of different work on feminist technology within the human rights space. If you have not heard of them, I definitely recommend checking it out. One of the things I also want to highlight, which sometimes I feel a little gauche doing that, is that this is a very scrappy project. In total, the entire amount of money this project has received from 2017 until now is under 15,000 euros. So it’s very much a small project. But I also want all of us to sort of keep that in mind, not that money defines worth, but just, again, how much that this sits within the art realm. I think it’s important when we’re discussing art research, how that can move at a very different pace, let’s say, than traditional academic research. And also just how the pay scales can at times even be very, very different. But I think that’s important when thinking about how the project has evolved over these past, almost now, eight years. I’m going to go back up to this section.

Sorry, I’m just trying to find my notes because they copied out of order. Give me one second. Well, effectively, this is how I approach both my research practice and art practice, because for me, they are one in the same. I might sort of present my work in different ways depending upon who I’m speaking to. But when I think of my entire practice as a whole, I do think of these as incredibly, incredibly linked.

So often when I start with an art project, as you can see, art ends up being kind of the last step here. For me, the first step is research because it’s practical. Since there’s not really enough time in this presentation to go into all of the structures within research, one of the things I want to point out is when I’m describing research, I’m going to be coming at it from a human rights perspective. There’s many conversations we can have around decolonization, for example, or what is considered witnessing or verification. But research in a human rights context sits at this really important level in which you can work and should be working with communities and help verify different kinds of experiences they’ve had. This, again, creates a baseline from places to work from.

I want to very much highlight that this can be a very asymmetrical and hierarchical way to approach documentation and participatory research. But I’m speaking at it from the perspective of what funders or even other kinds of research bodies expect. One of the reasons, again, we start with research is it’s a practical space. It can help verify something that folks have been experiencing. It can create data to benchmark from. It can create a series of hypotheses to sort of test and create findings around. The next step for me, though, is activism and advocacy.

And again, I want to really contextualize this in a human rights space and context. Research obviously can be activism driven. When I say the word activism, I mean professional activism or professional advocacy in which you are creating, let’s say, a campaign. You’re creating a coalition. You’re really engaging with lawmakers, regulators, policymakers. At times, you might even be directly engaging with, let’s say, a very large online platform, or as the DSA describes them as VLOPS.

Activism, at least in my practice, sort of pulls from what we’re finding in research, again, working with communities to say, well, what are our next steps? How do we actualize what we found? What does change look like? In a lot of cases, I will then partner with much larger organizations like the Mozilla Foundation or Fight for the Future or very specific coalitions within the digital rights space. Oftentimes, this is where folks will, again, start to create very large campaigns to push for different kinds of changes.

For me, it’s important that whatever research I’m sort of doing in step one, that it’s not sort of ending. Like, let’s say once a paper is submitted, we’re still staying with the idea that the ideas and findings have to go somewhere. Right. So this is where this sort of next step of advocacy, if you will, is incredibly important. What did we find? What are things we’d like to see changed and why? But almost, not almost, but also more importantly, is this last step of art. So in my process, art is sort of almost coming at the end of these two other steps.

Art can visualize things are invisible. They can make tangible current urgencies. They can make it explainable to audiences. Art can do a lot of what, let’s say, a journalism op-ed cannot do. Right. Art can illustrate, it can show, it can transform. And so for me, art is a very integral part of this sort of research and advocacy process inside of my personal practice. And again, I also want to highlight this is like my process. It doesn’t have to be everyone else’s process, but it’s how I approach thinking about work and thinking about impact.

Another way to describe this, though, is research driven art. And it’s a bit of what it sounds like. It’s art that is shaped and driven by research, be it mixed methods or other forms. It’s not necessarily data visualization, though I do make quite a bit of that, but it could be. But one of the ways I think about it is what if you made net art or technology based art with the same principles and ethos as photojournalism? Like photojournalism, research driven art uses specific structures and a sense of purpose to constrain it. These constraints work much like how the skeleton works. They stabilize the practice just as the rib cage stabilizes a body, but they do not define the entire practice nor keep it from moving and flexing on its own. So in this way, the research I do stabilizes and shapes my art, but it does not dictate the outcome.

Or another way to think about it is when I’m often starting an art project or part of my art practice from the previous two steps, I don’t know what the outcome is going to be. I don’t know if I’m making a painting or a photograph or an experimental video or an installation. Sort of the process of doing the art, doing sort of the art making from the research is what starts to determine what the artistic outcome is. So I’m never really approaching a topic with an idea in mind of like, yes, I’m going to make a massive sculpture. It’s usually at the very end where I’m like, ah, this is what the art will be now from these ideas.

Often what I’m seeking is an idea of usefulness. One of the ideas that my artwork is playing with is this idea of usefulness and how usefulness and interdisciplinary work is a necessary part of a research-driven arts practice. And this space of usefulness is directly inspired by Tanya Bruguera’s Arte Utile methodology, which means utilitarian art. Arte Utile draws on artistic thinking to imagine, create, and implement tactics that change how we act in society. Sort of, again, focused heavily on usefulness and on tool building and on communities.

For non-artists in the room, I’ve noticed that there’s often this sort of dichotomy, if you will, between aesthetically driven art and utilitarian art. In the sense that a lot of curators and even traditional contemporary art institutions, while they find usefulness interesting, what they actually are much more interested in is sort of, let’s say, beautiful or aesthetically driven art. That art should be much more opaque and gestural. Arte Utile actually really stands in opposition to that. It is really focused on usefulness. One could even think it’s very direct in terms of what it’s trying to achieve. So within an art space, this is, I would argue, quite different and almost quite revolutionary, if you will. So Arte Utile draws on artistic thinking to imagine, create, and implement tactics that change how we act in society.

Noor Khan, who I spoke to a few years ago when I was putting together this presentation, highlighted the strengths of work stretching across many domains, making art a necessary Trojan horse to discuss useful change. So for me, this is where I turn to the work of American artists, whose work we see here. Frances Singh, Joanna Moll, Adam Harvey, Mimi Onuoha, Forensic Architecture, and obviously many others. So these artists are pulling from research or investigatory-based practices and with work that manifests into a variety of outputs, artifacts, writings, and education. So the work of the practices of American artists or the anonymous group behind this work called ScanMap are great examples of how social justice and human rights driven art. With ScanMap’s current work tracking police scanners or American artists’ work that you see here, I’m Blue, If I Was Blank, I Would Die, and My Blue Window, two pieces that comment on the structure and violence of modern police forces.

So as I was mentioning sort of earlier, these sort of aforementioned works of Dune and Rabi, Mimi and Oja, Adam Harvey, who we see here, this is his Vframe work, can be viewed solely as art or as research, but are much more richly seen when viewed together as research-based artistic activist practices. So in this case, when we look at Adam Harvey’s Vframe work, which was nominated for an Ars Electronica award, this is a computer vision project that he’s worked on with Mnemonic, formerly called the Syrian Archive, where they would scrape images from social media, again, aspects of digital witnessing, and they would gather these images before they were deleted, because it often violated the terms of service on, let’s say, something like Twitter.

These images are incredibly important for firsthand documentation of war crimes. These are images we need. These are images that human rights organizations need, right, to verify and prove. Adam was also using these images to do something I think very interesting. What he was trying to do is look at and analyze parts of ballistics inside of Syria to understand where the ballistics came from, like who funded them and who made them. So he has made this non-open source, this closed source computer vision model that can actually start to identify, again, parts of ballistics.

Now, why is this an art project? An easy answer is that Adam is an artist and he considers it art. And I like that answer, and I think that that’s actually a fine answer. I think a better and also more complicated answer that I think Adam would agree with is also that this, again, sits in this argument I’m making of utilitarian art. And part of arte util acknowledges that a work an artist makes is still a work of art, even if it’s done in a different context. So the creation, let’s say, of a mutual aid network, if created by artists, can also be a form of their artistic practice, and we should acknowledge it as such. We should not see it as separate, right? We should not see it as separate from an installation they make, but as part of this much more cohesive practice. I’m a big fan of that.

And I also think this highlights some of what art can allow us to do. Art can allow for experimentation. Art can allow for different kinds of, again, this interdisciplinary collaboration. Art, as Noor Khan mentioned, can be this Trojan horse to do all different kinds of research, especially also in contexts in which it’s dangerous to be an activist. So art actually provides both a shield and a space of exploration. But I think that there’s something really important, again, in this staying in this space of usefulness and sort of pushing the boundaries of what art should look like or what it could look like. And so this is why I often really like to show this, again, this screenshot, because to me, this is both necessary human rights technology and also a piece of art. And I like that it sits in both of those places and pushes us and complicates us to think about art. And that’s why, you know, in this context, it becomes a form of this activist-based artistic practices. And this is the context in which design and actually research should be up here and art all sort of come together for me.

And while that’s a very wordy way to describe this form of art, I think it rings true of the intentions of the work. The works just aren’t to bear witness, though that alone would be worthwhile. They question, provocate and offer a solution to a problem.

This should not be viewed as a form of techno solutionism, however, because the solutions here the artists provide are not meant to create an end to all other potential solutions. They’re just one solution. They’re just one way to help sort of alleviate or, you know, sort of mitigate a slice of a much larger problem. I think at times they serve to offer temporary or open source fixes for gaps in inequity and violence created by society. And then via the creation of those projects, they become poetic witnesses of those gaps. We can think of this as a bandaid, as we say in the States, or a small bandage or a plaster. But this kind of bandaid is a similar space in which I pursue my own practice.

Bandaid as art exists as necessary provocations or patches while with participatory design and deconstruction. Artists, along with human rights researchers, activist technologists and communities can overhaul these systems or even potentially destroy them together. The togetherness and collaboration is key, though. But nonetheless, provocations within art and design can create new imaginaries for new realities and at times solutions we never saw possible.

So this brings us to feminist data set. So feminist data set is a multi-year project using intersectional feminism as a framework for investigating machine learning. We are using Professor Kimberlé Crenshaw’s work and definitions on intersectionality as a guiding point. Part of this is a data set. I will say the name is now a bit of a misnomer as it’s not just about data. But this is a trans inclusive data set and project that also focuses on racial justice. It’s extremely process driven. So the outputs of the project or the artifacts, if we’re using artistic language, the artifacts of the of the project are workshops on data and machine learning, different kinds of essays, research, printed matter and documentation. Within this project, we also utilize the data feminism white paper, the design justice principles and feminist manifest now, along with many other works in feminist STS studies as frameworks for investigations for participants to consider or utilize within the workshop.

I will explain this later in more detail, but one thing to sort of bear in mind is that with every workshop, we’re almost sort of starting over in a way, because the participants are allowed to sort of pick and design the parameters and the focus of the data set. They also decide things like how open it is, like who gets to use it, or at times are very lightly designing a governance model. These are things that I am actively trying to sort of write down and document to turn into a paper, but one of the downsides of this being, let’s say an art project is it’s hard to have the time to do that. One of the upsides of this being an art project means that we can be very iterative. So if a certain thread emerges in a workshop, that can be a thread that we tug and pull on for many months or even years, years to come. It also means that we can do sort of specialized workshops as well.

Currently, there are two nodes or steps within Feminist Dataset, though there will be many more as we continue to go through the machine learning pipeline. The first node has been data collection, the second is data cleaning. What you see from this photograph is from a project where we were collecting data as a public wall in a gallery where visitors could add notes of the data. That was the very first slide you saw. This is from our installation at Soho 20 Gallery. In this photo, we’re now sorting the data into very, very large groups.

In this photo, we were also sort of determining different large labels and buckets to consider with this amount of data. Within the project, these workshops are key as they are a mechanism to think through what community-driven equitable data collection could look like. In the workshops, we often discuss intersectional data, what does it mean, and how does that apply in this case to Feminist Dataset. Also, in these workshops, I gave a brief breakdown of machine learning data and the politics of data collection and how data is used in software and technology. Then, what we determine in the workshop often becomes very specific and contextual to the group of people that are there, which I find incredibly interesting.

As I was mentioning at present, I’m attempting to write about the findings and the varying frameworks that have been proposed. The most interesting is we do not have one total Feminist Dataset framework, but many notable findings to the workshops. Things that could be grouped together under different kinds of themes. A way to explain that further is in one workshop, all the participants were incredibly open to the idea that anything we worked on would be open source and that even for-profit companies could use it. In other workshops, people often heavily disagree with that, for example. Again, this is how the workshops, even on that minor note, will very much differ by who is in the room. That’s one of the reasons why I have decided to keep things specific to the workshops to honor what the different participants in those workshops wanted. This is just an example of how, in an art context, we have installed some of the work.

The screen is bits of text from what is in the dataset. On the table is different diagrams we’ve made to think through feminist data, as well as a very short manifesto we wrote in the very first workshop. This was us during COVID, because we kept going, as everyone did, I imagine, or I hope. This is on data collection. Within this data collection, we move very slow. The joke we say is this is farm-to-server table style data. Again, we often utilize different methodologies from data feminism, feminist manifesto, and many others asking what is legible data, what is consensual data, what is transparent data. Within the workshops, we have to go through and specify and define each of those points.

When folks say they want it to be transparent, we spend quite a bit of time actually determining what is transparency, like for us in the workshop. What does that mean? Do we list our names? Do we list how long it took? Do we list how much money the workshop was or was funded under? Do we list how we decided what went into the dataset? A big point is actually sort of taking these larger terms, like transparency or consent, and saying in practice, right now in our workshop, what does that mean? When we say consensual data, how do we define that? This project is really dealing with tensions and tradeoffs and bigger questions about technology. Often, we talk about the methods for citing data and storing it and referencing other literature in the field of equitable data, ethical AI, responsible AI, and feminist technology studies.

One of the things that this workshop is also very interested in is how difficult is it to follow the different steps or principles outlined also in different kinds of feminist literature? Where in the reality of making, do things start to fall apart and why? That’s one of the things that not only have I been documenting in feminist dataset, but something when I’m teaching responsible AI, something I have my students also work through. How difficult is it to follow, let’s say, these methodologies and why? Where do you have to make a concession? What is the concession? Does that still fit under a feminist framework?

Again, from these workshops, we’ve been collecting suggestions, frameworks, and protocols from the participants that hopefully will be published soon. As I was mentioning, again, transparency and consent often pop up all the time in these workshops as they should. For feminist dataset, generally, we list out the data we find and we list the person who submitted it if they would like to be listed. One of the things we’re grappling with is should we notify the authors? We include data that is self-published, alternatively published, academically published, etc. to be as inclusive as possible.

In some cases, again, this is really split. Some folks think that we shouldn’t have to notify the authors, but if they were listed in our dataset, that they can remove themselves. This is, I think, an interesting point. It feels like a minor point in the project, but I think all minor points under feminist theory are major points. How do we grapple with that? We could look to how archives and libraries are run. Is that a model we would like to follow? Why did we choose that model? Again, these are things still to be determined and that need to be determined.

I should also point out the project isn’t about gathering feminist data. It’s actually something a little bit more complicated. It’s finding texts that manifest intersectionality within the writing. For example, if someone wanted to submit an article on income inequality, an article that only represents the differences between men and women and posits those genders as a monolith is not intersectional. But an article that talks about the differences in payments between Black, brown, white, Asian, and Indigenous fems, along with non-binary folks, trans women, etc., would be intersectional and could be submitted. This is also why the data is very slow. In the workshops, we encourage people to read what they are submitting and really think about how does this manifest or engage with intersectional text. It’s not just the content or the domain of what the article or book or poem is about because it’s any kind of text. It is actually also how does it manifest or engage with intersectionality.

So I don’t know how many of you have seen this project before, but this is Thomas Twaite’s toaster project, which is a critical design project. Thomas Twaite’s builds a commercial toaster from scratch, from melting iron ore to building circuits and creating a new plastic toaster body mold. And then this dataset operates in a similar vein to this, just in terms of software.

And so I wish that there was a visceral way to sort of visualize sort of how broken our model will be because I’m sure as you were hearing, we’re taking all different kinds of text types. So it will not be necessarily very accurate or maybe not even very, like, useful useful. But what I’m interested in is very similar to Thomas in terms of what does it mean to hand build small technology. It’s not going to work the same way as big technology. And so then in a way this becomes, I think, a useful sort of visual image to think about what it is we’re trying to make. Because what does it mean to thoughtfully make with machine learning, to carefully consider every angle of making, iterating, and designing?

Every step of this process needs to be thoroughly reexamined through a feminist lens, and like Twaite’s toaster, every step needs to actually work. And so what we’re building is, again, very misshapen. Twaite’s toaster was a critique on thinking about large scale fabrication systems. So he took something as ubiquitous as a toaster, wanted to show actually how difficult it is to sort of make one. And again, this is a sort of a comment on mass industrialization and the impacts it has on the environment. What I’m interested in is something on a smaller scale of when we think of critiquing large tech, how possible is it for us to intervene? And this is, again, some of the deeper parts of feminist data set. Is feminist data set still a feminist project if I have to use AWS or I’m doing this on a MacBook Pro or I need to engage with any kind of Nvidia software stack? Those are not feminist technologies and they’re not necessarily feminist companies, but that’s the reality of working. And again, this is where I’m interested in concessions.

One of the parts we’ve been picking apart is accessibility and documentation. Is it better for us to make our own model or should we be using web ready APIs so other folks can actually have an easier time replicating feminist data set? And again, these sound like small things, but I actually think that they’re incredibly important in the context of the entire project. Those decisions are not offhanded decisions. I think it’s important to articulate why a decision was made and how that impacts, again, how that impacts the project. I know I think we’re running. I’m worried we’re running a little behind. I’ll just do two more slides and then I’ll try to quickly finish. But again, truth be told, this should have been the first slide. The inspiration behind a feminist data set was a little bit of a dare and a lot of hope. I’m an artist, designer and researcher, which means I’m ultimately a maker. So I want to make things and I want to test things. And the goal here is to take all different kinds of really outstanding work and methodologies and put them to the test by saying, how will we build software, maybe commercial software about machine learning? What does it mean to do it by hand and to scale it down? And how is it possible? Is it possible? Sort of bake in intersectional ethos.

Again, if we had more time, we could interrogate what all of these different words mean. For example, should it be open source? Is open source ethical or feminist? What are the drawbacks to it? I say that with a laugh because I used to work at the Wikimedia Foundation and I can tell you all the pros and cons personally of open source. But again, I don’t I don’t want to digress.

But back to this project, because it is, again, an art and research project that’s using varying forms of participatory methods to ask, what is feminist machine learning and then how can we create it? So by its nature, this project is about process, about grading and documenting the friction of working together and just of working and acknowledging that it’s there because applying a theory to reality will not be perfect. It will be bumpy. And so over time, this is actually the heart of feminist data set and what the project has transformed into in a way is the decisions that we make within the workshops around the larger parts of the project. And that’s something I would like to actually much more deeply invest in, potentially as a PhD or an academic project, actually. I realize I’ve mentioned all of this.

I can get more into the failures of the project later. And by failure, I mean these as as not like total failures. But for example, we try to make the project as accessible as possible. We try to make it free. We try to offer food. We try to raise money for folks to be paid to attend. But I’m sure if any of you have ever worked with volunteers, those that have the time to attend will be the ones attending. Right. Even if we do it online. So by nature, the folks that end up participating end up being of a certain race, of a certain class, or often end up being a lot of students. But it ends up being folks that let’s say do not have a lot of dependables and at times can sort of justify being there. That also then impacts the types of data we’re collecting, as I’m sure you would not be shocked to hear, even though we’re open to things being alternatively published, folks end up finding a lot of white creators. Right. If we just think of how the Internet itself is indexed and sort of what populates up to the front of a search engine, if you will. And so all these things will still even with taking massive steps to try to avoid that, that’s still something that pops up in the project. And again, I consider this a failure, but not a total failure, but let’s say an interesting finding, but a failure of the project, if you will.

Really briefly, this is our second note, then we can open it up to questions. This is called TRK or Technically Responsible Knowledge. It’s an open source project that examines wage inequality and creates an open source alternative to data labeling. So any of you can download our code. This is our wage calculator. But this is, for example, where you could use this to help label different kinds of image or text based data sets. Part of this was in Feminist Dataset we realized we needed a way to label and sort of clean data. And we did a lot of work in looking into different tools and projects to use. And it kind of came down to perhaps we should try to make our own. But this was also the project starting to think about micro service labor and the interventions around that. And I was also thinking about the act of data labeling.

Could this ever be a feminist act? So zooming out, part of that’s thinking, what does a workplace or collaboration need for the labor itself to be honored as feminist work? How do you how do you create an equitable work environment to just do data labeling? And that’s, again, part of this larger conversation that Feminist Dataset is looking at.

Additionally, one of the things to also think about is what does it feel like to do this labor every day for eight or nine hours or 10 or 12 hours a day? This is a part of larger works I’ve been doing with the Weizenbaum Institute with Dr. Milagros Miceli, thinking about, again, interventions between or with data workers and academia. So this is, again, much like Feminist Dataset, other projects sort of bleed into it. Other findings are bleeding into it and vice versa. Sort of this osmosis.

I’ll just read this last thing, then we can go to questions. If machine learning pipeline is death by a thousand cuts, I think of TRK as one bandaid for one small cut. And I just want to sort of show the wage calculator. The project doesn’t propose a solution for all issues related to machine learning or even like a minor one for Mechanical Turk, because so many issues related to machine learning are issues of a deeper, more ingrained societal inequity, which can only be addressed through large shifts and restructuring of society and legislation. But within that, as an artisan researcher, I try to look at what kinds of work can help alleviate or expose these issues.

So this wage calculator TRK focuses on how through pricing structures, platform incentives and the invisible nature of gig work, clients underprice, undervalue and fundamentally misunderstand how tasks are handled in human as a service platforms. Part of the design thinking behind TRK is to examine this equity or is to examine equity and transparency within interfaces and the design of tools and what problems tool design UX and UI can create in technology. Inspired by the data sheets for datasets white paper, TRK injects plain text information into the dataset with information that includes a description of what the dataset is and when it was made. UX or user experience design is a utilitarian intelligence focusing on architectural layouts, usability and user flows. But design has a politics to it. It can suppress or uplift content.

Design, much like technology, isn’t neutral. So as an artist, I use design as a material to confront and comment on the slickness and inequity of for profit technologies. Yeah, so thank you, because I know we want to have time to chat.

Thank you very much, Caroline. That was great. Super interesting presentation and the project that you showed. We have several questions in the document, as I mentioned. I’m not sure we’ll have time to go over all of them, but we can pick some of them. And I would like to start and then maybe also we have Jenny and Vera’s panelists and if they want to jump in and ask questions, please feel free to do so. Since this salon is called Regaining Power over AI, I will start by questions around activism, because also I find that it’s pretty interesting that you put this as part of your methods. And I saw that there is two questions about this part. And first, I will just ask the two questions and then I’ll let you answer. So the first question is, is activism a practice that we need to be better at to avoid technology like AI being driven or by others or serve others? And related to that, the second question is that activism is often a collective effort. Are you also working in a collective? And if so, what are the others? I mean, are they artists, journalists, activists, researchers? Lay people?

Totally. That are the two questions. I’ll start with the last one. Yeah. So when I’m doing any kind of human rights work and let’s say we’re engaging in research, I’m not sure if this person was here for the beginning of the of the workshop. It’s important to sort of take those findings and think about, OK, well, what like what does that sort of say about technology and how do we improve whatever friction we found? So I’ll give a maybe very direct example.

I was working on a project looking at barriers to secure messaging, working with abortion activists in Louisiana. Abortion is banned. So I’m here in Louisiana and some of the work that I do in regards to that right now is legal in terms of the activism. But questionable how long it will be. So we were working with different mutual aid organizations. Part of this was understanding, trying to understand barriers that they had or questions they had around secure messaging, a way to also sort of we don’t ever like to go into a space and just sort of take and not leave. So often we do a lot of knowledge exchange.

In this case, we did quite a few security trainings for those organizations and also gave them a detailed security plan to follow that fit their needs. We also worked with journalists in India who have been victims of spyware and have been victims of intense brutalization and censorship by the current administration in India. In these cases, we also did the same thing in which we also provided very specific security trainings for post our user interviews, along with sort of connecting folks with other organizations that could help with more detailed security needs they had, such as digital forensics, if they thought their phone had been compromised.

We found a bunch of things from this work, which was funded by Omidyar. One of the things we found was a potential sort of issue in regards to link sharing on Signal and WhatsApp. If you’re already being heavily monitored by a government that you’re in, let’s say they’re already monitoring your telecom and your internet, your ISP provider. We found that on link previews, which is a very normal thing to have on your phone, this is not anything probably anyone in this room needs to worry about, it creates very specific URLs. So if you’re sharing it, let’s say I took a URL of a Google Doc or my website, let’s say, and I shared it to Baptiste, it creates a unique sharing URL that Baptiste is now receiving. So it’s verification that we’ve been in contact, even if we’re messaging over end-to-end encrypted systems.

This is something we internally worked really hard, reaching out to Signal and WhatsApp to try to put out some documentation on this, or at least on WhatsApp’s end, to allow link sharing to be turned off and on. Signal’s amazing with turning things off and on. You can configure a lot of settings. They didn’t want to share this, sadly, but we also kept thinking about other findings we had had in terms of how Facebook could potentially be violating GDPR in terms of Facebook Messenger, because we looked at Messenger, Telegram, iMessage, Google Messages, Facebook Messenger, WhatsApp, and Signal.

And then this led us to actually sort of starting or being a part of a coalition with Fight for the Future and the Mozilla Foundation on secure messaging, specifically then focusing on Slack. We had done some prior research on Slack at our organization. And so some of this was pushing for different kinds of UI that would make it easier if you were facing harassment on Slack, like a block button. We got something close to a block button, but not a full block button from Slack.

But we were also pushing for end-to-end encrypted DMs if folks happen to be doing, let’s say, like union organizing in a workplace, that they can sort of safely talk. Slack was something that popped up a lot in terms of us engaging with the abortion activists in Louisiana, because a lot of them were using tools like Slack, which makes sense. Like it’s a tool that they’re used to in a work context. And a lot of context folks just actually don’t understand how security manifests, because security is complicated. It’s a particular part of technology. So in those cases, we’re working with larger coalitions.

In other examples, I’m a part of a series of very local-based coalitions, especially one here in New Orleans called Ion Surveillance. So in those contexts, we’re working together in terms of very localized issues, often trying to sort of apply pressure to the city council of New Orleans. My organization also is on other coalitions. So we’re in the Coalition Against Online Violence, for example. And so often out of that, there’s like immediate sort of response campaigns we’ll do, let’s say, if a journalist is jailed, trying to sort of create international awareness of that particular journalist.

So yeah, these are like never happening alone. I would say on the human rights side, almost all campaigns have like many, many campaign partners. It’s, I would say, rarer that, you know, like, let’s say like a Mozilla would launch a campaign and not tell anyone or engage with them. A massive thing we’ve also done, Fight for the Future, campaigns around abortion and technology that launch at South by Southwest. So like months before those things launch, we’re like reaching out to many different organizations, asking them to be a part of the message, asking them to share it with their organizational members. You know, these things might seem small, like even creating like social media assets. But the goal is often situate around very specific asks and then trying to galvanize all of our different communities.

And then the way that these often sort of come about is in a lot of these cases, these are things that specific community members have asked for or requested. So like with Fight for the Future, we were working with Plan C, Pills, Repo, and Censored and a few other abortion access or a few other abortion related organizations. They might provide security or privacy support. They might provide funding like a mutual aid group. And they didn’t want to be named, but they were very involved in shaping those campaigns.

So, yeah, like activism, good activism never happens alone. And then I guess the other question was about activism and research. This is just an example of like, yeah, we find stuff and we’re like, we have to do something about it. But that’s our remit as a human rights organization. Like if we find something, we’re like, we have to do something with this, like it has to go somewhere.

Thank you so much for all of that. I know we only have a couple of minutes left. And so I wanted to ask one, just one other question that’s actually sort of related or building off of what you’ve just talked about, which is around audience. There’s a question in the document. The idea of usefulness refers to an audience. And so what audience are you thinking about in your artistic practice? I think similarly, you’ve talked about some of the activist work that you’ve done sort of targeting the actors that may have influenced to change some of the behavior. I think artistic practice is an interesting question to ask around audience because it may be slightly different. There may be sort of indirect activism that is that where the goal, the goal isn’t sort of a direct lever, but more indirect. So and I think there’s a sort of adjacent question to that from the document, which is also sort of considering the modes of distribution or dissemination of artistic work. What are the limits or affordances of each?

That’s a lot to answer in a couple of minutes. But I guess I’ll start with the limits or affordances of art. It is at times very limiting. Like I’m not I’m not a famous artist, you know, like I’m not a like a Refik Anadol, which has like a much larger he’s got a much larger platform and reach. But I think I think art provides, again, this sort of safe space to, I think, do very experimental things or challenge folks to think about technology in a very specific way in which that if feminist days were, let’s say, a startup, we wouldn’t really be able to do. So I think in that case, like it provides it.

The Trojan horse sounds weird to keep reusing, but that’s really kind of what it is. It provides a context of asking a lot of sort of challenging and reflective questions, also sort of proposing ways to engage. But it does it in a in a space where folks feel very safe as non experts to engage. And that’s really important for me if we think about, let’s say, larger themes of data governance. That’s something I’ve been working on with DAX over in the UK, thinking about how right now there’s a government inquiry in the UK on AI and copyright and data. And they’re proposing a model that’s like sort of very binary of like an opt out. So there might be one kind of entity that would hold all of this artist’s work and you opt in or opt out. And the companies would have to engage with that. That’s not a really great reflection of how copyright works, because like an individual image or piece can have many different licenses to it.

So one things we’ve been thinking about, though, is like collective data, data governance for folks that are interested in, let’s say, sharing some of their images under different kinds of licenses. And Feminist Dataset is actually able to sort of do those things without saying that because you think you’re going to an arts workshop. Right. And you are. And that’s actually like what the workshop is. But it becomes a much easier interface, if you will, for folks to engage with versus if you were saying, like, come to a policy LARP and we’re going to do this. But that’s kind of what Feminist Dataset is. It’s a presbytery policy LARP.

But so I think art actually provides this space, both of safety for the general public to try to grapple with a lot of these complexities and also understand more about what what these what these things are. I forgot the first part of your of your question, Jenny, but I think some of it was about the direction that the research takes. A lot of the artistic work is coming out of research. So even if it’s not, let’s say, like funded research in my human rights lab, it’s research that the art project will just sort of do. So the audience sort of ends up being a variety of different folks.

So with TRK, some of it was thinking about artists I knew that didn’t really think about payment systems. And then it also we started or I started gathering data on like people that work in tech that just sort of need something done. Right. And they’re also not necessarily thinking about like is 10 cents or a dollar or 20 cents per labeling. Like, is that a good amount? They’re like, it’s just an amount like this is just one thing someone is doing. And it’s like, well, no, there’s a person behind the screen and this is their job all day. And like the one thing you’re doing is like or the one thing they’re doing for you is a part of a thousand tasks. And if you don’t see your task as a part of the entire whole, then even if you’re trying to price fairly, you can’t. And that’s where sort of the calculator came from.

So part of it was trying to educate other artists. And then also from reading a lot of different papers, also sort of becoming like I sort of already knew that this was a problem, but it was sort of then very validated again and again that this is also generally a problem in terms of how folks interact with data workers, because they can’t. There’s not necessarily a lot of overlap as a client with the particular data workers. You are sometimes working with the organization or the company and you’re not necessarily able to directly engage with folks. So there’s all these sort of other different issues in regards to like the labor of data cleaning. And so that is kind of what fed into the project.

Then also with the awareness, so a lot of our audience are like previous participants and reoccurring participants. Right. And so that’s kind of where then the structure of TRK and like why maybe we should make our own came from, was thinking about and having had conversations with participants like what kind of like would you want to label data? So we were like, yes, we would love to do that for the project. I’m like, OK, well, we should pay you. I want to be paid. I’m like, you should be paid.

Then like what kinds of tools do you want to use and how do we actually have an equitable back and forth? And like, what does that look like? And realizing that all the tools out there had these barriers between client, who would sadly be me, and the labeler, who’s the worker. And so that’s kind of where it comes from. If I were probably, I don’t want to say like a smarter artist, but a much more strategic one. I think sadly, my audience would be curators of thinking like, how do you install this? But that’s not how I like to make work. So it really did become who are, who is our community, who are like the workers, we’re all the workers.

So like what are what are ways to make these tools cognitively easier to engage with to do this work? And that’s a massive part, I think, of, again, sort of how FemisDataset engages with machine learning. A lot of data workers, this is what, this is why I’ve been so happy to spend the past two and a half years as a fellow at the Weissenbaum Institute with Dr. Milagros. This is her, a lot of her big focus, right, is labor equity with data workers. The data workers are, it’s an intellectual skill. This is a real job, but they are experts in machine learning. You can’t, on the other side, as a data worker, tell the client, actually, I think there’s a faster or better way to do this. And these are all these kind of issues that are hard to surface in art. And I don’t think TRK does an amazing job of surfacing these things, but I don’t think it could surface all of them.

But again, these are a lot of the things that Feminist Dataset considers and is considering as an art project.