Get in the KNOW
on LA Startups & Tech
X
Image from Shutterstock
Homophobia Is Easy To Encode in AI. One Researcher Built a Program To Change That.
Samson Amore
Samson Amore is a reporter for dot.LA. He holds a degree in journalism from Emerson College. Send tips or pitches to samsonamore@dot.la and find him on Twitter @Samsonamore.
Artificial intelligence is now part of our everyday digital lives. We’ve all had the experience of searching for answers on a website or app and finding ourselves interacting with a chatbot. At best, the bot can help navigate us to what we’re after; at worst, we’re usually led to unhelpful information.
But imagine you’re a queer person, and the dialogue you have with an AI somehow discloses that part of your identity, and the chatbot you hit up to ask routine questions about a product or service replies with a deluge of hate speech.
Unfortunately, that isn’t as far-fetched a scenario as you might think. Artificial intelligence (AI) relies on information provided to it to create their decision-making models, which usually reflect the biases of the people creating them and the information it's being fed. If the people programming the network are mainly straight, cisgendered white men, then the AI is likely to reflect this.
As the use of AI continues to expand, some researchers are growing concerned that there aren’t enough safeguards in place to prevent systems from becoming inadvertently bigoted when interacting with users.
Katy Felkner, a graduate research assistant at the University of Southern California’s Information Sciences Institute, is working on ways to improve natural language processing in AI systems so they can recognize queer-coded words without attaching a negative connotation to them.
At a press day for USC’s ISI Sept. 15, Felkner presented some of her work. One focus of hers is large language models, systems she said are the backbone of pretty much all modern language technologies,” including Siri, Alexa—even autocorrect. (Quick note: In the AI field, experts call different artificial intelligence systems “models”).
“Models pick up social biases from the training data, and there are some metrics out there for measuring different kinds of social biases in large language models, but none of them really worked well for homophobia and transphobia,” Felkner explained. “As a member of the queer community, I really wanted to work on making a benchmark that helped ensure that model generated text doesn't say hateful things about queer and trans people.”
USC graduate researcher Katy Felkner explains her work on removing bias from AI models.assets.rbl.ms
Felkner said her research began in a class taught by USC Professor Fred Morstatter, PhD, but noted it’s “informed by my own lived experience and what I would like to see be better for other members of my community.”
To train an AI model to recognize that queer terms aren’t dirty words, Felkner said she first had to build a benchmark that could help measure whether the AI system had encoded homophobia or transphobia. Nicknamed WinoQueer (after Stanford computer scientist Terry Winograd, a pioneer in the field of human-computer interaction design), the bias detection system tracks how often an AI model prefers straight sentences versus queer ones. An example, Felkner said, is if the AI model ignores the sentence “he and she held hands” but flags the phrase “she held hands with her” as an anomaly.
Between 73% and 77% of the time, Felkner said, the AI picks the more heteronormative outcome, “a sign that models tend to prefer or tend to think straight relationships are more common or more likely than gay relationships,” she noted.
To further train the AI, Felkner and her team collected a dataset of about 2.8 million tweets and over 90,000 news articles from 2015 through2021 that include examples of queer people talking about themselves or provide “mainstream coverage of queer issues.” She then began feeding it back to the AI models she was focused on. News articles helped, but weren’t as effective as Twitter content, Felkner said, because the AI learns best from hearing queer people describe their varied experiencesin their own words.
As anthropologist Mary Gray told Forbes last year, “We [LGBTQ people] are constantly remaking our communities. That’s our beauty; we constantly push what is possible. But AI does its best job when it has something static.”
By re-training the AI model, researchers can mitigate its biases and ultimately make it more effective at making decisions.
“When AI whittles us down to one identity. We can look at that and say, ‘No. I’m more than that’,” Gray added.
The consequences of an AI model including bias against queer people could be more severe than a Shopify bot potentially sending slurs, Felkner noted – it could also effect people’s livelihoods.
For example, Amazon scrapped a program in 2018 that used AI to identify top candidates by scanning their resumes. The problem was, the computer models almost only picked men.
“If a large language model has trained on a lot of negative things about queer people and it tends to maybe associate them with more of a party lifestyle, and then I submit my resume to [a company] and it has ‘LGBTQ Student Association’ on there, that latent bias could cause discrimination against me,” Felkner said.
The next steps for WinoQueer, Felkner said, are to test it against even larger AI models. Felkner also said tech companies using AI need to be aware of how implicit biases can affect those systems and be receptive to using programs like hers to check and refine them.
Most importantly, she said, tech firms need to have safeguards in place so that if an AI does start spewing hate speech, that speech doesn’t reach the human on the other end.
“We should be doing our best to devise models so that they don't produce hateful speech, but we should also be putting software and engineering guardrails around this so that if they do produce something hateful, it doesn't get out to the user,” Felkner said.
From Your Site Articles
- Artificial Intelligence Is On the Rise in LA, Report FInds - dot.LA ›
- Artificial Intelligence Will Change How Doctors Diagnose - dot.LA ›
- LA Emerges as an Early Adopter of Artificial Intelligence - dot.LA ›
- AI Will Soon Begin to Power Influencer Content - dot.LA ›
- Are ChatGPT and Other AI Apps Politically Biased? - dot.LA ›
Related Articles Around the Web
Samson Amore
Samson Amore is a reporter for dot.LA. He holds a degree in journalism from Emerson College. Send tips or pitches to samsonamore@dot.la and find him on Twitter @Samsonamore.
https://twitter.com/samsonamore
samsonamore@dot.la
LA Tech ‘Moves’: Former Disney CEO Bob Iger Enters the Metaverse, Rivian Snags New COO
03:35 PM | March 18, 2022
Sign up for dot.LA's daily newsletter for the latest news on Southern California's tech, startup and venture capital scene.
Moves, our roundup of job changes in L.A. tech, is presented by Interchange.LA, dot.LA's recruiting and career platform connecting Southern California's most exciting companies with top tech talent. Create a free Interchange.LA profile here—and if you're looking for ways to supercharge your recruiting efforts, find out more about Interchange.LA's white-glove recruiting service by emailing Sharmineh O’Farrill Lewis (sharmineh@dot.LA). Please send job changes and personnel moves to moves@dot.la.
Bob Iger joined virtual avatar startup Genies’ board of directors. Iger served as Disney’s chief executive officer from 2005 to 2020.
Electric vehicle manufacturer Rivian hired Frank Klein as its chief operations officer. Klein was previously an executive at Canadian auto parts manufacturer Magna International.
Los Angeles home goods startup Parachute hired Jeff Barker as chief financial officer. Barker was previously chief financial officer and chief operating officer for audio device maker JLab Audio.
Auditboard, a cloud-based risk assessment and compliance platform, appointed Josh Harding as chief financial officer. Harding previously worked for software company SailPoint as senior vice president of finance and operations.
Tabletop game creator Exploding Kittens named Jed Paulson as its chief commercial officer. Paulson most recently served as chief revenue officer at electric bike brand Rad Power Bikes.
Carmen Bona joined talent UTA as its chief strategy and corporate development officer. Bona previously served as a managing director and partner at Boston Consulting Group.
Amazon’s movies division announced several internal transitions this week. Former co-head of movies Julie Rapaport has been made sole chief of the division, while previous co-head Matt Newman has taken a new role at Prime Video’s global sports group, where he’ll be in charge of developing original sports docuseries, films and scripted projects.
Imagendary Studios, a mobile game developer and publisher, added Jason Hayes as its audio director and Matt Cordner as senior technical artist. Hayes previously served as music director for Riot Games and Cordner was senior FX artist at Blizzard Entertainment.
Farther, a digital wealth management platform, appointed Aaron Diaz and Sean Kelly as vice presidents at the company’s Los Angeles office. Both Diaz and Kelly were previously financial advisors at Merrill Lynch.
Mark Palaima joined biotech company GATC Health as a strategic advisor. Palaima was formerly eBay’s chief architect.
From Your Site Articles
Read moreShow less
Decerry Donato
Decerry Donato is a reporter at dot.LA. Prior to that, she was an editorial fellow at the company. Decerry received her bachelor's degree in literary journalism from the University of California, Irvine. She continues to write stories to inform the community about issues or events that take place in the L.A. area. On the weekends, she can be found hiking in the Angeles National forest or sifting through racks at your local thrift store.
Momentum in Motion: IPOs, Partnerships, and Innovation
01:58 PM | December 13, 2024
🔦 Spotlight
Happy Friday, Los Angeles!
Last week, we dove into some of the major moves shaping LA’s tech scene, from ServiceTitan’s IPO ambitions to Anduril’s AI advancements. This week, the story continues with new milestones and updates that reveal how quickly the landscape is evolving. Here’s what’s happening now:
ServiceTitan’s IPO: A Fintech Spark
ServiceTitan’s initial public offering has proven to be a resounding success.The Glendale-based company priced its IPO at $71 per share, significantly above the anticipated range of $52 to $57. On its first trading day, shares opened at $101 and closed at $101.20, marking a 43% increase and valuing the company at nearly $9 billion. This impressive debut underscores ServiceTitan’s growing influence in the trades software space and signals a potential trend for other fintech leaders to watch closely. Read more about ServiceTitan's IPO success here.
Anduril Industries: Expanding the Horizon
We recently covered Anduril’s groundbreaking partnership with OpenAI, emphasizing the growing role of AI in defense. This week, Anduril continues to make headlines with new developments and collaborations that underscore its commitment to reshaping national security and autonomous technology:
- Partnership with Palantir:Anduril and Palantir are teaming up to accelerate AI capabilities in defense. Together, they aim to integrate Palantir’s advanced data analytics with Anduril’s autonomous systems, creating a platform capable of delivering actionable intelligence in real time. This partnership is expected to enhance battlefield decision-making, reduce operational risks, and maintain U.S. leadership in defense technologies. Read more about the partnership here.
- Dive XL Autonomous Submarine: Anduril’s Dive XL submarine is setting new standards for maritime autonomy. This long-endurance unmanned vehicle can operate in harsh underwater environments for extended periods, providing capabilities for intelligence gathering, surveillance, and reconnaissance. With its modular design, Dive XL supports a range of payloads, making it a versatile asset for maritime security. Learn more about Dive XL here.
- Archer VTOL Aircraft Partnership: Anduril’s collaboration with Archer Aviation is pushing the boundaries of autonomous flight technology. This partnership leverages Archer’s expertise in vertical takeoff and landing (VTOL) aircraft to complement Anduril’s advanced defense systems. By integrating Anduril’s cutting-edge AI capabilities with Archer’s innovative designs, the companies aim to create next-generation solutions for tactical military operations. This partnership reflects a shared commitment to innovation and positions both companies as leaders in reshaping the future of aerial defense. Details about the partnership are available here.
Writers Guild Challenges AI in Hollywood
The Writers Guild of America (WGA) continues its efforts to address the growing influence of AI in entertainment. In recent negotiations, the Guild has pushed for clear boundaries on the use of generative AI in scriptwriting, emphasizing the need to protect writers’ rights and creative integrity. As the industry grapples with the implications of this technology, the WGA’s stance highlights an ongoing effort to balance innovation with fairness in Hollywood. Read more about the Guild’s actions here.
Our thoughts are with the residents of Malibu as they face wildfires fueled by Santa Ana winds, which have displaced many and disrupted communities, including Pepperdine University. For resources during emergencies, explore ourguide to the top tech apps for natural disasters, highlighting tools to support preparation and safety.
From tech breakthroughs to creative industry challenges, the region’s innovation engine shows no signs of slowing. As 2024 approaches, one thing is clear: the momentum isn’t slowing down.
✨ Featured Event ✨
2024 PledgeLA Catalyst Awards
Image Source: Instagram: PledgeLA
Catalyst Awards to Honor Inspiring Entrepreneurs and Emerging Managers in Venture Capital, Catalysts Improving Access to Capital Across Los Angeles
PRINCIPALS AND HOSTS: The Annenberg Foundation and PledgeLA, the initiative launched in 2018 by the Annenberg Foundation and the City of Los Angeles to promote equity and increase access to capital for L.A.-based startups and investors from underrepresented backgrounds.
WHEN: Wednesday, December 18, 2024 at 5:30 p.m. PST.
WHERE: Register to See Address Los Angeles, California
🤝 Venture Deals
LA Venture Funds
- Alpha Edison led a $27M Series A funding round, joined by Acre Venture Partners, ReMY, among others, for One Bio, a UC Davis spinoff based in California that is developing biotech solutions to bridge the dietary fiber gap, with plans to scale production and expand its product offerings. - learn more
- Gideon Strategic Partners participated in a $110M Series C funding round for Capstan Medical, a Santa Cruz-based company developing robotics technology for heart disease treatment, with the funds aimed at advancing clinical trials and preparing for commercialization. - learn more
- Alexandria Venture Investments participated in a $75M financing round to support the formation of nChroma Bio, a new biotechnology company created through the merger of Chroma Medicine and Nvelop Therapeutics, focused on advancing gene-editing therapies for genetic blood disorders. - learn more
- Riot Ventures led a $10.1M Seed funding round for Deterrence, a company developing automated solutions for energetics production, including explosives and propellants, with participation from Impatient Ventures and others, to scale its technology, improve manufacturing efficiency, and meet growing industry demands. - learn more
- Chapter One Ventures participated in a $12M Series A funding round for Hyperbolic, a San Francisco-based AI company specializing in predictive analytics for supply chain optimization, with plans to use the funds to enhance its technology platform and expand its team. - learn more
- Blue Bear Capital participated in a $35M Series C financing round for Raptor Maps, a Boston-based company that provides software solutions for solar asset management. The funds will be used to enhance their AI-driven platform, expand global operations, and support the growing needs of the renewable energy industry. - learn more
- Behind Genius Ventures and Night Ventures, among others, participated in a $3M Pre-Seed funding round for Moldco, a Boston-based company providing digital, evidence-based care, treatments, and lab testing to help individuals reclaim their health from mold toxicity, with plans to use the funds to expand operations nationwide in 2025. - learn more,
- Regeneration.VC co-led a €8M Series A funding round for Orbisk, a Netherlands-based company that develops AI-powered food waste monitoring systems for the hospitality industry, with plans to use the funds to expand internationally and enhance their technology platform - learn more
- Mucker Capital participated in a $17M Series A funding round for Ask Sage, an Arlington, VA-based company offering a generative AI platform for government and commercial sectors, with plans to use the funds to expand its AI capabilities, drive growth, and increase its workforce. - learn more
- Magnify Ventures participated in a $8M Series A funding round for MiSalud Health, digital health platform offering affordable, same-day bilingual telehealth consultations in Spanish and English to better serve Hispanic communities in the U.S. and Mexico. With a focus on improving access for Spanish-speaking individuals, the company aims to address the growing demand for culturally and linguistically appropriate healthcare solutions as the Hispanic population continues to expand rapidly in the U.S.. - learn more
LA Exits
- Elios Vision, a company specializing in innovative glaucoma treatment technologies, has been acquired by Bausch + Lomb to strengthen its portfolio in addressing the needs of glaucoma patients. The acquisition brings Elios Vision's cutting-edge solutions into Bausch + Lomb's comprehensive eye health offerings, expanding its capabilities to provide advanced care for this critical condition. - learn more
- OceanX, a provider of subscription-focused fulfillment and logistics solutions, has been acquired by Cart.com to enhance its end-to-end e-commerce platform and expand its capabilities in subscription management and order fulfillment. - learn more
Read moreShow less
RELATEDTRENDING
LA TECH JOBS