How LA-Based Super Hi-Fi Hopes to Change Streaming Audio Using AI

Sam Blake

Sam primarily covers entertainment and media for dot.LA. Previously he was Marjorie Deane Fellow at The Economist, where he wrote for the business and finance sections of the print edition. He has also worked at the XPRIZE Foundation, U.S. Government Accountability Office, KCRW, and MLB Advanced Media (now Disney Streaming Services). He holds an MBA from UCLA Anderson, an MPP from UCLA Luskin and a BA in History from University of Michigan. Email him at samblake@dot.LA and find him on Twitter @hisamblake

How LA-Based Super Hi-Fi Hopes to Change Streaming Audio Using AI
  • Super Hi-Fi's AI transports the skills of a trained radio DJ to digital music playlists. Spotify's former head of research Tristan Jehan recently joined as an advisor
  • Founded in 2018 by veterans of the digital music business, the company's customers include iHeartMedia, Sonos, Peloton and Octave Music Group
  • Its leaders envision a new audio listening experience — where everyone has a personalized, curated playlist, with artful, AI-generated sequences and layers of music, voice clips (e.g. news and podcasts), and branded messaging that drives new revenues to the music industry

Before the beat from "Baby Got Back" that underpins Nicki Minaj's "Anaconda" fades to silence at the song's end, a sound clip pops up, right on rhythm and with a similar energy, telling the listener what streaming service they're listening to. A new track seamlessly takes the baton from the Minaj song before the brief branded message concludes, and continues the upbeat mood as a music bed for a rapid sequence of audio clips – first a voice imploring listeners to get hyped, then a word from Kanye about his interview with Beyoncé, a snippet from that interview, and another in-the-spirit advert – before blending into the intro of the next song, Kanye's "Stronger": all of it interwoven as if it were a single track produced in a recording studio.


It is the automation of this art, once the preserve of skilled radio DJs, that Super Hi-Fi is bringing to the digital music industry. In doing so, the L.A.-based company thinks it can help music services and artists make more money, and give listeners a new and improved way of experiencing audio.

Super Hi-Fi's customers include iHeartMedia,

Founded in 2018 by a group of digital music business veterans, with customers that include iHeartMedia and Sonos, SHF recently announced that Tristan Jehan – a pioneer of applying AI and machine learning to music, most recently as head of Spotify's research team – has joined the company as an advisor.

Streaming, which has been powered in part by Jehan's work on recommendation algorithms, may have saved the music industry from piracy-fueled devastation, but Super Hi-Fi's founders believe companies like Spotify, Apple and Amazon have missed the mark in some important ways.

"We think digital services forgot that the primary user interface is your eardrums, not your eyeballs," Zack Zalon, Super Hi-Fi's chief executive, told dot.LA.

As a result, a listener cannot readily distinguish one service from another, he said. Visual branding can help, but especially as smart speakers grow more popular, listeners will have an increasingly difficult time knowing whether they are tuned in to Spotify, Apple Music, Pandora or any other service.

Zalon and Brendon Cassidy, another co-founder and Super Hi-Fi's chief technology officer, both built digital radio services for CBS, AOL, Yahoo and Cricket Wireless. They contrast the uniformity of streaming services to broadcast radio stations. Two stations in the same city, focusing on the same type of music, still tend to have their own personality, they say.

"It's not just the music," Zalon said. "It's the space between the songs that the radio stations are filling and (thereby) connecting with listeners in an emotional way."

Beyond differentiating the source of the sound, SHF believes that using the space between songs — as radio stations do — can transform the listening experience. Leaving aside stitching in voice clips and layering in all other kinds of audio, they say that even sequencing songs together is fraught with issues.

The two most common types of transitions that digital services use between songs are crossfading and brief pauses.

Pauses create dead air, which iHeartMedia's chief product officer Chris Williams told dot.LA is the "enemy of radio," because it takes listeners out of their flow and puts them in a position to make a decision: namely, whether to keep listening or not. "I want to minimize that," Williams said.

As for crossfading – which Zalon said tends to be applied uniformly across tracks, such as a six-second blend of the end of one song with the start of the next, irrespective of the songs' particulars – it doesn't always work. Williams said crossfading can be done without jarring the user or disrespecting the songs about 50% of the time, and ever less so as longer song lead-ins give way to snappy intros aimed at grabbing listeners' attention from the first note.

"Crossfades create crashes," he said.

A radio DJ, by contrast, is trained to avoid these issues.

"The techniques of trained radio DJs are pretty complicated," Zalon said. "Super Hi-Fi is AI that has all the skills of a trained radio DJ."

This AI is designed to help customers like iHeart provide listeners with custom playlists that artfully blend one song with the next – no matter which song it is – and incorporate additional sonic elements like branded logos, interview clips, voice segments and more to eliminate dead air while respecting the artistry of the underlying songs, and to potentially drive streaming revenues back up to or beyond the levels that CDs generated in their peak.

How Does It Work?

As Super Hi-Fi's first customer, iHeartMedia helped the company fine tune its AI model and algorithms.

"Their pool of some of the world's best music programmers and on-air DJs became in a way like our QA partners," Zalon said.

By now Super Hi-Fi has "fingerprinted" iHeart's entire music catalog – tens of millions of tracks, Williams said – to identify the songs' underlying characteristics: features like volume, rhythm, mood, vocal texture, tempo and more. Lots more.

"The data we have on the music takes up more file space than the music itself," Zalon said.

Super Hi-Fi's AI has also fingerprinted each of iHeart's 700 styles of sonic logo; there's a country version of the "You are listening to iHeart…", a hip-hop version, a reggae version, and so on.

So, too, with any audio file that any customer may want to include in a playlist.

"The real power of the AI is to understand all the content with enough depth to make pretty much any output decision that any service might want," Zalon said.

Results have been good for iHeart. Williams reported that since they've rolled out Super Hi-Fi across their streaming services, the company has seen listening durations double.

"That increases ad impressions," he said. "There are clear business upside reasons beyond aesthetically creating a better experience...and [achieving] clear differentiation [from competitors]."

The Next Song

After having recently reeled in Sonos and its new streaming service as a customer, Super Hi-Fi has its sights set on bigger streaming fish. Jehan may help them get to Spotify, and they'll also look to Apple Music.

Chief business officer John Bolton told dot.LA that Spotify's expansion into podcasts makes them a prime candidate to benefit from Super Hi-Fi's service. He pointed to their "Your Daily Drive" product – a mix of music they know listeners like and relevant news – and said he could see that sort of playlist growing to include podcast snippets, the weather, and various other forms of audio, all tailored to the listener's preferences and location.

"If we see this happen at scale you'll start to see listening experiences become something very different," said Bolton, who helped found Super Hi-Fi after leaving ByteDance, which acquired his social media-meets-music startup Flipagram.

Streaming platforms may well be working on their own blended-audio AI solutions, but Williams says iHeart has benefited by getting in with Super Hi-Fi early.

"Two years from now everyone should have this solution or some version but we'll have already taken this to the next step, so it gives me the opportunity to get ahead," he said.

Music lawyer Ed Buggé, partner at L.A.-based entertainment law firm Hertz Lichtenstein & Young, thinks Super Hi-Fi "could be an attractive target for digital streaming platforms (DSPs) as they look to differentiate their platform offerings."

Spotify's former chief economist agreed. Will Page said a major audio streamer could look to Super Hi-Fi as a way to stand out in a market of look-alike services competing for the same customers.

"All these services offer the same 60 million tracks for the same $9.99 price point. With so much similarity, the value of distinctiveness goes up," he said. "The end game could see it being acquired by a DSP."

More than Music

Even if Super Hi-Fi doesn't lure the big catches in streaming, it can serve other verticals, including digital fitness. Peloton is already a customer.

"They wanted something different," Zalon said. "They want to make sure there's no loss of energy between songs; there's nothing worse than having music drop off in the middle of a workout. So we've been working with them on the roadmap to use the AI to help create an exercise experience that never loses its energy." In addition to blending songs, this will include voice-over instructions from Peloton's instructors.

Retail music is another area the company's targeting. Octave Music Groups, which operates Apple Music for Business and manages the music that customers hear at Starbucks, along with many other bars, restaurants and stores throughout North America and Europe, is another Super Hi-Fi customer.

Just as a radio station manager works with a DJ to set ground rules for branding the station through its sonic choices – running a branded message every 15 minutes, a news break every 30, and keeping segues tight but breezy, for example – each customer works with Super Hi-Fi to customize the AI to fit its desired style.

In exchange for this AI-plus-support service, Super Hi-Fi receives a licensing fee. The company would not disclose its prices or finances, though Bolton said Super Hi-Fi has not raised any outside funding.

He thinks using AI to bring a new level of real-time curation, presentation and personalization can attract more advertising and subscription revenues to the music industry. That's all the more important in an era when live performance has evaporated.

    "We have a big, bold vision and tech that can transform playlists into listening experiences that we think will eventually make the industry much more money and provide a much better product for consumers," he said.

    ---

    Sam Blake primarily writes about media and entertainment for dot.LA. Find him on Twitter at @hisamblake and email him at samblake@dot.LA

    https://twitter.com/hisamblake
    samblake@dot.la

    Subscribe to our newsletter to catch every headline.

    Creandum’s Carl Fritjofsson on the Differences Between the Startup Ecosystem in Europe and the U.S.

    Decerry Donato

    Decerry Donato is a reporter at dot.LA. Prior to that, she was an editorial fellow at the company. Decerry received her bachelor's degree in literary journalism from the University of California, Irvine. She continues to write stories to inform the community about issues or events that take place in the L.A. area. On the weekends, she can be found hiking in the Angeles National forest or sifting through racks at your local thrift store.

    Carl Fritjofsson
    Carl Fritjofsson

    On this episode of the LA Venture podcast, Creandum General Partner Carl Fritjofsson talks about his venture journey, why Generative-AI represents an opportunity to rethink products from the ground up, and why Q4 2023 and Q1 2024 could be "pretty bloody" for startups.

    Read moreShow less

    AI Is Undergoing Some Growing Pains at a Pivotal Moment in Its Development

    Lon Harris
    Lon Harris is a contributor to dot.LA. His work has also appeared on ScreenJunkies, RottenTomatoes and Inside Streaming.
    AI Is Undergoing Some Growing Pains at a Pivotal Moment in Its Development
    Evan Xie

    One way to measure just how white-hot AI development has become: the world is running out of the advanced graphics chips necessary to power AI programs. While Intel central processing units were once the most sought-after industry leaders, advanced graphics chips like Nvidia’s are designed to run multiple computations simultaneously, a baseline necessity for many AI models.

    An early version of ChatGPT required around 10,000 graphics chips to run. By some estimates, newer updates require 3-5 times that amount of processing power. As a result of this skyrocketing demand, shares of Nvidia have jumped 165% so far this year.

    Building on this momentum, this week, Nvidia revealed a line-up of new AI-related projects including an Israeli supercomputer project and a platform utilizing AI to help video game developers. For smaller companies and startups, however, getting access to the vital underlying technology that powers AI development is already becoming less about meritocracy and more about “who you know.” According to the Wall Street Journal, Elon Musk scooped up a valuable share of server space from Oracle this year before anyone else got a crack at it for his new OpenAI rival, X.AI.

    The massive demand for Nvidia-style chips has also created a lucrative secondary market, where smaller companies and startups are often outbid by larger and more established rivals. One startup founder compares the fevered crush of the current chip marketplace to toilet paper in the early days of the pandemic. For those companies that don’t get access to the most powerful chips or enough server space in the cloud, often the only remaining option is to simplify their AI models, so they can run more efficiently.

    Beyond just the design of new AI products, we’re also at a key moment for users and consumers, who are still figuring out what sorts of applications are ideal for AI and which ones are less effective, or potentially even unethical or dangerous. There’s now mounting evidence that the hype around some of these AI tools is reaching a lot further than the warnings about its drawbacks.

    JP Morgan Chase is training a new AI chatbot to help customers choose financial securities and stocks, known as IndexGPT. For now, they insist that it’s purely supplemental, designed to advise and not replace money managers, but it may just be a matter of time before job losses begin to hit financial planners along with everyone else.

    A lawyer in New York just this week was busted by a judge for using ChatGPT as part of his background research. When questioned by the judge, lawyer Peter LoDuco revealed that he’d farmed out some research to a colleague, Steven A. Schwartz, who had consulted with ChatGPT on the case. Schwartz was apparently unaware that the AI chatbot was able to lie – transcripts even show him questioning ChatGPT’s responses and the bot assuring him that these were, in fact, real cases and citations.

    New research by Marucie Jakesch, a doctoral student from Cornell University, suggests that even users who are more aware than Schwartz about how AI works and its limitations may still be impacted in subtle and subconscious ways by its output.

    Not to mention, according to data from Intelligent.com, high school and college students already – on the whole – prefer utilizing ChatGPT for help with schoolwork over a human tutor. The survey also notes that advanced students tend to report getting more out of using ChatGPT-type programs than beginners, likely because they have more baseline knowledge and can construct better and more informative prompts.

    But therein lies the big drawback to using ChatGPT and other AI tools for education. At least so far, they’re reliant on the end user writing good prompts and having some sense about how to organize a lesson plan for themselves. Human tutors, on the other hand, have a lot of personal experience in these kinds of areas. Someone who instructs others in foreign languages professionally probably has a good inherent sense of when you need to focus on expanding your vocabulary vs. drilling certain kinds of verb and tense conjugations. They’ve helped many other students prepare for tests, quizzes, and real-world challenges, while computer software can only guess at what kinds of scenarios its proteges will face.

    A recent Forbes editorial by academic Thomas Davenport suggests that, while AI is getting all the hype right now, other forms of computing or machine learning are still going to be more effective for a lot of basic tasks. From a marketing perspective in 2023, it’s helpful for a tech company to throw the “AI” brand around, but it’s not magically going to be the answer for every problem.

    Davenport points to a similar (if smaller) whirlwind of excitement around IBM’s “Watson” in the early 2010s, when it was famously able to take out human “Jeopardy!’ champions. It turns out, Watson was a general knowledge engine, really best suited for jobs like playing “Jeopardy.” But after the software gained celebrity status, people tried to use it for all sorts of advanced applications, like designing cancer drugs or providing investment advice. Today, few people turn to Watson for these kinds of solutions. It’s just the wrong tool for the job. In that same way, Davenport suggests that generative AI is in danger of being misapplied.

    While the industry and end users both race to solve the AI puzzle in real time, governments are also feeling pressure to step in and potentially regulate the AI industry. This is much easier said than done, though, as politicians face the same kinds of questions and uncertainty as everyone else.

    OpenAI CEO Sam Altman has been calling for governments to begin regulating AI, but just this week, he suggested that the company might pull out of the European Union entirely if the regulations were too onerous. Specifically, Altman worries that attempts to narrow what kinds of data can be used to train AI systems – specifically blocking copyrighted material – might well prove impossible. “If we can comply, we will, and if we can’t, we’ll cease operating,” Altman told Time. “We will try, but there are technical limits to what’s possible.” (Altman has already started walking this threat back, suggesting he has no immediate plans to exit the EU.)

    In the US, The White House has been working on a “Blueprint for an AI Bill of Rights,” but it’s non-binding, just a collection of largely vague suggestions. It’s one thing to agree “consumers shouldn’t face discrimination from an algorithm” and “everyone should be protected from abusive data practices and have agency over how their data is used.” But enforcement is an entirely different animal. A lot of these issues already exist in tech, and are much larger than AI, and the US government already doesn’t do much about them.

    Additionally, it’s possible AI regulations won’t work well at all if they aren’t global. Even if you set some policies and get an entire nation’s government to agree, how to set similar worldwide protocols? What if US and Europe agree but India doesn’t? Everyone around the world accesses roughly the same internet, so without any kind of international standard, it’s going to be much harder for individual nations to enforce specific rules. As with so many other AI developments, there’s inherent danger in patchwork regulations; it could allow some companies, or regions, or players to move forward while others are unfairly or ineffectively stymied or held back.

    The same kinds of socio-economic concerns around AI that we have nationally – some sectors of the work force left behind, the wealthiest and most established players coming in to the new market with massive advantages, the rapid spread of misinformation – are all, in actuality, global concerns. Just as the hegemony of Microsoft and Google threaten the ability of new players to enter the AI space, the West’s early dominance of AI tech threatens to push out companies and innovations from emerging markets like Southeast Asia, Subsaharan Africa, and Central America. Left unfettered, AI could potentially deepen social, economic, and digital divisions both within and between all of these societies.

    Undaunted, some governments aren’t waiting around for these tools to develop any further before they start attempting to regulate them. New York City has already set up some rules about how AI can be used during the hiring process while will take effect in July. The law requires any company using AI software in hiring to notify candidates that it’s being used, and to have independent auditors check the system annually for bias.

    This sort of piecemeal figure-it-out-as-we-go approach is probably what’s going to be necessary, at least short-term, as AI development shows zero signs of slowing down or stopping any time soon. Though there’s some disagreement among experts, most analysts agree with Wharton professor and economist Jeremy Siegel, who told CNBC this week that AI is not yet a bubble. He pointed to the Nvidia earnings as a sign the market remains healthy and not overly frothy. So, at least for now, the feverish excitement around AI is not going to burst like a late ‘90s startup stock. The world needs to prepare as if this technology is going to be with us for a while.

    Rivian CEO Teases R2, New Features in Instagram AMA

    David Shultz

    David Shultz reports on clean technology and electric vehicles, among other industries, for dot.LA. His writing has appeared in The Atlantic, Outside, Nautilus and many other publications.

    Rivian CEO Teases R2, New Features in Instagram AMA
    Rivian

    Rivian CEO RJ Scaringe took to Instagram last weekend to answer questions from the public about his company and its future. Topics covered included new colors, sustainability, production ramp, new products and features. Speaking of which, viewers also got a first look at the company’s much-anticipated R2 platform, albeit made of clay and covered by a sheet, but hey, that’s…something. If you don’t want to watch the whole 33 minute video, which is now also on Youtube, we’ve got the highlights for you.

    Read moreShow less
    RELATEDEDITOR'S PICKS
    LA TECH JOBS
    interchangeLA
    Trending