AI Is Undergoing Some Growing Pains at a Pivotal Moment in Its Development

Lon Harris
Lon Harris is a contributor to dot.LA. His work has also appeared on ScreenJunkies, RottenTomatoes and Inside Streaming.
AI Is Undergoing Some Growing Pains at a Pivotal Moment in Its Development
Evan Xie

One way to measure just how white-hot AI development has become: the world is running out of the advanced graphics chips necessary to power AI programs. While Intel central processing units were once the most sought-after industry leaders, advanced graphics chips like Nvidia’s are designed to run multiple computations simultaneously, a baseline necessity for many AI models.

An early version of ChatGPT required around 10,000 graphics chips to run. By some estimates, newer updates require 3-5 times that amount of processing power. As a result of this skyrocketing demand, shares of Nvidia have jumped 165% so far this year.

Building on this momentum, this week, Nvidia revealed a line-up of new AI-related projects including an Israeli supercomputer project and a platform utilizing AI to help video game developers. For smaller companies and startups, however, getting access to the vital underlying technology that powers AI development is already becoming less about meritocracy and more about “who you know.” According to the Wall Street Journal, Elon Musk scooped up a valuable share of server space from Oracle this year before anyone else got a crack at it for his new OpenAI rival, X.AI.

The massive demand for Nvidia-style chips has also created a lucrative secondary market, where smaller companies and startups are often outbid by larger and more established rivals. One startup founder compares the fevered crush of the current chip marketplace to toilet paper in the early days of the pandemic. For those companies that don’t get access to the most powerful chips or enough server space in the cloud, often the only remaining option is to simplify their AI models, so they can run more efficiently.

Beyond just the design of new AI products, we’re also at a key moment for users and consumers, who are still figuring out what sorts of applications are ideal for AI and which ones are less effective, or potentially even unethical or dangerous. There’s now mounting evidence that the hype around some of these AI tools is reaching a lot further than the warnings about its drawbacks.

JP Morgan Chase is training a new AI chatbot to help customers choose financial securities and stocks, known as IndexGPT. For now, they insist that it’s purely supplemental, designed to advise and not replace money managers, but it may just be a matter of time before job losses begin to hit financial planners along with everyone else.

A lawyer in New York just this week was busted by a judge for using ChatGPT as part of his background research. When questioned by the judge, lawyer Peter LoDuco revealed that he’d farmed out some research to a colleague, Steven A. Schwartz, who had consulted with ChatGPT on the case. Schwartz was apparently unaware that the AI chatbot was able to lie – transcripts even show him questioning ChatGPT’s responses and the bot assuring him that these were, in fact, real cases and citations.

New research by Marucie Jakesch, a doctoral student from Cornell University, suggests that even users who are more aware than Schwartz about how AI works and its limitations may still be impacted in subtle and subconscious ways by its output.

Not to mention, according to data from Intelligent.com, high school and college students already – on the whole – prefer utilizing ChatGPT for help with schoolwork over a human tutor. The survey also notes that advanced students tend to report getting more out of using ChatGPT-type programs than beginners, likely because they have more baseline knowledge and can construct better and more informative prompts.

But therein lies the big drawback to using ChatGPT and other AI tools for education. At least so far, they’re reliant on the end user writing good prompts and having some sense about how to organize a lesson plan for themselves. Human tutors, on the other hand, have a lot of personal experience in these kinds of areas. Someone who instructs others in foreign languages professionally probably has a good inherent sense of when you need to focus on expanding your vocabulary vs. drilling certain kinds of verb and tense conjugations. They’ve helped many other students prepare for tests, quizzes, and real-world challenges, while computer software can only guess at what kinds of scenarios its proteges will face.

A recent Forbes editorial by academic Thomas Davenport suggests that, while AI is getting all the hype right now, other forms of computing or machine learning are still going to be more effective for a lot of basic tasks. From a marketing perspective in 2023, it’s helpful for a tech company to throw the “AI” brand around, but it’s not magically going to be the answer for every problem.

Davenport points to a similar (if smaller) whirlwind of excitement around IBM’s “Watson” in the early 2010s, when it was famously able to take out human “Jeopardy!’ champions. It turns out, Watson was a general knowledge engine, really best suited for jobs like playing “Jeopardy.” But after the software gained celebrity status, people tried to use it for all sorts of advanced applications, like designing cancer drugs or providing investment advice. Today, few people turn to Watson for these kinds of solutions. It’s just the wrong tool for the job. In that same way, Davenport suggests that generative AI is in danger of being misapplied.

While the industry and end users both race to solve the AI puzzle in real time, governments are also feeling pressure to step in and potentially regulate the AI industry. This is much easier said than done, though, as politicians face the same kinds of questions and uncertainty as everyone else.

OpenAI CEO Sam Altman has been calling for governments to begin regulating AI, but just this week, he suggested that the company might pull out of the European Union entirely if the regulations were too onerous. Specifically, Altman worries that attempts to narrow what kinds of data can be used to train AI systems – specifically blocking copyrighted material – might well prove impossible. “If we can comply, we will, and if we can’t, we’ll cease operating,” Altman told Time. “We will try, but there are technical limits to what’s possible.” (Altman has already started walking this threat back, suggesting he has no immediate plans to exit the EU.)

In the US, The White House has been working on a “Blueprint for an AI Bill of Rights,” but it’s non-binding, just a collection of largely vague suggestions. It’s one thing to agree “consumers shouldn’t face discrimination from an algorithm” and “everyone should be protected from abusive data practices and have agency over how their data is used.” But enforcement is an entirely different animal. A lot of these issues already exist in tech, and are much larger than AI, and the US government already doesn’t do much about them.

Additionally, it’s possible AI regulations won’t work well at all if they aren’t global. Even if you set some policies and get an entire nation’s government to agree, how to set similar worldwide protocols? What if US and Europe agree but India doesn’t? Everyone around the world accesses roughly the same internet, so without any kind of international standard, it’s going to be much harder for individual nations to enforce specific rules. As with so many other AI developments, there’s inherent danger in patchwork regulations; it could allow some companies, or regions, or players to move forward while others are unfairly or ineffectively stymied or held back.

The same kinds of socio-economic concerns around AI that we have nationally – some sectors of the work force left behind, the wealthiest and most established players coming in to the new market with massive advantages, the rapid spread of misinformation – are all, in actuality, global concerns. Just as the hegemony of Microsoft and Google threaten the ability of new players to enter the AI space, the West’s early dominance of AI tech threatens to push out companies and innovations from emerging markets like Southeast Asia, Subsaharan Africa, and Central America. Left unfettered, AI could potentially deepen social, economic, and digital divisions both within and between all of these societies.

Undaunted, some governments aren’t waiting around for these tools to develop any further before they start attempting to regulate them. New York City has already set up some rules about how AI can be used during the hiring process while will take effect in July. The law requires any company using AI software in hiring to notify candidates that it’s being used, and to have independent auditors check the system annually for bias.

This sort of piecemeal figure-it-out-as-we-go approach is probably what’s going to be necessary, at least short-term, as AI development shows zero signs of slowing down or stopping any time soon. Though there’s some disagreement among experts, most analysts agree with Wharton professor and economist Jeremy Siegel, who told CNBC this week that AI is not yet a bubble. He pointed to the Nvidia earnings as a sign the market remains healthy and not overly frothy. So, at least for now, the feverish excitement around AI is not going to burst like a late ‘90s startup stock. The world needs to prepare as if this technology is going to be with us for a while.

Who Gave the AI Access?

🔦 Spotlight

Happy Friday, Los Angeles.

The newest employee in the office does not need a desk, a salary or a coffee order. But it may have access to your company’s emails, financial records, customer data and software systems.

That raises a fairly important question: Who decides what an AI agent is allowed to do?

El Segundo-based cybersecurity company Saviynt is building its next chapter around the answer.

Carrick Capital Partners announced this week that it closed a $600M continuation vehicle for Saviynt, including a new $255M investment in the company. The transaction allowed Carrick’s existing investors to either take liquidity or remain invested, while also providing liquidity to Saviynt employees through a tender offer.

Continuation vehicles are not exactly known for making gripping Friday reading. The company behind this one is considerably more interesting.

Saviynt develops identity security software that helps businesses determine who can access their applications, data and infrastructure. Increasingly, however, “who” does not refer exclusively to a person.

Companies are deploying AI agents that can retrieve information, write code, communicate with customers and complete multistep tasks with limited supervision. These digital workers need access to company systems to be useful, but every new permission also creates another opportunity for sensitive information to be exposed or an unintended action to be taken.

In other words, AI agents may be tireless employees. They are not necessarily trustworthy ones.

Saviynt is addressing that problem through Zuma, its platform for discovering, securing and governing AI agents, large language models and other nonhuman identities alongside a company’s human workforce. The goal is to give businesses one place to determine what every identity can access, whether it belongs to an employee, a contractor, a software application or an autonomous agent operating at machine speed.

The opportunity appears to be growing quickly. Saviynt has surpassed $300M in annual recurring revenue, up from approximately $10M when Carrick first invested. The company says bookings have increased by more than 80% this year while customer retention remains at 96%.

The new investment was completed as part of the final close of Saviynt’s previously announced $700M Series B, which valued the company at approximately $3B. Carrick’s continuation vehicle was led by Coller Capital and co-led by HSBC Asset Management, giving the investment firm more time and capital to remain behind one of its strongest-performing companies.

For Saviynt, the funding will support further development of its identity platform, deeper integrations with major cloud and software providers and its push to become a central security layer for the agentic workplace.

That ambition reflects a broader change taking place inside companies. The first wave of enterprise AI focused largely on what the technology could generate. The next phase is about what it can actually do, and whether businesses can maintain control once AI moves from answering questions to taking action.

Saviynt is betting that identity will become the gatekeeper.

AI agents are gaining access to the digital workplace, whether corporate security teams are ready for them or not.

Someone still has to hold the keys.

More from this week’s LA startup and venture scene below.

🤝 Venture Deals

    LA Venture Funds

    • Fulcrum Ventures participated in Critical Materials Group’s $10.3M seed round, led by Overmatch Ventures and joined by Victory Six Advisors. The Austin-based defense manufacturer will use the funding to develop and commission modular, automation-ready production systems designed to expand domestic manufacturing capacity for munitions and advanced energetic materials. - learn more
    • Fusion VC participated in Newlight’s $9M seed round alongside lomarlabs, BIRD Energy, Undeterred Capital and CiRi Ventures. The San Francisco-based maritime technology company recently demonstrated its hydrogen-hybrid retrofit on an 8,500-nautical-mile commercial voyage, reducing fuel consumption by 24% and carbon dioxide emissions by 28%. - learn more
    • Rebel Fund participated in Metal’s $4.5M seed round alongside a16z, Y Combinator, Gaingels, Indus Valley Capital, Phaze Ventures and Pioneer Fund. Metal will use the funding to build an AI-native operating system that helps founders identify relevant investors, manage outreach and automate other parts of the venture fundraising process. - learn more
    • UP.Partners participated in Reframe Systems’ $40M funding round, led by Energy Impact Partners and joined by Counterpart Ventures, E12 Ventures, Global Brain, Thin Line Capital and LACI Impact Fund. The homebuilding startup will use the capital to expand its network of robotics-powered microfactories, which it says can construct homes three times faster and at 35% lower cost than traditional methods. - learn more
    • Clocktower Technology Ventures participated in Sharpi’s $4M seed round, co-led by NXTP and ONEVC and joined by MAYA Capital. The Brazilian startup will use the funding to expand its team and develop autonomous AI agents that connect WhatsApp conversations with enterprise systems to automate B2B sales tasks such as order processing, customer follow-ups and demand generation. - learn more

    LA Exits

    • Extensiv, a California-based provider of warehouse management and fulfillment software, has been acquired by Descartes Systems Group for approximately $120M in cash. The acquisition adds Extensiv’s AI-enabled inventory, order, billing and omnichannel fulfillment tools to Descartes’ logistics network, strengthening its offerings for third-party logistics providers and ecommerce brands. - learn more
    • DocSolutionUSA has been acquired by Stewart Information Services alongside ProTitleUSA, adding mortgage document generation and automation capabilities to Stewart’s title services platform. The companies provide title, document and due diligence services for mortgage servicers, investors and capital markets clients; financial terms were not disclosed. - learn more
    • Fysh Foods, the Los Angeles-based plant-based seafood brand founded by creator and entrepreneur Zoya Biglary, has been acquired by City Roots Hospitality in an all-cash deal with undisclosed terms. City Roots plans to introduce Fysh Foods’ raw fish alternatives across its New York City restaurants and potentially expand the brand beyond the city as its restaurant portfolio grows - learn more

      Download the dot.LA App

      An LA AI Company Just Won Entertainment’s Backing

      🔦 Spotlight

      Hello LA.

      The entertainment industry has spent the past several years debating what generative AI could take from creators.

      This week, some of its biggest companies put money behind an AI startup promising to build something for them instead.

      Los Angeles-based Stability AI raised $76M in Series B funding from an investor group that includes Electronic Arts, Sony Music Group, Universal Music Group and Warner Music Group. AMD Ventures and Pacific Alliance Ventures also joined the round, while LA-based MANTIS Capital and Sound Ventures are among the company’s existing backers.

      The financing brings Stability AI’s total funding under CEO Prem Akkaraju to $232M, including two equity rounds and convertible notes. The company plans to use the new capital to expand its creative production tools, applied research and professional services across music, gaming and entertainment.

      The amount is notable. The names attached to it are the bigger story.

      Generative AI’s arrival in entertainment has been anything but quiet. Artists have questioned whether their work was used to train models without permission. Studios have faced pressure over how the technology could affect jobs. Record labels have pursued AI companies in court while simultaneously exploring how the same technology might fit into their businesses.

      Now, several of the world’s largest entertainment companies are investing directly in one.

      That does not mean the industry has resolved its concerns about AI. It means some of its biggest players would rather help shape the technology than wait to see what it becomes.

      Stability AI is positioning itself for that opening. Rather than focusing solely on general-purpose models, the company is building tools specifically for professional creatives. Its recently launched Stable Audio 3.0 was trained on fully licensed music and lets artists generate, edit and arrange audio through a web platform or directly inside digital audio workstations.

      Image Source: Stability AI

      That licensed-data approach is central to the pitch. The next phase of creative AI will not be decided only by which company produces the most impressive model. It will also depend on which companies can earn the trust of the artists, studios and rights holders whose work gives those models value.

      For its new strategic investors, the round offers more than financial upside. It creates a closer view into how generative AI may change production, a voice in how the tools develop and an opportunity to establish rules before those rules are established for them.

      For Stability AI, the backing provides something equally important: credibility inside industries that have every reason to scrutinize what it is building.

      The company now has capital and access to some of the largest catalogs, franchises and creative workforces in entertainment. What it does with that access will determine whether this becomes a meaningful alliance or simply an impressive collection of logos.

      Either way, the industry is no longer watching from a safe distance.

      It has entered the room.

      LA’s Air-Taxi Plans Are Coming Downtown

      While Stability AI is trying to change how entertainment gets made, Archer Aviation wants to change how people get to it.

      AEG and Archer announced plans to develop downtown Los Angeles’ first vertiport at L.A. LIVE, creating a potential new stop in Archer’s proposed electric air-taxi network ahead of the 2028 Olympic and Paralympic Games.

      Image Source: Archer

      The planned site would sit beside Crypto.com Arena and allow passengers to travel to and from the entertainment district aboard Archer’s Midnight aircraft. The company says its network could turn drives that take an hour or longer into electric flights lasting approximately 10 to 20 minutes.

      Archer has already identified SoFi Stadium, USC and Hollywood Burbank Airport as possible locations, with its recently acquired Hawthorne Airport expected to serve as the network’s central operating hub. As the official air-taxi provider of LA28 and Team USA, Archer has an unusually visible deadline for turning those plans into something tangible.

      AEG and Archer have completed an initial feasibility study of the L.A. LIVE site, including reviews of land use, airspace, power availability and community impact. The next phase will examine operations and the passenger experience.

      There is still a substantial distance between a proposed vertiport and a functioning air-taxi network. The infrastructure must be built, regulatory approvals must be secured and passengers must be persuaded that flying across the city is safer and more practical than staying on the ground.

      Still, few locations could make that future feel more real than L.A. LIVE. Millions of people already pass through the district for concerts, games and major events. Placing a vertiport there would bring urban air mobility out of the concept stage and directly into public view.

      Together, this week’s announcements show Los Angeles becoming a testing ground for two technologies still moving from promise toward everyday use.

      One could reshape how entertainment is created. The other could reshape how Angelenos reach it.

      In a city famous for both its creative industries and its traffic, that feels appropriately on brand.

      More from this week’s LA startup and venture scene below.

      🤝 Venture Deals

        LA Companies

        • Atorie raised a $9.5M seed round from investors including a16z speedrun, Night Capital and Lightspeed Venture Partners’ Jeremy Liew. The AI-powered fashion startup connects consumers directly with luxury manufacturers to offer high-quality goods without traditional designer markups, and will use the funding to expand logistics, production and its AI shopping tools. - learn more
        • Long Beach-based Maglut Heavy Industries emerged from stealth with $3.1M in pre-seed funding from Wave Function, Nova Threshold and Julian Capital. The startup is developing a chromatography-based system to process and refine rare earth elements domestically, with pilot tests producing materials at more than 99.9% purity. - learn more

        LA Venture Funds
        • MANTIS Venture Capital participated in Voya Energy’s $35M Series A, led by Energy Impact Partners and joined by John Doerr, StepStone, Founders Fund, Overmatch and Seven Stars. The Hayward-based startup will use the funding to commercialize its aluminum-fueled generators, which provide clean, off-grid power for data centers and other energy-intensive operations without combustion or local air emissions. - learn more
        • Regeneration.VC participated in eComID’s $17M seed round, led by Systemiq Capital and joined by Course Corrected, Stadium and returning investor CapitalT. The Stockholm-based startup will use the funding to expand internationally and scale its AI-powered Shopping Passport, which helps retailers personalize sizing and product discovery while reducing returns. - learn more
        • Clocktower Technology Ventures participated in Helcim’s $53M Series C, led by BDC Capital’s Growth Venture Fund and joined by new investors Curql Collective and LA-based Gold House Ventures. The Calgary payments company will use the funding to expand its platform, develop additional financial services and serve more small and midsize businesses across North America. - learn more

        LA Exits

        • Altruist agreed to be acquired by Vanguard, giving the Los Angeles-based wealth technology and custody platform greater resources to expand its tools for independent financial advisors. Altruist will continue operating as a standalone business under its existing leadership and brand after the deal closes, which is expected later this year pending regulatory approval; financial terms were not disclosed. - learn more
        • Personality AI has been acquired by WildBrain for approximately $11M in cash and 1M WildBrain shares upfront, with additional payments tied to future performance. The startup develops kid-safe conversational AI experiences for entertainment characters, including “Hey Peppa Pig,” and will help WildBrain expand its franchises into interactive products across toys, apps and digital platforms. - learn more

          Download the dot.LA App

          This Torrance Startup Just Raised $1B to Mass-Produce Hypersonic Missiles

          🔦 Spotlight

          Happy Friday, Los Angeles.

          Castelion has spent the past four years trying to prove that hypersonic missiles do not need to take decades to develop or cost so much that the military can only afford a limited supply.

          Now comes the harder part: producing them at scale.

          The Torrance-based defense startup raised a $1B Series C at a $13B valuation. The financing includes $800M in equity and a $250M revolving credit facility, making it one of the largest recent raises for an LA defense technology company.

          JPMorganChase’s Strategic Investment Group, Andreessen Horowitz and Carlyle co-led the round. Lightspeed Venture Partners, Lavrock Ventures, Altimeter, General Catalyst, T. Rowe Price and LA-based Interlagos Capital also participated.

          Castelion will use the capital to ramp production of Blackbeard, its low-cost hypersonic strike missile, while developing a longer-range precision weapon and new defensive systems. Hundreds of millions of dollars will go toward expanding manufacturing at Project Ranger, the company’s 1,000-acre production campus in New Mexico.

          Image Source: Castelion

          Blackbeard was designed in California, will be built in New Mexico and is expected to enter service in 2027. Castelion says it has already secured more than $500M in U.S. military contracts over the past 18 months and moved the missile from a clean-sheet concept to an official program in fewer than four years.

          That timeline is central to Castelion’s pitch. Traditional defense programs are often associated with long development cycles, limited production runs and eye-watering costs. Castelion is applying the rapid testing and vertically integrated manufacturing approach popularized by commercial space companies to weapons production.

          But a $13B valuation changes the standard. Castelion is no longer being judged as a promising startup with an impressive prototype. It is being funded like a company expected to become a major part of the American defense industrial base.

          The question is no longer whether a startup can build a hypersonic missile. It is whether one can manufacture thousands of them without losing the speed, discipline and cost advantages that made it disruptive in the first place.

          For LA’s defense ecosystem, that shift matters. The region has become home to a growing number of companies promising to modernize how America builds critical hardware. Castelion now has the capital, contracts and facilities to show what happens when that promise reaches the factory floor.

          The next test will not be in a pitch deck. It will be in production.

          More from this week’s LA startup and venture scene below.

          🤝 Venture Deals

            LA Companies

            • Long Beach based Ampaire raised a $19M Series B led by DiamondStream Partners, with strategic participation from Alaska Star Ventures and IAGi Ventures, bringing its total funding to $68M. The hybrid-electric aviation company will use the capital to expand flight operations, produce additional Eco Caravan aircraft, advance regulatory certification and scale its manufacturing capabilities. - learn more

            LA Venture Funds
            • SUM Ventures participated in AssistMe’s €6.5M funding round, which was led by CRB Health Tech and Vorwerk Ventures and included several returning investors. The German care technology company will use the capital to expand across Europe, prepare for a U.S. launch and further develop alea, its digital platform for supporting caregivers and improving nursing-home operations. - learn more
            • CIV led Hypercubic’s $5.3M seed round, with participation from Y Combinator, Afore Capital, Pioneer Fund, Multimodal Ventures and several angel investors. The San Francisco startup will use the capital to develop AI agents that can analyze, document and rewrite decades-old COBOL systems, helping enterprises modernize critical mainframe software faster and with less risk. - learn more
            • Plus Capital participated in Wispr Flow’s $280M Series B, led by Menlo Ventures and joined by existing and new investors, valuing the AI voice company at $2B. The funding brings Wispr’s total capital raised to $361M and will support its expansion beyond dictation into meeting tools and proprietary speech technology, including its new Canto model. - learn more
            • Alexandria Venture Investments participated in Leal Therapeutics’ $30M Series A extension alongside new investor Eli Lilly and returning backers including OrbiMed, Newpath Partners and SV Health Investors’ Dementia Discovery Fund. The biotech company will use the funding to advance clinical trials of LTX-001 for schizophrenia and LTX-002 for ALS, with initial schizophrenia trial data expected by year-end. - learn more
            • BroadLight Capital participated in Higgsfield’s $400M Series B, led by DST Global and joined by investors including Goldman Sachs Alternatives, Smash Capital, Fifth Wall and Intel Capital. The AI video and image platform, now valued at $5.4B with $700M in annualized revenue, will use the funding for R&D, global infrastructure, AI hiring and international expansion. - learn more

              Download the dot.LA App

              RELATEDEDITOR'S PICKS
              Trending