|
|
|
|
|
|
|
|
|
|
The Morning Download: Putting AI Memory in Context
|
|
By Steven Rosenbush | WSJ Leadership Institute
|
|
|
|
|
|
|
|
|
|
Inside Micron's R&D fab in Boise; an automated wafer stocker robot. Todd Meier for WSJ
|
|
|
|
|
|
Good morning. Capital continues to pour into the development of memory chips crucial to AI. Companies are investing in the construction of more production capacity as well as the development of more advanced memory that will extend the capabilities of AI agents.
The WSJ’s Robbie Whelan reports from Boise, Idaho, where Micron is spending $50 billion to more than double the size of its 450-acre campus, including the construction of two new chip factories, or fabs.
Highlights from the story:
-
The first fab’s inaugural silicon wafers are expected to roll off the factory line in mid-2027, making DRAM, a type of memory used to make the high-bandwidth memory chips, or HBM, that are increasingly essential to advanced artificial-intelligence computing. Both plants should be in production by the end of 2028.
-
Micron just broke ground on a $100 billion fab complex that represents the state of New York’s largest-ever private investment. It also has announced a $9.6 billion fab investment in Hiroshima, Japan.
Higher-bandwidth memory allows AI agents to run for longer periods of time, which means they can address more complex problems.
As Micron and rivals such as Samsung and SK Hynix boost memory chip production capacity to satisfy unrelenting demand, there’s a related effort to develop new memory architectures. In January, Nvidia said it was creating a new class of AI-optimized memory that will complement existing tiers such as DRAM and shared memory.
By computing a "thought" once and reusing it, agents can perform complex, multi-step tasks within more reasonable timeframes and budgets, Nvidia senior vice president for networking Kevin Deierling said in a conversation with me last month.
If an AI was engaged with helping someone plan a trip to the Eiffel Tower, and the user asked for nearby restaurant recommendations, the model would need to recall the location of the tower. Stored context makes that retrieval faster and more efficient, Deierling said.
The amount of working memory, or context, that AI requires to generate a response has grown dramatically. When ChatGPT hit the scene a few years ago, the typical context size was whatever the user typed into the prompt window. To improve performance and accuracy, context has expanded to include much wider sources of information. As a result, the context window has grown from thousands of tokens to millions, according to Deierling.
As a result, there’s a need for ever more chips, as well as increasingly efficient forms of memory.
“Human beings are actually both painfully slow and impatient, at the same time. We're painfully slow, and computers make fun of us, behind our back,” Deierling said. “You ask a computer something, and then you go away. And we simply can't afford to leave the context of that query in the memory,” Deierling said.
|
|
|
|
|
Content from our sponsor: Deloitte
|
|
|
Principal CIO: ‘AI Isn’t a Shortcut—It’s a Mirror’
|
|
In AI transformation, new ways of working are where the real value lies, says Principal Executive Vice President and CIO Kathy Kay. Read More
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Gabby Jones/Bloomberg News
|
|
|
|
|
|
Silicon Valley's defense dilemma goes live. Anthropic's Claude AI was deployed in the U.S. military operation to capture former Venezuelan President Nicolás Maduro, but that involvement is now fueling a broader standoff between the AI startup and the Pentagon.
Claude's involvement came through Anthropic's partnership with Palantir Technologies, whose tools are used by the Defense Department and federal law enforcement, people familiar with the matter tell the Journal.
Anthropic's usage policies explicitly prohibit Claude from facilitating violence, weapon development, or surveillance.
Now contract renewal talks have stalled over guardrails Anthropic wants built into Claude's military use, Bloomberg reports. Those issues include prohibitions on mass surveillance of Americans and autonomous weapons deployment without human oversight,
This conflict has escalated tensions with the Trump administration. Axios reports Defense Secretary Pete Hegseth is now considering designating the company a "supply chain risk."
Meanwhile, rivals position themselves as more accommodating. SpaceX and xAI are competing in a Pentagon contest to develop voice-controlled autonomous drone swarms, Bloomberg reports. OpenAI is also contributing to the effort, but solely to the “mission control” element that will convert voice and other instructions from battlefield commanders into digital instructions, says Bloomberg.
|
|
|
|
|
|
India is hosting a global summit of Big Tech leaders and heads of state in New Delhi this week. Rajat Gupta/EPA/Shutterstock
|
|
|
|
|
|
India's AI bet: Do more with less. While U.S. tech giants look to deploy $670 billion in AI infrastructure, India is looking to become a significant AI player, but without breaking the bank.
At a global summit of Big Tech leaders and heads of state in New Delhi this week, India will promote its approach of developing cheaper AI tools aimed at solving local problems as a path for other middle-income nations.
|
|
|
|
|
“You don’t need large models...You need models that are smaller and customized for every setting”
|
|
— Arghya Bhattacharya, CTO of startup Adalat AI that offers AI tools for India's judicial systems
|
|
|
|
|
|
|
Infosys, one of India's IT giants, is betting on Anthropic to fight back against AI disruption fears. The company is embedding Claude models across telecom, financial services, manufacturing, and software development, WSJ reports. Both companies will develop custom AI agents tailored to specific industries and business functions, Infosys said.
|
|
|
OpenAI's new hire. The AI startup just poached Peter Steinberger, the founder of OpenClaw, the viral open-source agent platform that amassed 1.5 million agents in weeks by letting users run AI locally to manage emails, calendars, and apps. OpenAI CEO Sam Altman in a post on X Sunday called the Austrian developer, "a genius with a lot of amazing ideas." Steinberger will join OpenAI's Codex team, the FT reports.
|
|
|
|
|
|
|
|
|
|
Stored aircraft at an Air Force base near Tucson, Ariz. Ash Ponders for WSJ
|
|
|
|
|
|
Now powering AI data centers: Jet engines. Jet engine leasing and repair company FTAI Aviation plans to start selling a modified version of the engine used in the Boeing's 737 to power data centers this year, WSJ reports. Others pursuing this conversion include private equity-backed ProEnergy, which sells turbines adapted from the same engine that powers the Boeing 747. Siemens Energy and Mitsubishi Heavy Industries already sell power turbines modeled after jet
engines.
|
|
|
Public outcry over sexualized deepfake images continues. Ireland's Data Protection Commission is the latest entity to investigate X over Grok's ability to generate sexualized deepfake images of real people, the WSJ reports. The probe examines X's compliance with EU privacy rules.
|
|
|
|
|
|
Everything Else You Need to Know
|
|
|
The Rev. Jesse Jackson, a longtime civil-rights activist and Democratic political leader, has died. He was 84. (WSJ)
Goldman Sachs is preparing to remove race, gender identity, sexual orientation and other diversity factors from the criteria its board will consider when identifying potential candidates, according to people familiar with the matter. (WSJ)
Hotel magnate Thomas Pritzker is retiring as executive chairman of Hyatt Hotels, saying he wanted to protect the company after new documents were released detailing the extent of his association with Jeffrey Epstein and Ghislaine Maxwell. (WSJ)
Record CEO turnover at U.S. public companies has put the biggest class of incoming chief executives in years at the helm of massive enterprises—and the newcomers are younger and less experienced than before. (WSJ)
|
|
|
|
The WSJ Technology Council
|
|
|
The WSJ Tech Council brings together CIOs, CTOs and CISOs advancing innovation and shaping the future. Join this trusted community where tech executives connect with peers to explore emerging trends and gain the perspective they need to stay ahead of disruption.
Request Information
|
|
|
|
|
|
|
|