Strategy in the Face of Chaos
Audio companions to my writing on strategy, technology, AI, cybersecurity and building technology businesses.
Each edition explores one of my published articles through an AI-generated discussion or debate, offering another way to engage with its central ideas. These are not interviews or original podcast episodes, and the voices are not mine. The written article remains the definitive version.
This channel is currently a pilot, and the format will evolve as I learn what works.
Strategy in the Face of Chaos
Untether AI’s speedAI 240 Slim: Advancing Energy-Efficient AI Solutions for Edge Applications
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
This pilot audio edition explores the argument behind Untether AI’s speedAI 240 Slim: Advancing Energy-Efficient AI Solutions for Edge Applications.
The discussion examines one of the central constraints in AI inference: the energy and latency involved in moving data between memory and processing units. Untether AI addresses this through its At-Memory Compute architecture, placing computation close to the data to improve throughput, reduce latency and lower power consumption.
It considers why these characteristics matter for edge applications such as autonomous vehicles, agriculture and machine vision. These environments require high-performance inference within strict limits on power, space and response time. The discussion also explores Untether AI’s benchmark performance and its work with Mercedes-Benz and Arm.
The episode asks whether Untether AI could become a disruptive force in AI hardware. Its focus on underserved, power-constrained applications gives it a differentiated position. However, NVIDIA and AMD retain significant advantages through scale, established software ecosystems and integration across wider AI infrastructure.
This is an AI-generated discussion based on my published article. It offers another way to engage with the ideas, but it is not an interview or a recording of me. The written article remains the definitive version.
Imagine you're in a vehicle, right, driving down a highway of maybe 70 miles an hour, and suddenly a shadow falls across the road ahead.
SPEAKER_01Right. And in a fraction of a second, the car's onboard AI has to make this massive life or death calculation. Like, is that a pedestrian stepping out or just a cloud passing over the sun?
SPEAKER_00Exactly. To make that decision, the digital brain in that car has to ingest, process, and output millions of pieces of data in literal milliseconds.
SPEAKER_01It's an incredible amount of work.
SPEAKER_00It is. And if you are a technology or business leader trying to navigate the real-world application of AI, well, that fraction of a second is where the entire industry is currently bottlenecked.
SPEAKER_01Yeah, that's the real hurdle right now.
SPEAKER_00So today's mission is to deliver a rigorous executive audio briefing tailored specifically for you. And we're basing this entirely on a recent highly detailed analysis by author Victor Holman.
SPEAKER_01Right. No hype today, just the facts.
SPEAKER_00Exactly. We are doing a deep dive into Untether AI newly shipping Speed AI 240 Slim Inference Accelerator cards. And we're going to look at why scaling AI is, you know, no longer just about building bigger data centers.
SPEAKER_01Because the real battleground has fundamentally shifted. I mean, for the last few years, the focus has entirely been on training these massive models in giant warehouse-sized facilities.
SPEAKER_00Where power and space are basically unlimited.
SPEAKER_01Exactly. But the new frontier like where this technology actually becomes commercially viable across different industries is deploying those trained models efficiently at the edge. Right. Running an AI model inside a moving vehicle or like a remote agricultural drone or a factory floor sensor, that is an entirely different engineering physics problem than running it in a server farm.
SPEAKER_00Okay, let's unpack this. Because to really understand why Untether AI's new hardware is commanding so much attention in Hallman's analysis, we have to look at the foundational flaw in modern chip architecture.
SPEAKER_01The data movement bottleneck.
SPEAKER_00Exactly. The physical limitation.
SPEAKER_01Yeah. And it's rooted in how traditional processors have been designed for decades, really. You have your memory units, the physical silicon where the data is stored, and then you have your compute unit.
SPEAKER_00The logic gates where the actual math happens.
SPEAKER_01Right. And they sit in different physical locations on a circuit board or even within a chip itself. So to perform an AI calculation, the system has to constantly shuttle massive amounts of data back and forth across a bus.
SPEAKER_00It's constantly moving data.
SPEAKER_01Constantly. And every single time those electrons travel across that physical distance, it consumes a measurable amount of energy and introduces latency. Right. So when you are processing billions of parameters a second, you end up spending significantly more time and electrical power just moving the data around than you do actually doing the math.
SPEAKER_00I always visualize traditional architecture like this massive commercial kitchen. You have a world-class, incredibly fast chef, and that chef represents your compute unit.
SPEAKER_01Okay, I like this.
SPEAKER_00And you have this colossal fully stocked pantry, which is your memory, but the kitchen was designed poorly. So the chef is separated from the pantry by like a hundred-foot-long hallway.
SPEAKER_01Right. So the chef can chop an onion in a microsecond, but that speed is totally irrelevant if they have to sprint down a hundred-foot hallway just to grab the onion.
SPEAKER_00Exactly. And then sprint all the way back to the cutting board. The physical running back and forth ruins the efficiency of the entire kitchen.
SPEAKER_01That's a great analogy. And untethered AI essentially looked at that kitchen and realized, you know, the solution wasn't to just hire a faster chef.
SPEAKER_00Right. They took a completely different approach.
SPEAKER_01Yeah, it's called at memory compute architecture or AMC. Instead of forcing the chef to run to the pantry, they aren't just moving the pantry closer. They are giving every single chef their own fully stocked mini fridge directly at their cutting board.
SPEAKER_00Which is brilliant.
SPEAKER_01It is. And what's fascinating here is the physical engineering of that concept. AMC physically places the compute elements directly adjacent to the memory cells on the silicon itself.
SPEAKER_00So they're creating these incredibly short, direct, parallel connections.
SPEAKER_01Exactly. They bring the computation to the data, it reduces the physical distance those electrons have to travel from millimeters down to micrometers.
SPEAKER_00So they effectively eliminate the hallway.
SPEAKER_01Totally. That minimizes the latency and it drastically cuts the power consumption associated with data transfer. It is an architecture purpose-built for real-time AI inference.
SPEAKER_00Where high data volumes and diverse workloads just have to be processed instantaneously. And the physical proof of this is in the ML Perf inference 4.1 benchmarks that Holman details in his analysis.
SPEAKER_01Right, the industry standard.
SPEAKER_00Yeah. For context, ML Perf isn't some internal marketing metric that they just made up. It's the gold standard benchmark developed by ML Commons, which is a consortium that includes the absolute heaviest hitters in the industry.
SPEAKER_01So these are highly verified numbers.
SPEAKER_00Exactly. And the verified results for Untether Speed AI 240 preview card showed it achieving up to six times the energy efficiency and three times lower latency compared to the competition.
SPEAKER_01I mean, six times the energy efficiency is a paradigm shifting number for hardware engineers. It really is. Oh absolutely. And if we look at the ResNet 50 image classification benchmark, which, you know, is a standard test for how fast a system can identify objects and images, this single PCI card hit an industry-leading 70,348 samples per second. Oh wow. Yeah. It secured the highest throughput in both the data center and edge categories for the specific form factor.
SPEAKER_00Processing over 70,000 images in a single second on one Slim card is just staggering. But um that sheer performance metric kind of begs a massive question.
SPEAKER_01What's that?
SPEAKER_00Well, if this data movement bottleneck is universally understood by every hardware engineer on Earth, the undisputed titans of the industry companies like NVIDIA and AMD, they aren't just sitting on their hands, right?
SPEAKER_01No, no, definitely not.
SPEAKER_00Holman's analysis points out they're taking completely different philosophical approaches to solving the exact same physics problem.
SPEAKER_01Yeah, the established giants are innovating heavily in memory compute integration, but they are doing it in ways that leverage their existing massive architectures.
SPEAKER_00Right.
SPEAKER_01Let's look at NVIDIA's strategy. Their solution to the kitchen hallway isn't the mini fridge. Their solution is to basically build a 10-lane superhighway between the chef and the pantry.
SPEAKER_00Okay, so just bigger, faster data pipes.
SPEAKER_01Exactly. They are utilizing high bandwidth memory, specifically HBM3, combined with their proprietary NV Link interconnects.
SPEAKER_00So they are stacking the memory chips vertically, like tiny silicon skyscrapers, to pack more memory into a smaller footprint. Yeah. And then using these massively wide data buses to just shove as much data through as quickly as possible.
SPEAKER_01Yeah, that is the absolute core of their Blackwell architecture. They combine that vertically stacked HBM3 memory with a massive L2 cache. Trevor Burrus, Jr.
SPEAKER_00Which is like a small pool of ultra fast memory, right?
SPEAKER_01Right. To keep a working subset of data as close to the compute elements as possible. And AMD is taking a similar route with their MI300 X GPUs.
SPEAKER_00Right, relying on vast pools of HBM3 memory.
SPEAKER_01Yeah, to keep giant AI models, like large language models stored relatively close to the compute units just to minimize those off-chip transfers.
SPEAKER_00Okay, but here's where it gets really interesting for me. If Nvidia and AMD are pouring billions into building wider, faster highways, while Untether AI is literally building the house on top of the grocery store with AMC.
SPEAKER_01Right.
SPEAKER_00We also have to factor in IBM and Intel, who are experimenting with even more radical departures, according to the article.
SPEAKER_01Oh, absolutely. IBM as North Pole processor takes this modular approach they call on-ship memory.
SPEAKER_00How does that work?
SPEAKER_01Well, they are co-locating memory and processing on the exact same silicon dye, allowing the data to be processed within the memory plane itself. But Intel's research is where the physics get truly wild.
SPEAKER_00Right. Intel is exploring analog computation for ultra-low power edge AI. And I want to make sure we don't just gloss over analog computation because um it sounds like a step backward to 1950s technologies.
SPEAKER_01It really does.
SPEAKER_00But it's actually a brilliant workaround for power consumption because AI neural networks fundamentally rely on endless repetitive math called multiply accumulate operations or MCs.
SPEAKER_01Aaron Powell Right. It's basically just multiplying inputs by weights and adding them up.
SPEAKER_00Exactly.
SPEAKER_01And in a digital chip, doing millions of mass C operations means flipping billions of microscopic transistors on and off. Right. Which uses digital logic gates and consumes constant electrical power. Aaron Ross Powell Right.
SPEAKER_00It adds up fast.
SPEAKER_01Trevor Burrus So Intel is researching how to use mixed signal circuits directly inside the memory cells to perform those M mass C operations in analog.
SPEAKER_00Wait, so instead of flipping digital switches to represent ones and zeros.
SPEAKER_01They measure the physical voltage drops across the circuit. They are using the raw physics of electricity to do the math instantly. It completely bypasses the digital conversion process.
SPEAKER_00Oh wow. So that drastically cuts down on both latency and the energy required. So we have NVIDIA's superhighways, IBM's on chip memory, Intel's analog voltage math, and untether AI's at memory compute. As a strategic leader looking at this landscape, does Untether AI actually have a fundamentally better architectural philosophy, or are they simply optimizing for an entirely different set of constraints?
SPEAKER_01That is the crucial question. And it brings up deployment realities because NVIDIA and AMDA's architecture is brilliant for absolute scalability and raw, unadulterated processing power.
SPEAKER_00Which is why they completely dominate the training of massive LLMs in hyperscale data centers.
SPEAKER_01Right. The Ten Lane Superhighway works perfectly when you have a dedicated power plant feeding your data center and massive cooling towers to handle all the heat.
SPEAKER_00But Untether AIA's philosophy is strictly optimizing for severe physical constraints.
SPEAKER_01Exactly. Limited electrical power, restricted physical space, and the demand for absolute real-time low latency.
SPEAKER_00Because 70,000 images a second is a fantastic lab trick. But the business reality comes down to who is actually buying this when Nvidia already owns the data center ecosystem.
SPEAKER_01Yeah, let's look at the financial stakes Holman pulls from SNS Insider Research.
SPEAKER_00Oh, the numbers are huge. We are looking at an AI chip market valued at just over $61 billion in 2023. And it's projected to soar to over $621 billion by 2032.
SPEAKER_01A 29.4% compound annual growth rate. But the crucial detail for any strategic roadmap is that the inference segment, the chips running the models in the real world, is growing even faster at over 30%. Trevor Burrus, Jr.
SPEAKER_00Right. And that specific growth is driven almost entirely by the demand for low-latency, energy efficient solutions in edge environments.
SPEAKER_01Aaron Powell, which is precisely where Untether AI is carving out its territory.
SPEAKER_00Aaron Powell Because the physical engineering realities of the edge basically dictate the hardware. Like we mentioned the autonomous vehicle scenario earlier. You literally cannot put a massive 700-watt data center GPU in the trunk of an electric vehicle.
SPEAKER_01Trevor Burrus, no, the heat generation would melt the surrounding components.
SPEAKER_00Trevor Burrus, Right. And the sheer power draw would drastically reduce the car's driving range just to run the sensors.
SPEAKER_01Aaron Powell Exactly. And Holman provides concrete evidence of this market traction. Mercedes-Benz is actively collaborating with Untether AI to deploy this specific at-memory compute technology in their autonomous vehicle programs.
SPEAKER_00That's a huge validation.
SPEAKER_01It is. And beyond that single automaker, Untether AI has announced a major integration with ARM. They are combining their inference acceleration with ARM's latest automotive enhanced technology.
SPEAKER_00So they're creating a standardized, power-efficient platform specifically for advanced driver assistance systems across the broader automotive industry.
SPEAKER_01Yes.
SPEAKER_00That automotive integration is a massive foothold. But you know, to ensure this briefing remains rigorous, we have to address the elephant in the room, which is the incumbent mode.
SPEAKER_01Right, the reality check.
SPEAKER_00Yeah, Untether AI has a clear hardware advantage in these power-constrained edge scenarios, but silicon is only half the battle. Nvidia and AMD are deeply entrenched across every major cloud provider.
SPEAKER_01And more importantly, NVIDIA has spent nearly two decades building CUD, their proprietary software platform. If we connect this to the bigger picture, the software ecosystem is the ultimate defensive moat. Millions of developers are trained to write AI applications that compile perfectly on NVIDIA hardware.
SPEAKER_00So a newcomer can build the most power-efficient chip in the world, but if developers have to spend months rewriting their models to actually run on it, the chip will just fail commercially.
SPEAKER_01Exactly. Broad software compatibility, seamless integration with existing AI frameworks like PyTorch or TensorFlow, and industry standard support, those remain monumental hurdles for untethered AI.
SPEAKER_00Overcoming that software friction is arguably a harder challenge than solving the physical data bottleneck in the first place.
SPEAKER_01I would agree with that.
SPEAKER_00So we have established untethered AI's technical dominance in this specific power-constrained edge niche. And we've laid out the immense scale of the incumbent software emotes. For a business leader trying to categorize this threat, we have to ask if Untether AI fits the classic definition of a disruptor. Clayton Christensen's theory from the Innovator ES dilemma usually describes a disruptor as a company that enters at the low end of a market with a cheaper, technically inferior product and then gradually moves up.
SPEAKER_01Yeah, and untether AI processing 70,000 samples a second is certainly not an inferior product.
SPEAKER_00No, definitely not.
SPEAKER_01It isn't a traditional low-end entry, but Holman's analysis demonstrates how Untether AI actually perfectly exhibits the three core characteristics of Christensen's disruptive model.
SPEAKER_00Okay, let's hear them.
SPEAKER_01First, they are serving an underserved market.
SPEAKER_00So the edge in industrial applications.
SPEAKER_01Right. The mainstream giants are entirely focused on massive cloud tasks and scaling absolute performance for LLMs, which leaves the specific power-constrained needs of automotive and agricultural AI largely ignored by traditional architectures.
SPEAKER_00Makes sense. What's the second characteristic?
SPEAKER_01Second, untether AI is prioritizing distinct performance metrics.
SPEAKER_00Oh, so they are completely changing the scorecard. Instead of competing on who can achieve the absolute highest throughput with unlimited power, they are forcing the competition to be judged on performance per watt.
SPEAKER_01Exactly. They are offering a competitive alternative built entirely around energy efficiency and thermal management.
SPEAKER_00Which leads to the third disruptive characteristic, right? Gradual market encroachment.
SPEAKER_01Yes. By establishing a dominant foothold in niche edge applications today, they are funding the incremental refinement of their at-memory compute architecture.
SPEAKER_00So as the technology matures and proves its reliability in these really harsh environments, untethered AI is positioned to slowly expand up the stack.
SPEAKER_01Exactly. They can begin encroaching on broader, more traditional data center roles, particularly for companies looking to drastically cut their server farm electricity bills.
SPEAKER_00And as that incremental growth happens, it will begin attracting mainstream enterprise customers, which actively forces the incumbents to pivot and respond. So what does this all mean for your strategic understanding of the landscape? Synthesizing Holman's analysis, Untether AI is a highly specialized contender. They're not going to render NVIDIA obsolete in the hyperscaler data centers tomorrow.
SPEAKER_01No, probably not.
SPEAKER_00But their at memory compute architecture is capable of forcing the entire AI hardware trajectory to reevaluate the physics of data movement.
SPEAKER_01Right.
SPEAKER_00Scaling AI into the next decade isn't just about building wider highways and bigger data centers. It was about bringing the compute directly to the data. It is about allowing complex AI to live efficiently on the edge inside our vehicles and industrial sensors without burning through massive amounts of power.
SPEAKER_01Which leaves us with a critical strategic question as we look toward the 2030s.
SPEAKER_00Oh.
SPEAKER_01Yeah. As AI models become deeply integrated into mobile devices, autonomous infrastructure, and remote environments, will the raw, unconstrained computational power that defines today's AI monopolies eventually become a secondary concern?
SPEAKER_00That's a fascinating thought.
SPEAKER_01We may be rapidly approaching a future where the ultimate winner of the AI hardware war isn't the company that can process the largest volume of data, but the company that can process it using the least amount of electricity.
SPEAKER_00It fundamentally changes how we look at that commercial kitchen. The ultimate winner isn't going to be the restaurant with the most massive pantry. It's going to be the engineer who finally figured out how to put a mini fridge right next to every cutting board. Thank you for joining us on this deep dive.