A lot has changed since we last mapped Israel’s machine learning infrastructure ecosystem. It’s been nearly eight years since our first landscape and seven years since the sequel, and if you told us then that we’d be living in a world in which Israelis colloquially refer to one of the most impressive consumer products ever built as “Ha-chet”, we wouldn’t have believed you.
As a quick TLDR of where we are though: the AI infrastructure opportunity has shifted dramatically three times in less than a decade. First it was about training. Then it was about inference. Now it’s about agents.
The Training Era: 2015-2022
The common AI use cases in a post-AlexNet world through the launch of ChatGPT were predicting between binary outcomes. Is this a cat or not? Is there an abnormality in this MRI scan or not? Does the progress on the construction site match the plans? And so on and so forth.
The major bottleneck in that era was training.
Companies asked themselves some variations of the following questions:
How do we get enough data to train our models, can we tag/label the data accurately, how can we train the models efficiently, how can we ensure that our data scientists’ work mirrors that of a software engineer? All of these questions boiled down to a clear output: training a model that would accurately predict a binary outcome.
The Inference Era: 2022-2025
The birth of LLMs saw the emergence of the AI hyperscalers, which took upon themselves the massive costs and burden of training “super” models – now known as foundation models. The nature of these models is that they are not predicting a binary outcome, rather they are predicting the next most likely outcome (token) based on an almost infinite number of possible outcomes. They aren’t predicting as much as they are generating (hence generative AI).
And so the major bottleneck became inference. How do you serve these massive models efficiently? How do you make them fast enough for production? How do you control costs when you’re paying per token?
In fact, the costs of inference began to create a shift away from closed source models to open source models (see below image). Why pay a premium when you can fine-tune Llama, Kimi or Qwen for a specific use case and run it on your own infrastructure?
The Agent Era: 2025-Present
Models got good enough that the bottleneck is no longer how to serve them efficiently. The bottleneck is how to get them to do useful work reliably in the real world. Which is to say: the bottleneck is now agents.
The technical challenges of making agents work at scale are fundamentally different from the challenges of serving LLMs efficiently.
How can agents retrieve context correctly and efficiently? How can agents be enabled to make continually better decisions as they reason, plan, act? How do agents interact with one another and how can agents be trusted? How do you evaluate whether an agent actually did what you asked it to do?
For the most part, these are engineering challenges.
Exactly the type of problems Israeli founders are uniquely qualified to solve.
Opportunities in the Agent Era
When thinking through the AI infrastructure opportunities facing us in the agent era, we’re excited by three key categories:
- Sovereign Models
- Agentic Infrastructure
- AI Integrity
Models: The Importance of Agentic Sovereignty
The dependence on American or Chinese model providers is a strategic vulnerability. When agents control infrastructure, manage sensitive data, and make decisions, you can’t outsource that capability entirely.
The good news is that the ecosystem is being built.
Nebius just deployed 4,000 Nvidia HGX B200 GPUs in Israel, one of the country’s first publicly available Blackwell deployments. They’ve signed deals for 80MW of capacity across two data centers, investing nearly $900 million.
Crusoe acquired Atero in an effort to establish a local presence and take advantage of the local talent pool.
And of course, Israel has become Nvidia’s HQ2. Between the Run:AI and Deci acquisitions, and the planned campus in Kiryat Tivon that will match the scale of Nvidia’s Silicon Valley headquarters, Nvidia’s bet on Israel is a massive step towards ensuring Israel’s standing in the global AI race.
Kudos to AI21, AAI, Decart and a few stealth companies for being the first companies to dip their toes into local model building, but more attempts are needed.
Agentic infrastructure: building for systems, not humans
But models are just one piece.
Think about what’s required to make agents work in production:
- Orchestration frameworks that can manage complex workflows and handle branching logic
- Observability tools to help monitor agent actions
- State management systems that can track context across hours or days of agent operation
- Tool integration layers that give agents reliable access to the real world
- Memory systems that go beyond simple vector databases
Evaluation frameworks are perhaps the most critical missing piece. As agents begin to proliferate the enterprise, companies need robust ways to test behavior, measure performance, and safely train systems through feedback loops. The currently available tools simply aren’t robust enough.
Agents need to remember conversations, learn from past mistakes, be evaluated and recall relevant context at the right time. This requires new approaches to storage, retrieval, and reasoning about when information is relevant.
In other words, an entire new infrastructure stack must be built.
AI Integrity
While AI security may be the soup du jour, we believe that a more appropriate title for AI security in the agentic infra era is: AI integrity.
AI integrity is the assurance that agents do exactly as intended, and nothing more (see this previous weekend’s Moltbot mania for a demonstration of the importance of this).
This goes beyond preventing hacks or breaches; it’s about trusting agents that can read emails, move money, write code, and interact with customers and sensitive data. The real question isn’t whether an agent can be compromised, but how it can be proved to be acting with integrity.
Israel has a natural edge here, rooted in its culture of adversarial thinking, where security emerges from system architecture rather than add-on defenses.
Agent integrity demands the same mindset: safety can’t be layered on, it must be baked in. In practice, this means provable execution paths, behavioral sandboxing, and rigorous adversarial testing of agent behavior. It also requires interpretable decision-making, so humans can audit and trust critical actions.
The companies that solve AI integrity won’t sell traditional security tools, they’ll build the foundational infrastructure for trustworthy agents, on par with systems protecting financial, defense, and nuclear assets.
The Path Forward
So where does this leave us?
We believe Israel is uniquely positioned to be a major player in the agent infrastructure wave. The pool of local talent is incredibly well versed in making systems more efficient, more secure, and more reliable. In other words: helping make AI systems work in production.
The shift from training to inference to agents isn’t just a change in bottlenecks. It’s a change in the types of companies that will win. And everything about that change favors Israeli engineering talent.