EpisodesConversation

Conversation

Is NVIDIA’s Moat a Psyop? 28x More Efficient AI Chips | Elias Almqvist, Zetta (YC S24)

1h 10m
Is NVIDIA’s Moat a Psyop? 28x More Efficient AI Chips | Elias Almqvist, Zetta (YC S24)

Transcript

In this conversation
Nihal

get started. And yes, we are really happy to have you here today because you are touching a topic that we are all super curious about and we haven't seen so many brave hearts tackling that area for us.

Elias· Zettascale

Yeah.

Nihal

Why don't we get started with you introducing yourself as a human being for the ones that haven't got to know you yet.

Elias· Zettascale

Cool, yeah. I'm Elias, or how you wish to pronounce it. I'm from Sweden. I'm 24 right now. I dropped out of university to start my company. Yeah, my background is basically like I'm self-taught in literally everything. I went to university because I had nothing else to do and I sort of realized that I was wasting my time. And then I started my company. That's like pretty much it. So I come from like a small town called Möndal, or Eklund, which is like a bit outside of Gothenburg. there's not much else to say there honestly, like yeah, I'm moving out to the Bay Area on the SF for YC and I've been here since then I guess.

Nihal

Yeah, actually what struck me was when I take a look back at your background, Sweden, small town, but then you had this really super pretty picture with your father that in the bay area when you were a kid. It sounds like you had little bit of the seeds in your head like at the earlier days, right?

Elias· Zettascale

Yes, yes. Yes. Yeah, so when I was like nine, 10, or whatever, around that age, I wanted to be an inventor, because I didn't know what a founder was. So it of started off with me being like, I wanted to invent these things, sort of like taking apart my toys, and then there's a whole other thing. But it was sort of tinkering. I wanted to be an inventor, then I sort of thought I would have found a voice. And then about nine years ago, I think, I went there on vacation to San Francisco with my father and my mother, mom and dad. And it's of sort of like serendipitous that I sort of just ended up here because I remember like really enjoying being here in San Francisco. Back then I wasn't like interested in like the tech culture. I was just a kid. But it's sort of like I went here and it's sort of like the SF bug like slowly creeped into my head and it's been growing ever since and then I, yeah, had to move here.

Nihal

Hmm. Hmm first time we talked and you told me that this is the only place I want to be right now on the earth. And this is, I don't want to be anywhere else. Like it was really quite like convinced Tom, but it was not like wherever I go.

Elias· Zettascale

Yeah, like I mean... Silicon Valley is like weird. Like people will show you on for doing the most like the most ambitious thing ever even if they think it's stupid. If you go to Europe and or like Sweden that is my like paradigm example like people will look down upon you like they'll try to like like bring you down because you're different and that's like extremely draining it's like you can't really grow from there and you sort of have to fit in them.

Nihal

Mm.

Elias· Zettascale

the people actually like who likes to proceed, I guess the immense peer pressure or jump them as low as people call it. sort of like the people come out of Sweden, like the technical talent, the founders are really strong because they survive this hellscape of like, you're not welcome here, you're too weird basically. And they always use, they just end up in San Francisco. Then there's a tech hub in Stockholm that's doing really well. But at the end of the like every single... founder out of Sweden. It's really weird and a really strong... And like when I talk to VCs about it, they're like, oh, we want to look at the Nordics now because of this like hellscape and the people who survive it are really good for some reason. So I do really think that like, trains the faithful, guess, whatever the saying is.

Nihal

Yeah. It is sometimes it makes you feel unfair kind of because actually you're just in a different like it's not like you fit in or you don't fit in but it's just like your environment is similar. I had a professor in the university he once told me this is like taking a kitten and putting it into puppies then that kitten will think why am I not barking? Why am I not strong? But actually the place that

Elias· Zettascale

Yeah.

Nihal

that kitten belongs to its next to the other kittens. I think there's a little bit of similarity to that. It always makes me think.

Elias· Zettascale

Yeah. Yeah, like, I environments are extremely important. Like, we humans, adapt to our environment. And if you're fundamentally, like, not supposed to be, like, you don't really fit into a specific environment, there's going to be conflicts. You're not going to be happy. You're just going to be depressed and you're just going to try and, like, walk around. So then eventually you're just going to break. So it's really important just to find the environment where you really, like, in. For me, it's San Francisco. For me, it's like tech in Silicon Valley, like building deep techs.

Nihal

you

Elias· Zettascale

a deep-ed company. For others it might be like, I don't know, what is the finance capital? It's like London or New York. If you finance, go there. If you like art, I have no clue where that is, I'm not, yeah, it doesn't matter. If you like those things, you should be in those places, but if you really want to be in tech and you want to do these, you want to build something revolutionary in tech. You have to be in Simcoe Valley, because that's where everything's happening. And that's also why it's sort of like a magical place.

Nihal

Mm. Yeah. Yeah. I think I totally get you and probably many audience or audience will also resonate with that. By the way, we had audience of like a really great mix in terms of university students as well, but at the same time, professionals like who has. it could make some experience and builders and researchers. Like we had a very pretty wide range of age at the same time all coming around the deep deck and building hard stuff. But among all the other topics, subjects, how did you end up in CHiPs?

Elias· Zettascale

And chips, ⁓ well I've always been interested in computers. It's like literally the only thing I've been interested in since I was like 10. Like sure, I made my side inventions of making whatever, but it all started from being like I've always been interested in computers and computing in general. And that snowballed into being interested in mathematics and that's snowballed into being interested in computer engineering. the point of computer engineering is to try to give this machine to the mathematicians or the physicists or the statisticians or the AI people to be able to do these amazing things and it's sort of like the sense of like it's like a tool to basically accelerate our own intelligence. It's like you move like if you have a simple calculator versus you have a supercomputer you can do a lot more with a supercomputer and I think

Nihal

Mm.

Elias· Zettascale

That's what I realized. I'm gonna go see why I like JEPs and computing and computer engineering and computer architecture. I don't know, I mean, I was really obsessive. It just sort of happened, to be honest.

Nihal

Do you have moments in your mind that when you really recognized, ⁓ okay, this is the thing that I need to pursue and I try to understand in depth.

Elias· Zettascale

There's never like, oh shit, I gotta do this thing. It's more like a successive, like, accumulative evolution throughout time. So it started with me being like, I cannot like computers, I like video games, I wanna build my own video game. Then it started with me tinkering with, like, when I was very young with my own toys and like taking them apart and doing all of these, whatever. That sort of just snowballed into this direction and I just kept following the things I'm extremely interested in.

Nihal

Mm.

Elias· Zettascale

and this is the place I ended up in. It's like, yeah, I have my own company, whatever. This is not a job for me, it's not work. I'm having fun every single day. Just follow that instinct and you'll be good.

Nihal

Yeah. Okay. Then actually we can phase into now Zeta scale. Like I've been following your work ever since you popped up on my CIS company directory. And then you guys had a little bit of like changes with the name and with the papers. You thought, we have a more advanced way. We're going to update this and so on. Perhaps let's get started with the very big picture. How do you see the industry right now? And also what's your vision? And then afterwards we can work backwards from there to today and then take it to the specifics that I have in my mind.

Elias· Zettascale

Yeah. Yeah. What it looks like, like right this very section for AI specifically is we have this huge like LLM boom, like Shatchi, BT and Claude and Claude code, all those sort of like language models. And then we have all these visual models and we have mid journey and all these sort of like modalities that's like we have basically infinite data of like the whole internet. This is what the internet runs on basically. And this exploded because they started scaling really well on GPUs. Because they fit really well on the GPUs. that's not a coincidence, that's not a mistake. That's more of like, that was like, more probably than not bound to happen sooner than later. So, AMD, even Google, with their GPUs, they're sort of like, speed running into this like, LM direction of trying to make the best frontier model the best.

Nihal

Hmm.

Elias· Zettascale

chat bot, the best companion. Everyone's speeding in this direction, and then you have companies show up like Grok, Cerebras, a bunch of other companies trying to make LLM-specific chips. So everyone's double focusing on this thing. But then we sort of forget the point of AI and the point of computing is...

Nihal

Yep.

Elias· Zettascale

if you're sort of focusing too much on this one thing and you're betting on it, you're sort of neglecting everything else. So I usually say AI is not a bubble, LLMs are. And that's very hard for some people to believe because they've deployed like millions of dollars of capital into LLMs, but I'm not saying they will be worthless. They'll be worth hundreds of billions of orders, maybe a trillion. But when it comes to the future of AI, that...

Nihal

Mm.

Elias· Zettascale

there's more to AI that we can actually do when we only scratch the surface. So what I'm predicting what the market's gonna be like is like, yeah, sure Nvidia's rushing to make GPUs and do all of these cool things, but. What essentially the market needs, they need versatility. We need to enable the AI people to do their actual research. That's the whole point of computer engineering in the first place, We give them the machine to model their mathematical models or whatever they're trying to compute. Without that, it may force them to go in a specific direction, also called the hardware bias. They'll essentially end up kind of blinded by that and then we'll have a diminishing return.

Nihal

Okay.

Elias· Zettascale

and people are starting to notice that now. And then people are gonna be like, shit, no AI's a bubble. It's not, it's just these people betting a lot on this specific thing. And it's gonna be very important in the future for like, agenda workloads. But if you're doing more discovery, like material discovery or protein folding, like what DeepMind did.

Nihal

Mm.

Elias· Zettascale

You have to be smart, have to support those, you need versatility. So that's why ASICs is their call. I don't think they're the future.

Nihal

But you are in a place in order to really nail it, you need to have your very own AI vision for the AI models, vision for how this is going to continue. And also the vision around the co-design of hardware and software, right? And this is, yeah.

Elias· Zettascale

Yeah. Yes, the only company I would say is doing, well, NVIDIA has your AI, whatever, department. What the companies are doing with wireless Google, Google DeepMind, like, they have DeepMind, they actually have Frontier Research, they have this communication layer with the TPU team that they, oh, we sort of need this thing, could you please, like, help me? That is the most important part.

Nihal

Mm.

Elias· Zettascale

If you don't have that communication, you're just gonna end up with that company filled with hardware people thinking that the hardware is so amazing. But then you sort of push out the hardware and then the compiler becomes like, depending on what the hardware is, like mega complex and all these MLP like, I don't wanna deal with this shit. And then you sort of like have a really cool hardware project. That's like an art project, but it's kind of useless. And yeah.

Nihal

But actually Google is one of the scroll of it. If you take a look at it, right? Because also the startups that get out of it, including GROK was also from Google. And there you also can see this reasoning is very strong. And then they look at it more like an entire ecosystem in a place where they can make their differentiation and double down on it rather than, hey, let's cover everything and nothing. Because so far it worked for Nvidia. Like correct me if I am wrong. It was...

Elias· Zettascale

Yes.

Nihal

course, they started with something again, back then nobody had seen coming, but it then turned out to be an AI enabler. And afterwards, now, but also it locked down the entire industry. Like we are talking about some company like, please, the worst, most valuable, and 92 % of the market share belongs to them. If you, actually, I remember a conversation around CUDA, how such a big block is.

Elias· Zettascale

Yes. Yeah. Yes, there's two parts to this actually. I'll get to CUDA later. It's not a mod by the way, if you want to keep listening. So yeah, sure, you have Grok spinning out of these companies and they're making this bet on this, whatever, language models basically. I was saying that that's gonna be useless in the short term. That's gonna make a few billion dollars in the short term. They got acquired or made a license of 20 billion dollars. Cool, great for them.

Nihal

All right. Yep.

Elias· Zettascale

But there's also the difference between a billion dollars and a trillion dollars. If you're limiting yourself and the whole of humanity and the future of humanity to these transformer models, what are you doing? You want them to write more emails and be better writing my slow AI vibe coded app, then that's not really the future of humanity. If we do that for the next 100 years, then that's gonna be depressing.

Nihal

Mm-hmm.

Elias· Zettascale

Yeah, this is like with Google and DeepMind, they're sort of on the right track. I think they're too big of a company to make like a revolutionary step in the hardware direction unless they do something very like, like a major restructure or something. But yeah, and then when it comes to code, see it's simply, it's not a mode. Sure, Nvidia has this massive market share, but if you like, okay, so. If you go and ask the machine learning engineers, the AI engineers, do they like writing CUDA? Is that productive? It's not really productive, it? Their job is to basically model, like create a model that fits whatever data set they have, how it work, and having to optimize every single model to their, like, with writing CUDA kernels. You sort of are bottlenecked by the actual hardware. And then when it comes to the actual, like, Okay, so what was the replacement like? What was the back one look like? So just because CUDA is currently the best thing on the market doesn't mean it's great.

Nihal

Yeah, this is, I think similar to, yeah, yeah, I think my point was more into the ecosystem view in a way they call it end to end. okay, okay. I'm patient. I'm gonna be patient.

Elias· Zettascale

That is the major part. Yeah. you I'm getting to that. So that is the first part. So people are suffering. What we are seeing now is that the entire machine learning community, they don't really want to write these kernels. They just want to say to PyTorch or JAX and like, oh, it works, cool. It's mostly abstracted away from them. The people who writing these kernels are kind of wasting time, in my opinion. When it comes to the entire ecosystem, what we're currently seeing is a major incentive, like this massive incentive structure to move off of CUDA, because people don't want to be vendor-locked by Nvidia because they have massively inflated prices. That's the first part. Also because they have this huge hardware bias with GPUs and are really hard to pro... like not hard, but it's not fun, I guess. Unless if you're really weird and you like write CUDA kernels. So this is massive incentive structure to just... get away from Nvidia, OpenAI went to AMD and Cerebros recently and all these other companies are trying to like go to AMD and RockM is like foraging and what if you look at the entire ecosystem this has sort of happened before. So we're all moving to this open source ecosystem. MLIR is doing really well. This is like the backend for my Sheila you can view. And. Simply because they don't have this incentive, it's slowly accumulating the open source community, slowly building up this open source standard, which really benefits all of the labs, which is essentially the end customer, or even the hyperscalers, because they don't have to deal with this software hell. And QoO does a moving target, and they have to be backwards compatible, and it's proprietary, so you can't fix it yourself. And we all know how AMD handles software, and if Nvidia slips up, they're done. So for CUDA to actually be relevant in 10 years or even two years, they either have to open source it or they'll have to, some open source thing will just take over. That's the reality of it.

Nihal

How do you see, let's say 10 years forward we went, like 10 years forward, how do you see the overall picture of work? It's more architecture agnostic, it's more in the direction of highly specialized and then more collaborative.

Elias· Zettascale

I envision, like the way you do modeling today is like you actually define your model like declaratively like this is my model, this is our structure like if you define a transformer model you define the transformer it's gonna be like this XYZ your like your compiler or basically your hardware is the point of it is to basically be able to represent that model as closely as possible and again coming back that you sort of need versatility for that but if you have too much versatility like an FPGA you sort of like the compiler is super complex and all that sort of shit. coming back to CUDA and why the open source thing always just wins, it's just an open source model because U.N. customers are developers and what we really saw before, and if you just look at the history, what won? Was it Windows, Unix, or Linux? It was always Linux. Because this is massive incentive from like a bunch of players, like decentralized players to actually use this thing.

Nihal

you

Elias· Zettascale

And it sort of created this open source community that's really strong. And then I know Microsoft is trying with Windows Server. Is anyone using that, to be honest? That's like, yeah, okay. And it's mostly like if you're making something for developers, you sort of have to be open source, on the software end. If you show up at something proprietary and yell at them and tell them, okay, you gotta use this thing. Here's the documentation, good luck. Like, if they can't fix it themselves, like the whole point of being like an engineer is being able to fix the thing that you're actually working with. And if you're constrained with something some big company's telling you to do, then you're not gonna be happy with that. And that's another incentive to like just move off CUDA. it's like, it's basically just technical depth in the market right now.

Nihal

What do you think about the switch cost? because right now there are different point of views, but in the end of the day, what is working is NVIDIA ecosystem at the moment. Right. And yeah, exactly. And also the, there is something called inertia when people come across with something new, not everyone is open as you in the way that the approach things. This is early adapters.

Elias· Zettascale

Yeah. Like, it wouldn't happen overnight. That's the thing. Yes.

Nihal

piece that you need to get, like how are you going to approach this? And also what would be your differentiation so that you promise them something so amazing they won't even think of what they have in their mind. They will just drop and come with you.

Elias· Zettascale

Okay, in reality if you show up with the world's best chip and your software is like utterly like horrible, no one's gonna use it. That's what happened in Graphcore. Right? Even if you show up with the worst hardware in the world, with the worst software, obviously no one's gonna use that. And if you show up with like mediocre hardware, like worst hardware in the world, okay software, and it's currently the best software out there, then people are gonna use that. And that's literally just Nvidia. So it's really just a curve. So like, yeah, the only adopters won't be like these mega huge hyperscalers from day zero. It will be researchers. It will be people doing this frontier edge research, basically scientific discovery with the help of AI, ⁓ small startups, like people willing to actually try this thing because they are basically small, they can try it. And what you basically have to do is you sort of have to traverse the curve up and then...

Nihal

Hmm.

Elias· Zettascale

reach more and more people, but as for our go-to market or whatever lingo you wish to use is we're basically seeding our chips and our ecosystem in these small number of people. And then over time that the ecosystem develops. And it's not like we're developing our own software ecosystem because that would be stupid because MLIR already exists. They like the open XLA project is like it exists. are using it today. Like if you, you ever use Jack's for example.

Nihal

Hmm. So then you

Elias· Zettascale

They use, Miter is just that big evil blob thing, but that's not a story.

Nihal

No, but actually then you aim for really ahead of the curve folks and most of the times these excited people are doing novel work, right? So then you're not just really going to very established companies, AI folks that building in that space, rather you go for after the researchers or startup folks. And actually you kind of draw parallels to the ecosystem. So you're not just trying to boom and then get bigger. No, no, no.

Elias· Zettascale

Yes. Yes.

Nihal

You're going hand in hand and then slowly, slowly, you just get more steak. Did I understand you correctly?

Elias· Zettascale

Exactly. Yeah. So like it's a it's cumulative like it's compounding over time and that's just the ecosystem part. So a small short thing about our hardware is it's meant to be as versatile as possible so you can fit your model into the best as possible. The good part of that is like as we're developing the ecosystem more and more people at the same time these early people because they are on the frontier literally. will have more and more people making breakthrough AI papers like, attention is all you need. It's gonna go viral. And over time, we'll end up with two more and then five more and then 10 more and it will be a compounding effect until we have such a Cambrian explosion of AI research and possibility that I said, was that like an event yesterday where I spoke? was like.

Nihal

Hmm. Yeah.

Elias· Zettascale

It's going to be an explosion of AI progress and research and it's going to have an extreme compounding effect on literally every single industry. Not only in writing emails and doing accounting, it's going to be affecting material discovery, space technology, literally every single thing you can think of will get so accelerated that the future is going to be massively insane. That's the future I really want to live in and we're heading in that direction.

Nihal

Yeah.

Elias· Zettascale

And I have this privilege, like I'm literally on the frontier walking on this. And that's like the best feeling in the entire universe, to be honest.

Nihal

This is what you wanted yesterday, That's why you're so impatient, like, hey, this needs to come today, like, immediately. At the same time, when you collaborate with those folks, you co-build your product together with them, which gives you also advantage because you don't want to just drop something to somebody and then you don't hear something back or you don't have shared the same enthusiasm. Is that also a part of your approach where you aim for small teams? There is no like overhead, there is no decision making trouble and also results speak for themselves because imagine if something happens in science space and then you are a part of that success and then this is gonna be much more effective than only let's say cost conversations or only, yeah I'm gonna replace, I will have my productivity X and X but in that case, you're kind of like being a part of a breakthrough.

Elias· Zettascale

Yes. Yeah.

Nihal

What sort of folks are you looking for right now to co-build your product together, to be more specific? Like, okay, researchers, any specific area, any specific...

Elias· Zettascale

Anyone doing any sort of frontier edge AI research? the most people I'm talking to is like, I'm doing this like, and then like biology, like research stuff, whatever. And they're like, ⁓ I'm trying to run this new kind of a model. And it's not really running well because I'm doing this crazy thing. Could you please like, can I try it on your thing? Like people who really need the versatility to be able to scale their models. So most of the like the AI researchers.

Nihal

Mm.

Elias· Zettascale

everyone's doing all these sort of like toy problems and what the missing key part is being able to scale the model to such a like a scale of literal like volume scale in parameters but to a scale where it's actually useful. like chat GPT started off at like GPT like that sort of something like can I can honestly useless but then sort of like someone figured out how to scale it on GPUs and then wow cool now we have

Nihal

Yeah

Elias· Zettascale

a multi-billion dollar, like trillion dollar industry, believe, soon, from that direction alone. And there's, like, this is one direction. There's like hundreds to thousands of new, like, AI papers out there that's yet to be written, yet to be scaled. And those are the sort of people, like, we like partnering with. So yeah, we basically have a list of, like, everyone who's, like, want to test it out.

Nihal

Mm.

Elias· Zettascale

We have our own cluster. We call it the monolith from 2001, that movie, with our prototypes in it. So what we usually do is send us your SSH pop key and just try it.

Nihal

Have you ever said, we are not a good match? Like, folks reach out to you or you had the conversation and then you realize this is gonna be more like a, loving us rather than contributing.

Elias· Zettascale

Yeah, there's been some people like, I'm training this Frontier SLM model. And it's like, yeah, I guess that's cool, but we really want you to be weird and try to do something novel and new. In reality, we do have limited capacity. We have one server rack and we have limited power. So we can't serve a hundred people at the same time until we build our data center, spoiler alert, but.

Nihal

Hmm. you

Elias· Zettascale

That is not to me like, we have to be bit picky. It's... Yeah.

Nihal

Can you walk us through what exactly, I know you can't reveal everything, but at least if I am today, let's simulate the environment that we are having a conversation, and then you ask me to give it a chance to the scale. Why would I do this? Why would I leave my setup? I have my frustrations, of course, but then there is the over the switch costs. What is the problem that you give me? And also what is the reasoning behind your product that is gonna step by step just get better?

Elias· Zettascale

Yeah. Yeah. Okay, cool. So if you're a researcher, you're doing research, you're writing code in like Jax or PyTorch or whatever public ML framework. Torch is another conversation, but there's Torch MLIR and stuff, we do support it currently, but you would essentially just take your code and install our drivers and that would be it. All you have to do is click on the download button and then maybe the install button or like just do like whatever distro you're on on Linux like suda apt install on xp drivers and that's it. So for the switching cost of your software like when I spoke with OpenAI with this like they had the switching cost is like not like oh can we swap out the hardware and then play with it. It's a huge huge like investment for any AI lab to just rewrite all of the code.

Nihal

Mm.

Elias· Zettascale

which is also why we see all of these other chip startups like Grok, TenSore, and who have their own custom software stack that they're trying to get everyone to use. But that's the actual major, major investment for these people. That's where the friction is coming from. As I said before, if you have the world's worst chip in the world and kind of okay software, then that's gonna sell more than having the best chip in the world with bad software.

Nihal

Mm. That's right. Yeah, yeah. I think that was a very pretty good example to demonstrate. And how did you end up with your numbers? Because right now you're clearly saying 26 dots. Seven, was that? Like 27x, let's say.

Elias· Zettascale

Okay, those benchmarks are old. We have new benchmarks very soon and we're just benchmarking like literally every model we can get our hands on. ⁓

Nihal

Mm. How do you, like, because benchmarks, there are benchmarks, there are benchmarks. Like, how do you, what is the overall steps that you do?

Elias· Zettascale

For the old one, that was a power simulation of like, what is the maximum flops we can produce on our chip and what is the maximum power drain. That's a very simple thing. It doesn't mean that those computations are useful, but it's the maximum peak of the old chip. What the benchmark we're doing now, with our compiler, that's also using MLIR, we have a profiler. So if you're running like a transformer model, for example, we can see like, what is the speed up of X, Y, and Z. And that will always be dependent on how much bandwidth you essentially have like your effective bandwidth. And then what we actually do with our chips, because they are reconfigurable, not in a spatial sense, as in like we don't take this node and this node and put them together. What we actually do is we have a different execution model. So when you compute your, your tensor, like a matrix, you should do a matmul and then you have like an only new activation. And then you sort of have to like move off the matmul results to some other circuit that's doing the your activation, whatever. But there is a way to fuse them together to essentially just be in one go. And the less times you have to read from memory and the less time you have to write from memory and those intermediate steps, the faster your chip becomes, the more throughput you get and the more energy efficient you become. So that is what we're benchmarking on like a transformer. Like how much fusion can we do in a transformer given these constraints? And then it's like a bunch of like information dependence. constraints which limits your fusion, but we have some cool neat tricks around those as well.

Nihal

Tell me more about it. What sort of trade-offs do you take then? Because you can't maximize on everything.

Elias· Zettascale

It's basically higher latency. That's it. But if you're doing like large like training runs and even large inference with a big batch size, that doesn't really matter. But if you're doing stuff on the edge, if you want to have like a... I don't know, running your whatever model on the edge for like a drone or something or on your phone, then the high latency is not really good for that. But as also notice, that's not really the workloads we care about because we care more about the actual computational things that will lead us to ASI or whatever you want to call it.

Nihal

You mean the spray intelligence? Yeah. What is the definition of intelligence for you?

Elias· Zettascale

being able to adapt and getting what you want. I think like... For humans, if you get the AI to get what they want, then I know that's not do that actually, but for humans at least, I would think it would be like there's a lot of very smart people who aren't intelligent enough to sort of achieve their goals and they sort of like burrow down in this one direction and he gets stuck and then they like obsess over details that doesn't really matter and then they waste time. And then they end up writing. 50 papers and they end up with a PhD and that's it versus someone who actually built something that's very useful for a of people and you move the needle for humanity. That is the major difference between the two.

Nihal

Are you following certain folks' papers, like, regularly, religiously, like, whenever they publish certain things and then you get your hands on it?

Elias· Zettascale

I wouldn't say it's specific people, but within the company, we're just like, oh, we see a cool paper and we send it in this channel. And everyone's like, oh, this is kind of interesting. So we just have this like, oh, if something cool appears, like, cool. I don't really believe in that this magical figure will produce the best AI papers. And in most cases, it's the people you least, you expect the least of, like the misfits who actually come up with these revolutionary things. It's not like the big famous AI guy, unless, I don't know, they can still do it, but it's always going to be someone unrecognized because they have something to prove.

Nihal

Mm. Yeah, when it comes to facts and truth, like it doesn't come with any sort of chip on your shoulder. Like if you got it, then you reveal it to the world. Totally agree. At the same time, like there are certain people, for example, it's always, I'm always curious to take a look like, what did they do? If I come across with it, then my first instruction is not bookmark go. okay. Let me just quickly take a look at it. That was more the intention of my question.

Elias· Zettascale

Yeah. okay. yeah, when I say papers, I just read it, because if I have a stack of papers, I'm never gonna read that.

Nihal

No.

Elias· Zettascale

It's like if something takes you like five, ten minutes to do, should do it straight away. I was just gonna forget it because you wouldn't care as much about it.

Nihal

Yeah, but actually when you speak about your Slack channel and the team, right now you guys are four, right? Like, at the moment. Yeah, four people. Five people. And how do you see the pace?

Elias· Zettascale

We're five people. Five and a half. Yeah.

Nihal

How do you see your milestones? Like you set your goals milestones and how do you see all the building team at the same time and building the product in parallel together?

Elias· Zettascale

It's not that we build it in parallel, it's that we build it... So one major thing we haven't said is like we don't have roles. I think roles are the most cringe thing ever. We don't have specialties. We're expecting everyone to basically learn everything. Like the whole stack. If you join in and you know this area really well, then yeah, you should obviously work on that. But you should also learn the abstraction layer above you and the dissection layer below you. Because if you don't, everyone's just gonna get very like compartmentalized and...

Nihal

Yeah.

Elias· Zettascale

going to be aligned at all and that's just bad. how I structure my team is we have it all the way up to the AI researcher, all the way down to the silicon and that's really important. It's sort of like what Bell Labs did back in the day. Like if you sit next to the guy or like the person in either of those abstraction layers and you do something really like a design choice that's like really stupid. They can like look at you and like why are you making my life hard, mate?

Nihal

Hmm. Yeah. ⁓

Elias· Zettascale

Please don't. And the reverse is as well. So if you do a really great design choice, they will cheer you on. They'll like, hell yeah. Having that communication is extremely important, which is also coming back to Google DeepMind, which is why I think they're great. But they're also lacking the business. So they'll have this huge corporation that's compartmentalized. So the only sort of way to have that is just have all those people in the same room sitting next to each other. And that's it.

Nihal

But then at the same time, how do you now set your goals? Because what you're tackling is not like how you say this. I'm automating finance departments of buildings, like buildings, or I am happy. Do know what I mean? It's not like a regular sauce. Therefore, that you have so many different pieces in the puzzle to always keep them together.

Elias· Zettascale

Yeah. Yeah. Yeah, there's always communication. if somewhere in the stack something's like falling off and you sort of have to fix it, then you feel the effect up at like the AI level because you don't do the thing. So really how we set goals is like, it's not like, okay, let's make this huge, perfect product in next month. It's more like, okay, what can we get the most done today? and we sort of like map out on an exponential curve, because you sort of have to lay the foundation first to have a very strong foundation, and then you sort of build upon it as you go. As the reverse is if you just like take huge steps and jump forward, you have to take huge steps back and jump forward, and it'll just take more time. So if you follow this exponential curve, and you basically just tell them, okay, just be very rigorous in the actual foundation of the product, if you work on a module for the chip, be very rigorous.

Nihal

Hmm

Elias· Zettascale

how's that going to affect the above stack? If you aren't, it sort of falls apart. So you sort of have to set the goals on an exponential curve, if that makes sense.

Nihal

Are you getting any sort of, because sometimes when we are doing it on our own, I call it bubble, but I mean more in a team environment, then you always need some sort of external people, which are in this case, your customers, early customers are to burst your bubble. Like, how do you stress test the ideas before investing too much?

Elias· Zettascale

Yeah. It's, we have some sense of like what's needed for AI internally in the company because we have the, we're fully vertically integrated with AI all the way to Silicon. The only solution that I just talk to your customers, there's no magic thing, you just go talk to them. Like I go to random like SF functions and then like meet people there who's working on this new AI lab and it doesn't be really cool actually. And I just talk to them like, what are your major, what pains you with the current ecosystem that you're using?

Nihal

Mm. Hmm.

Elias· Zettascale

And most of the cases, you're like, this GPU shit, I can't scale my thing, whatever, whatever, whatever. And that's pretty much it. It's that simple.

Nihal

And it was my favorite answer I could ever hear. Are you in any partnerships with any sort of other companies where you get help?

Elias· Zettascale

But we have manufacturing partnerships. So like with TSMC, GSME, we're also producing our PCBs here in the Bay Area with Sierra Circuits. We're trying to keep all of the manufacturing here in the US. That's pretty much it, yeah.

Nihal

I recently I reflected on the fact that when it comes to manufacturing part of the let's say chain and even that there is this part certain companies have certain priorities because of their capital and dominance in the space like it how do you see tackling this piece because in the end of today it's not like you know I go and immediately they give me what I want

Elias· Zettascale

No, okay, so one thing I've learned is, Star Wars and B.R.I.B. very much internally and externally a people problem. That's something you can't escape. So even with like the manufacturing and like getting the supplies and components, you sort of have to like, it's very subjective for every single supplier. Some of them wants to talk over Zoom, some of them wants to meet in person, some of them wants to like chat over phone, or somebody just wants to email.

Nihal

Hmm.

Elias· Zettascale

It's very diverse of what they want you to do. You sort of have to navigate this maze and then you sort of, because you're startup, you have to convince them that like, I'm important. Otherwise, they're just gonna go to like this huge company is gonna pay them a lot of money. like, know, most more more probably not, you get like the the worst team on their end, that's gonna deal with you. And then you sort of like have to like fight off and like just fix everything. So you sort of have to.

Nihal

Mm.

Elias· Zettascale

I think the word I keep using is like everyone in the team is trying to talk with suppliers and stuff in parallel. I tell them to be annoying as much as annoying as like... be annoying. If you're not feeling annoying then you're sort of like not putting in the effort, right?

Nihal

Hmm. I think you tried to end it in a good way for your own result. ⁓

Elias· Zettascale

Okay, we don't try to annoy them, but it's more like you have to follow up to an extent where you think you're being annoying.

Nihal

Mm-hmm.

Elias· Zettascale

So yeah, that was it. We had this a few weeks ago, this one supplier just wouldn't reply to our emails or they wouldn't answer the phone. So we just had this idea, like, okay, let's take an Uber down to your office and chat in the lobby. coincidentally, right when we said that, they responded, so we didn't have to do that on the screen. But if it requires you to go down to their office and actually speak to the key people that are there,

Nihal

Yeah.

Elias· Zettascale

Just do it. I would also avoid getting a restraining order. I don't think that would be productive. There are limits, of course, but you sort of have to keep knocking on their door. Because people, if they don't care, they're going to forget. And that's just a human factor.

Nihal

I think one of the stories that I remember from Shapo from Entangle, from your badge, and then he was talking about, he was sending cakes to the person, just all of them, and he was like, just stop sending me cakes. I'm going to respond what you want. guess, or even they went one step ahead and then they put a billboard in front of the company. In a way, like, will you answer us? ⁓ I mean, that's the spirit.

Elias· Zettascale

Yeah. You That's funny, that's funny.

Nihal

But these are like the conversations where you see how much of care is in there. Like nothing else would make you do this. Like I don't believe in monetary motivation when someone tells you such stories. In your case, why would you care? Yeah, why would you care? Like why would you care so much about this specific?

Elias· Zettascale

Yeah, he's a great guy. Like me?

Nihal

Yeah, me as a person. It's one thing to be interested in and you know that you have a talent, you have an eye for it, but another thing is you really care so much and you move it forward and try to make something that can change people's lives.

Elias· Zettascale

Yeah, ⁓ Wait- Like talking with a supplier or?

Nihal

Hmm?

Elias· Zettascale

like talking with suppliers and manufacturing people.

Nihal

No, no, no, like overall for you, Alias, like why, why do you really care overall so much? Like, you know, you could have just come to it, like you could have still tinker in so many different ways. It doesn't have to be, yeah, this structure.

Elias· Zettascale

overall okay Yeah, I think it's your part. The first part is like my own personality. I'm extremely obsessive. So I think that that's a major part, but the main part is like I am like I can see that I one of the main hiring questions I ask is what do think the word is going to look like in 10 years? And what do you want it to look like in 10 years? Or are those the same? And if you answer like I think it's gonna look like something horrible.

Nihal

Hmm

Elias· Zettascale

And then you answer, okay, what do want it to look like? Not horrible. And then, okay, you sort of like are lacking the point. But my point is like, I envision this future in 10 years, where we basically have this abundance thanks to AI, and the only scarcity that's really left is this like scientific discovery part. And for me to care about this, it's more like I really just want to build a future for all of the people I really love, and all of the people I care about, because they deserve it. Like that's it, like, okay sure I can be like, shit I wanna become a billionaire. Okay, why don't you just go work in finance? Like what's the point of doing this then? Like the point is actually changing, like putting a dent in the universe so you can make the universe a better place for the people that you love. And if you're not really doing that, you don't have to do it in tech, you don't have to do it in whatever, whatever. That's really the point of everything, right? Just leave the things as you found them in a better place. That sounds like very cliche, but that's just the truth.

Nihal

But also, among all the stories and novel motivation behind everything you do, there must be some moments that it got really tough too, right? Like did you guys have this near-death moment and also doubting and trying to see, okay, like you just need to get over this and then probably it's gonna be better. Like let's talk a little bit about those moments.

Elias· Zettascale

Yes. I mean the only real failure is when you die. Right? Everything that keeps you, like if we do a mistake and whatever, as long as we don't die it doesn't really matter, as long as we learn from it and move forward. There's gonna be periods where you feel like this gut wrenching pain of anxiety, like you really like, it's extremely comfortable but at the start it's like sort of less and then as you progress forward it's like

Nihal

Mm.

Elias· Zettascale

You don't really get used to it, but you get more tolerant to it and you become more and more resilient. It's like from my past, hardship trains the faithful. It's not like I magically spawned and everything was fine and then I magically appeared in this position. It's just constant suffering. That is what it's like when you build a startup or when you build a company. It's just constant suffering. There's no glamour to it. And to keep going when things are tough is like...

Nihal

Mm. Hmm.

Elias· Zettascale

Like, I don't, giving up is cringe. I don't know why people do that. Like, it's fundamentally like, don't get it. Why would you do that?

Nihal

In context it changes, right? Like for some people, when they give up, they find the next thing that is the right match for them. Some they don't give up and then they persist on the thing. I always say like sometimes you don't know what you do, but you at least know what you don't want and then you start experiment. And if you drop one experiment for a better experiment,

Elias· Zettascale

Yeah.

Nihal

Actually, it's still not really quite giving up, but it's a definition of giving up, right? Like how would you define giving up?

Elias· Zettascale

It's- if you're pivoting, then it's like you're pivoting for a reason to find something better. ⁓ But, like, for me at least, like, I have this vision of the future, like, this is something I really need to work on. It's not like, I have this my own personal mission that I want to fulfill. Like, I know my own purpose. I know I need to do this. This is why I exist. So, like, for me, like, giving up is not an option. I don't see why I would do that. Or anymo-

Nihal

Yeah.

Elias· Zettascale

And that's also me being like kinda failing to see other people's perspective because I've always been like this. I've always had this mission in my head that I want to improve the world in the way I think it, in a way that I think is good. But the semantics between of getting from A to B doesn't matter. So if you start, like with our chips, we started doing spatial reconfigurability.

Nihal

Mm.

Elias· Zettascale

Over time, you sort of realize that that was not a good approach to what we're trying to do. We then realize that that makes two be of a chip. You can't make many of them and you end up as like three person something. Make one chip a year and then you can sell them. So it's not about like the definitions of giving up. It's like you give up. I'm not gonna do this anymore, et cetera, et cetera. But what you really should be aiming for is like, okay, if something's not working. figure out what's not working and then basically evolve from there, like build on top of it, like remove the things that's not working and then build. That's like the core thing of how you keep going forward. And like pivoting for like SaaS companies, guess, it's like, ⁓ we have a million ARR and we were sort of stuck in this growth, like we're plateauing. Who am I to tell you to pivot? But if you're not reached... 10 million, 100 million, you gotta do something,

Nihal

More or less. Then I have an assumption that you look into this resilience factor a lot in your co-founder probably as well. How did you guys meet? How did you had this click?

Elias· Zettascale

Yes. That was a funny story. I was still in university. This VC invited both of us and a few other people to like a conference thing for a week in London. It was at a girls boarding school that they rented out. So we just met there and then we like just started talking about like the future we really want to build and what's really needed. And I remember my co-founder's name is Prithvi and I remember like meeting him and the first thing you said like there's something here there's like something like on the tip of her tongue the first thing and we just like took a walk around the the compound basically it was like good nature like good whatever quite a beautiful place to be honest and we sort of sort of talking about that and then we were like wait this this is like yeah we should do this and then that's when i met him and then i was on the uk then i fly back and then A few miles after that, we go to the YC. Yeah, yeah, no, actually we used to call it like co-founders at the first site.

Nihal

Cool. ⁓

Elias· Zettascale

Because we're we just like... We both got our own visions. I'm basically the computer architect and he's basically the AI researcher. This duo doesn't really exist, but just typically the AI researchers hate the hardware people.

Nihal

But this is perfect. But this is the perfect combination for what you're doing because you need to have the vision and understanding, right?

Elias· Zettascale

Exactly. And that's how we model the company. We're keeping this dyad between the hardware people and the AI and software people. It's something you gotta have. You can't remove some abstraction layer and then hope that your customers will love you. You sort of have to... This is the communication part. This is why Bell Labs was so good back in the day. It's something we're emulating. You can't just be this tunnel vision on this one thing. And that's why all of these other chip companies are sort of I don't know what the hell's happening with them,

Nihal

I realized that I am really enjoying the conversation, but probably Cameron is also waiting for us to prepare more questions

Cameron

⁓ Firstly, thank you, Tom, for the conversation. was really interesting to just kind of, you know, sit there and absorb and learn, to be honest.

Elias· Zettascale

Yeah, thanks for having me.

Cameron

And I like the... The fact that everything tied back to making the world a better place for the people you love in the future that you want to see. I thought that was...

Elias· Zettascale

Yeah. Have you guys seen the TV series called Silicon Valley, where everyone just keeps saying, like, we're making the world a better place, and then it's just some tech bullshit? ⁓ Yeah, I kind of see the irony, like, yeah, we're tech bros. I don't call myself a tech bro, I'm a deep tech bro. I do deep tech shit. Bye.

Nihal

you Yeah.

Cameron

There's some merchandise in that I'm sure somewhere. Yeah, absolutely. I always enjoyed that and they ended up just creeping like slowly steadily back to like the standard server rack after trying to like, know, create something so different.

Elias· Zettascale

Yeah, yeah, yeah. It's like we weren't making like Bits soup, we making the world a place. Probably. Sure.

Cameron

Yeah.

Nihal

you

Cameron

Nice. So what I'll do is I'll run through a of the questions that were either sent through beforehand or that have come through on the, on the chat. So we have one from Axel and his question was, what's your thought on in-memory compute like neuromorphics? Is it something that is interesting to explore? Just wanted to get your opinion on that area.

Elias· Zettascale

It's, I think, in memory compute, like everyone's doing that now, like everyone's spamming S from everywhere. The problem is you have this memory or like memory bandwidth inflation, you sort of not thinking about how you do the computation. So the key part of our philosophy is like if you can fuse A and B together to sort of reduce the data movement, you sort of have this compounding effect as well as you scale your memory bandwidth.

Cameron

Mm-hmm.

Elias· Zettascale

It sort of becomes an exponential plot rather than a linear increase over time. Doing a memory compute, like it has its advantages, it has its disadvantages, like you end up with a very big chip and then your yields are really bad. I, it worked for Grok, I guess, and they made a cool, cool demo. But apart from that, if you need versatility, like reconfigurability and versatility is quite the, the most key factor here.

Cameron

Okay, perfect. And then I think one of the topics that has come across, obviously, with a lot of the previous founders that we've had on is how did your experience at YC shape you? ⁓ I think if I listen to kind some of the parts one things which stuck out, seems to be maybe to me a holdover from YC and other people's comments is just get in the room, just be

Elias· Zettascale

Yeah.

Cameron

next to the people that you need to be at because at YC you've got everyone and you've got this intense network and kind of competition of the people around you, obviously, which may not be the same within the company, but just this idea of getting the right people in the room so they can be having that short feedback loop. yeah, if you could expand on that. Great.

Elias· Zettascale

Yeah. I mean, and while you see like my personal experience, like I came from a university from the small town where I'm sort of like the only person who really cared about tech startups. Like you sort of find your, your, your, your flock, I guess, or whatever it's called. Like when I got there, I felt like a huge relief, like a huge weight to lift it from my shoulders that I had to like conform and I could just be myself. And it's like true for, for Silicon Valley and stuff in general, you just be yourself and

Cameron

Hmm. Yeah.

Elias· Zettascale

Like people will share you on it's like a magical place where that's like that's happening. It's insane but like even like NYC like like Chappell and Philip and Brandon Like you're always seeing them making making progress and then when you're doing something bad and you're like lacking like performance you sort of see that directly and sort of like shit I have to catch up and I have to like double down and like having like very like

Cameron

I'm

Elias· Zettascale

high quality founders around you is really important because you are the average of the people around you, right?

Cameron

Yeah, no, I completely agree. And yeah, an inspiring group of people for sure. think Philip seems to be the one that's kind of driving that, at least at the moment, he's got the headlines. we'll put a yeah, he's doing it. So far, I'm sure it'll ebb and flow.

Elias· Zettascale

So far, so far, yeah. Well, we tried to not make as much like PR stuff because we were very secret. We're gonna have a like huge surprise launch and shit, but yeah. We'll see.

Cameron

need to put one of your chips up on you know

Elias· Zettascale

I'm gonna do the holding the chip like this on stage and everything.

Cameron

Yeah, absolutely. And then I have a question from Olga that's just come through. And she said, from your perspective, is there one small thing that we could implement in our daily work life to make the atmosphere in Europe a bit more pleasant for investors and creative people that don't maybe have access to the funds? I think tying back to maybe some of the comments you had at the beginning around I guess one of the benefits of people coming from Sweden, as founders, is they've had to go through that kind of hell of the initial phase.

Elias· Zettascale

Yeah. Like Europe has the infrastructure, they have the capital. I think the problem is a bit of the bashing on farmers and entrepreneurs that they're bad. There's this huge stigma of being an entrepreneur and then ⁓ if you're successful, you're not cheered upon. People want praise, yes, in Silicon Valley. You're always trying to the, martyr yourself down. I think the work is like, accepting of people as being different. Not everyone wants to conform to the people around them. If Europe, it's a cultural problem, basically. If they let people...

Cameron

Mm-hmm.

Elias· Zettascale

grow themselves, be a bit egotistical, but still do it for the good of the people. Let them run if they want to run. Don't start putting in words like, if you're to run, you're going to fail. Don't diminish things. There's no point. Encourage people to do the things they want to do. That was very much it.

Cameron

Mm. Yeah, don't diminish the flame, I guess. Perfect. And I guess, just to tie back to the start of the question, is there one thing that you think people can implement into their day or is it a question of, I guess, ear into your difference?

Elias· Zettascale

Well you can't change what other people do, yeah. You can't change what other people do, that's... If you could, that would be mind control and I think that would be against the Geneva Convention, but... What you can do yourself is like, if you see someone being really successful and you're like, you should cheer them on. Even if someone's like struggling, like, cheer them on. And it's that simple.

Cameron

Perfect. And I'm just seeing what other questions we have coming through. I think there was a comment which was pushing back on the hardware innovation that maybe Shenzhen's a place to add to the, there's an area of innovation.

Elias· Zettascale

Yeah, I haven't been to Shenzhen. I might visit one day.

Cameron

And I guess we might have touched on it at the start, but in case there someone who joined just a little bit late, you maybe can revisit it, is what was the trigger that kind of made you decide that you wanted to drop out? And I guess what gave you the self-confident?

Elias· Zettascale

It wasn't a trigger. I told my parents before I started university that I was gonna be in university for like one year and I was gonna drop out and make my own company. That became like 1.8 years. So it was not according to plan, but it's more like... When I got there, was like, okay, actually when I actually got there, I was like, okay, cool. This is kind of easy because I didn't really struggle and then I asked like the student...

Cameron

Man of your word.

Elias· Zettascale

They're called SOOVS in Sweden. I don't know what called in English. But I asked them, could I do my bachelor at the same time as my master? I tried to double my coursework. And they straight up told me, no, you don't have enough points. And Sweden has this point system. like, you have to wait until this course is here, and then you have to wait until the exam is in two months. And it's like, no, I don't have time for this. So what I did is just ended up going to student groups and then a PhD group, making drones and making rockets.

Cameron

Okay.

Elias· Zettascale

And that's basically what was my entire time. And then I was like, okay, cool. I'm sort of wasting time here. I'm not happy. I should just drop out. I read the essay, not at that point, but by Sam Altman that not taking risk is riskier than taking risk. And that was like boggling in my head. I was like, okay, cool.

Cameron

do the thing you wanna do and do it now. Mm.

Elias· Zettascale

There's no point in staying in the same place. You have to keep taking risks otherwise you won't get anywhere.

Cameron

Yeah, I mean, I feel like just through my own experience, that seems to be the biggest. Obviously, Silicon Valley is kind of like this to the extreme, but the US versus non US cultural shift and risk appetite seems to be the thing that always stands out to me.

Elias· Zettascale

Yeah. Yeah, the Europe thing seems to be more, well, everyone's just more risk averse, which I, we're just sort of embarked into the culture, I guess, that's something like be comfortable with taking risk. Just keep rolling to die and eventually you'll roll the thing you want to roll and you'll have success.

Cameron

Okay. So, gotta fail, gotta learn. And we have a question who's come in from Rasmus. So he said he's a master's student at Chalmers University and he's seeing energy and efficiency as the critical bottleneck for scaling large AI models. So his question is while optimizing hardware architecture is a clear solution, adoption is often the hardest part. How do you plan to integrate your technology seamlessly into the dominant Nvidia ecosystem? If you do. ⁓ without disrupting ongoing AI training and development pipelines. No, no, you referenced Nvidia earlier on, so I'll link

Elias· Zettascale

It's not overnight thing. We'll start with the researchers, the people doing the actual frontier edge research. ⁓ Those will be the people most willing to change things. When it comes to CUDA and having this entire ecosystem already exist, people are already moving off that to an open source standard, like using MLIR, for example, as a backend. There's this massive incentive structure to move off CUDA.

Cameron

Mm-hmm.

Elias· Zettascale

Simply like open AI already would move to AMD like the and thought bigs that you're suiting Nvidia cars AMD cars you're using everything they can get our hands on So the friction is like the software layer really we're not trying to reinvent the software layer We're just following the direction if if you want to make the argument, okay could also win because they have like Nvidia has like a bajillion software engineers working on it. Okay, sure So does sort of Microsoft and look where you know when the server ended up is it Linux one and in the end? Like if you're making a product for developers, you sort of have to have it be open source. You sort of have to have it be like this massive non-centralized thing where everyone's contributing to it. It's been proven in history that's the most optimal model to do things.

Cameron

Perfect. So for now, that is the majority of the questions that came through from the group that were able to join us today. So what I'll do is I will hand you back to Nihal. She'll probably go through some closing remarks and I know she always has a question that will pop up in her head anyway, so you'll probably get a few more. But it's been a pleasure from my end. I'll probably pop up at the very end just to say goodbye to everyone. But it's been an absolute pleasure.

Nihal

Yeah.

Cameron

I guess, and I'll leave you back with Neho.

Elias· Zettascale

Also by the way, cute is not a mod. When I posted that on Twitter, where Ike's people got really mad. It always funny.

Cameron

Thank you. Yeah.

Nihal

Thank you, Cameron. I have a couple of questions that I realized, actually that'll be really cool to ask him. Because one thing that always make me think was if you take a look back at the history of AI, when does this date back to 1950s? The very original idea, like the exactly something tangible.

Elias· Zettascale

Yeah, it goes back way, way back.

Nihal

Look how many decades we're talking about if we had the chance to go back to the know time machine Single couple of things that they missed that's why we ended up today Requestioning all the way that we run all this Models how we you know what I mean because you're not able to match the need versus the foundation that we need to provide Like, what do you think? Why did we miss this forks? Like, you know, this like the paths that we make decisions and then change forks.

Elias· Zettascale

It's mostly just infrastructure, right? So like all of the directions everyone's going in, so we have this huge hardware bias as it's called in AI research. You sort of have to conform with the hardware is capable of. And then back in the day, what existed was Nvidia, know, graphics cards. And then they figured out how we can do like, match models really well. Cool. Let's go in that direction. And Magmals and Covolution and all those sort things are still going to be extremely important in the field of AI but there's also a lot of more things we need to not support but account for I So if we go back in time basically, it's not like we can't really tell them to do something because they already know what they're trying to do is one of a mathematical problem and they probably must be like they've solved it really. We have all these Cambrian explosion models. The problem is not that they don't exist. The problem is that you can't really scale them because of the hardware bias. So if we can also teleport back in time, I'm sure we can figure out a way to give them the state of the art XPUs, art chips by the way, as you call it. Then they could probably scale their things much, much more earlier than the present. So who knows if it's, if you go back a few decades like.

Nihal

Mm. Mm-hmm.

Elias· Zettascale

what would be the societal impact of dropping this state of art technology today on them, I don't know. And I don't think we should mess with timelines because time travel is wibbly wobbly, I guess. But if we did that, then we'd accelerate that timeline quite drastically because it's exponential. So I'm not even going to guess what it's going to look like.

Nihal

Because I sometimes think the markets or the society have a tendency to prioritize economics over physics, you know, that the longer term benefits. And that's most of the times due to causes of all these lock-ins or all this like persisting on something. But then we don't stop and question and say, actually, this can be done better.

Elias· Zettascale

Yeah. The thing is with the market is, like the free market like, economic experience, it's chaotic machine. It won't always be perfect. Sometimes it overshoots, it will sometimes undershoot, but at the end of the day, the market always corrects. So when we show off with our chips and then people can use them to scale whatever models they're trying to scale, that won't be an overnight thing. It won't be like, ⁓ instantly $20 trillion on our bank accounts. It will be like maybe a decade thing.

Nihal

Mm.

Elias· Zettascale

So the market sort of, know, of them are plateauing. The market will self-correct. It might be a bubble burst, but it won't really be a bubble burst. But people will keep working on AI technology. We've already seen what's possible, and I don't think this is going to be a massive bubble burst because of all of the downstream economic value that we already are getting from these models. But it's basically going to be another exponential of this market growth.

Nihal

And if zeta scale fails, let's say in the future, like what would be the root cause you think? Or what would be the biggest assumption that cause it?

Elias· Zettascale

I don't know, terrible happens. I get hit by a bus or something, don't know.

Nihal

No.

Elias· Zettascale

Or like our chips randomly explode because of physics, which is quite improbable. Like, if it never existed in the first place. Nvidia would become the next Intel. the market would still converge. People would have taken longer to realize that we sort of need these more diverse models because different models are better at different things. The market adjusts, the incentives are still there, but when it comes to other dying out, that's never gonna happen.

Nihal

I kinda sensed that you would end it with a sentence like similar to that

Elias· Zettascale

Like, it's a non-zero probability that I get eaten by a shark.

Nihal

Like out of your control, forces out of your control. If you like physically disappear, that will be the end of the scale. I think it's a perfect way of looking at it. You start to bring it to the... There's even not a finish line, but you start to put your impact in the world, not to, okay, I'm out now. Like, awesome. Actually, I'm gonna invite Cameron because we have a tradition.

Elias· Zettascale

Yes.

Nihal

and it's time to do a little tradition. We asked our previous guest to pose a question for the upcoming guests. And in this situation, you had a question from Ashie. Ashie is one of the co-founders of Spatium. I'm surprised you guys know each other, like for context. Very smaller than we think.

Elias· Zettascale

⁓ No, we all know each other. Silicon Valley is a small place.

Nihal

So, the next question for you is the following. As a CEO, how do you juggle between all the tasks and still stay sane?

Elias· Zettascale

Stay sane, okay. At Setup, we have this thing called full autonomy. So I don't give a shit how much you work or when you work, as long as you get stuff done. And it's up to you to not burn out. I think for me, it's just having a nice routine of waking up, even though my sleep schedule might not be perfect, and going to the gym, exercising, keep eating healthy, and... Yeah, that's it. I might try fencing, that might be fun. I'd to let them some frustration, but...

Nihal

Dancing.

Elias· Zettascale

I did that, it's my partner. That's it.

Cameron

Yeah, my partner got me into climbing, so I understand the whole kind of getting into their hobby.

Nihal

Have you tried?

Elias· Zettascale

Yeah. I mean, Having a partner is very important. I'm to support you.

Cameron

Absolutely. Keep you grounded. Tell you when you're being an idiot sometimes.

Nihal

It-

Elias· Zettascale

Exactly.

Nihal

Then probably also is the only person you call up right when you feel like my gosh did you hit the wall? Let's just say you're not. You go to person. Yeah, very happy that you found your partner in crime. Yeah, amazing. Do you have any sort of thing that you wanted to say but we didn't happen to ask you?

Elias· Zettascale

Yes.

Cameron

Yeah. No.

Elias· Zettascale

Yeah. ⁓

Nihal

perhaps for Deep Tech founders or anyone who wants to achieve something meaningful.

Elias· Zettascale

I can't think of anything, but a cool thing to ask is like, a common question I get is like, okay, don't you need like billions of dollars to do this thing? Because chips are expensive, right? No. Okay, so I'm gonna leave with a few contrarian takes. CUDA is not a moat, that's a massive Psi-op made by Nvidia to increase their stock. LLMs are not AGI, they are plateauing. and we sort of need to keep doing AI research to reach AGI and ASI. And the notion of that you need a lot of capital to build hardware is fundamentally stupid. And if you do do that, then you're just having a huge inflated round and you raise a billion dollars and then you're just gonna waste all them.

Nihal

you

Cameron

I think we got some YouTube shorts out of that now.

Nihal

And boom. Yes.

Elias· Zettascale

I'm excited to read the comments. I'm very excited to read the comments now.

Nihal

I think you have this a lot. Thank you. But Cameron and I, we both believe that if you want to achieve something, even if it is the chips, one of the most critical technology right now from, or watch the fridge to cars, like everywhere, you just go for it and then try to be resourceful about it. And especially right now, building is not the problem, like, right? Like,

Cameron

You

Nihal

to make decisions around what's going to make a difference is the differentiation and the courage of yours, like people like you. Yeah.

Cameron

Yeah, I think it's the conviction, right? The barrier to entry has been lowered in so many areas, it's having the conviction to actually move forward with it, least from my opinion to a degree.

Elias· Zettascale

I just like building things.

Nihal

And.

Cameron

That's

Nihal

Yeah, exactly. Thank you so much for being with us today and sharing your experience, your insights, your life story. I have a feeling you're going to knock on your door again in the future to see how you're doing and how

Elias· Zettascale

Thank you for having me. Yeah. Combine the office, check out the cluster as well.

Cameron

Absolutely.

Nihal

we for sure. So thank you. ⁓

Cameron

Perfect. And then,

Elias· Zettascale

Yeah, thank you for having me.

Cameron

this is for the...

Elias· Zettascale

conversation.

Cameron

Absolutely, thank you very much.

Nihal

Thank you.

Scroll to bottom

Pull up a chair.

Learn out loud with us.