Fabricated Knowledge

This transcript is lightly edited for readability. Doug: Welcome to Fabricated Knowledge. This is a podcast edition today, and I have the special honor of having Val Bercovici from Weka. And today, we wanted to specifically highlight an important trend I think everyone should be paying attention to. Disaggregated PD. I also wanted Val the opportunity to tell the story of Weka and AI-enabled storage today anyway. So take it away Val. Val Bercovici: Thanks, Doug. So, Val Berkevich, Valentin, for those who did sometimes see my name online, a long-time infrastructure guy focused on storage for most of my career. I ended up being the CTO at NetApp about 10 years ago, following a cloud acquisition they made called SolidFire.At that time, I was also actively involved with Google in open-sourcing their Kubernetes project. That was cool; it helped create and co-create the Cloud Native Compute Foundation (CNCF) under the Linux Foundation. I enjoyed category creation, and I was relatively early in the cloud, early in machine learning, and not so much in general AI, but I'm catching up on general AI right now. And yes, I'm at Weka right now, focusing as Chief AI Officer on our AI product strategy and marketing and education. Because, as we're going to talk about Doug, there are a lot of very new and different things happening about infrastructure when you look at AI workloads. It bears almost no resemblance to traditional CPU-based data centers, and that's what we're going to dive into right now.Doug: Yeah, so maybe to start, I do think that the important thing is context, right? Things are changing quite rapidly all the time. Let's walk through how inference in a single GPU node was done and what has changed recently to shake up the entire ecosystem with this disaggregated pre-fill / decode. I think a listener needs to understand what pre-fill decode is on the traditional node, and then we can discuss what's changing by splitting the two.Val Bercovici: Sure, and let's maybe introduce some overall financial context here and Semi-Analysis is leading this. But estimates are now that about four out of every five dollars of AI is gonna go to inference. So inference is not a trivial thing here; it's a very, very material thing in the future. One of the reasons why inference matters so much is that LLMs, in general, and large reasoning models, in particular, are concerned with this aspect.LRMs fundamentally use inference-like techniques as part of their test-time computation, which is scaling now in the inference dimension far more than in the legacy pre-training dimension. So, all of these kinds of workloads matter a lot in terms of how infrastructure is allocated and spent, as well as the revenues collected from it. So specifically on Disag Prefill, which is a relatively new phenomenon. It's mostly a 2025 thing. Some research papers obviously were written well before last year, and last year on it. And it's a way to scale the entire process of inference. And this is specifically to be technical, not for image models, which are classically or video models, known as diffusion generative AI models, but the more popular text-based, if you will, and video-based as well, transformer models. So this is a crucial qualifier. But having said that, transformer models are really where a lot of the innovation is happening and new features are happening right now, and a lot of the focus is on engineering and efficiency.So, Disagg Pre-Fill / Decode, to open up the topic, one of the simpler ways to start is to consider the following: if you're not necessarily a layperson, but you're not particularly AI savvy, and you've just been in tech for a while, think of zip files. Please consider the process of archiving data, then compressing and decompressing it. So a lot of AI discussions focus on encoding. That is optimizing the training of models and getting that loss function down and getting this fit right between underfit and overfit. That's basically what it takes to compress the internet, as we like to say in the industry semantically, right? That's fundamentally what a lot of AI models are, compressing some fraction, or if you're OpenAI, most of the world's knowledge online into a multi-gigabyte file that gets loaded into these GPUs. So the encoding part is the compressing part, and inference is all about the decoding. It's all about how you decompress that model and how you do that by serving the model hundreds and thousands of times across hundreds and thousands of GPUs and how you decompress that model for every user. So every time you and I sit down at a ChatGPT session, or more often now, how an agent consumes these AI models on your behalf if you're running Cursor or other popular AI apps, the Manus AI agent, for example, how that decoding is happening. And that decoding now is really very different from the unzipping or unarchiving of old days, fundamentally because, if we were to use a zip example, that was a CPU and was traditionally a single core. So it was a single-threaded or single-instance process focused on compression. Therefore, it's a very serial process for compressing the data, and a similarly serial process for decompressing the data. Lots of those bottlenecks are with regard to just a CPU serially waiting on memory and storage. GPUs, again, maybe the main takeaway for everyone here is that nothing happens in serial on GPUs. The whole point of GPUs, the reason we spend an obscene amount of money on GPUs in AI data centers today, is that everything wonderful happens in parallel. That's the only reason, now, Gen.AI in particular exists is that it's a very parallel process run in these GPU kernels. And so the process of Encoding the data into a model and particularly as we're talking today the process of decoding the data from these models as we serve them at inference time is a very very parallel process and asking the way these models work asking the GPUs to do different things in parallel turns out to be incredibly inefficient.So, let me give you a tangible example of that, and we'll dive into the details of why the reality of the market is the way it is today. When you go to OpenAI's pricing page, Google's pricing page, Anthropic, Together AI, and other popular model providers, you'll find that they have three classes of pricing.One is the price per million input tokens. One is the price per million cached input tokens. And one is the price per output token. This caching tier, or rather, this caching class of pricing, also known as context caching or prefix caching, goes by multiple names and was introduced about six to nine months ago because people were discovering that it was very inefficient to reprocess the same data repeatedly. If you can process cache data, it is much more efficient. And we've seen OpenAI now join the pricing wars, along with Google and DeepSeek, setting the stage for a race to the bottom. Some great reports that SemiAnalysis wrote, you know, going back even two years ago. There's kind of an industry standard of about a 75 % discount to the user or to the API caller, whether it's a developer or an agent, if you use a cached input pricing option. But there's a giant caveat.That is, all of these providers, as they're inferring, will only give you about five to 15 minutes. They can't even specify beyond that, five to 15 minutes of prefix or content caching before you lose the context, you lose the cache. And so why is that?Let's dive a little bit deeper into how this decoding and inferencing work. The way it works is that you build a working memory. So as you're inferencing a model, as you're decoding the model, you actually can't jump into the decoding part, right? That's the valuable part. That's where those reasoning tokens pop up.Doug: I'll slightly interject here. I want to make sure that we have this, just because Val, you're going pretty quickly. It's a hard space. But specifically, let's discuss the prefill decode, right? Because we're talking about it, we're throwing these words around. On the prefill side, I think it's helpful to understand that that's compute-bound, and the decode side is memory-bound. And so traditionally, how this was done was all on one node, GPU was essentially, you know, this is a virtualization problem essentially. What we're talking about now is the shift to the understanding that the decode to in the, you know, the pre-fill, which is, I believe the input side of it versus the output tokens, the ratio is exploding so that you're talking 10 to 1, 20 to 1 of these tokens that are moving. And so what people are finding out is that decoding is becoming a big issue in inference scaling. And so, maybe that context is helpful, because I feel like we're diving in very quickly, and I wanted to make sure we...Val Bercovici: I was jumping ahead to illustrate the fact that most people skip the prefill, right? They skip this fundamental critical path item in parallel with how inferencing works overall because we're used to paying for input and output. So what is this prefill thing? Prefill is literally how we prepare to do the decode. Decode and transform models can't happen without pre-fills. So, what is prefill?It's a bit challenging to fathom if you're not in the machine learning or AI space, but it's utilizing a significant portion of the data that we have at inference time. So the data we want to process, which is supposed to represent that, but as we're doing inference, we ask, know.Upload a complex legal PDF or upload terms of service for the stuff we sign up for every day, or upload some complex documentation, and start to ask questions about it, right? Specifically. So we're not asking about general world knowledge. We're not asking about mathematical or scientific knowledge. I want to know if I can get my insurance carrier actually to cover this particular claim. And it's a very long and complicated document. So that's a very good example of what it takes to do inference. So, Prefill is taking your question about that document as well as the document itself. And after it's tokenized, after we convert the words, charts, and graphs into these AI tokens that we all use, we have to put context around these tokens. And we have to vectorize these tokens. And again, hard to fathom, but we add anywhere from 10 to 20,000 dimensions to each of these tokens. Because every word will have a different meaning based on its context, right? The C word here is everything context. So, Prefill is all about adding all these dimensions of context and creating these query key value KV matrices, which take kilobytes of these tokens and convert them to six orders of magnitude to gigabytes, so way past megabytes. We're approaching the gigabytes of working memory for this key-value cache, also known as a KV cache. has to be stored in memory.It can't be stored because we're accessing it all the time. The only way that...Doug: We also explained KV cache, right? Because I know we often use these phrases, I think it might be helpful to discuss them. KV being key value cache, right? Val Bercovici: Yeah, these are multi-dimensional structures. So, the keys and the values themselves are multi-dimensional matrices, and they each consume, again, gigabytes of data as they're transformed from words and tokens, graphs and tokens, to ultimately tokens, the keys and values. And it is how the algorithms of inference work. These algorithms are often referred to as flash attention or simply attention algorithms. One of the most popular ones for Nvidia processors is Flash Attention. And all of this ultimately involves matrix multiplication. And so that's why we've gone from these very, I think you know, fortunately timed graphics processors, which were always the matrix multiplications to render pixels on screen. We had these arithmetic logic units, ALUs, and these graphics processors, and we had thousands upon thousands of them. And NVIDIA, wisely recognizing that machine learning and Gen.AI utilize very large matrix multiplications. Now, they are adding more and more of these tensor cores, not just regular GPU cores, to their GPUs. These tensor cores specialize in performing large matrix multiplications at a more AI-optimized precision. Ironically, and I'm not sure if we want to go down this rabbit hole, but AI doesn't require as much precision as traditional graphics or high-performance computing. So, lower precision, same results, basically, same general level of accuracy, but way faster processing. So the flash attention algorithms essentially are what makes AI work on ⁓ GPUs.Doug: So let's specifically get back to the pre-fill deep, and like maybe I want to do some paraphrasing here. So, essentially, when you run an inference workload on a GPU, what happens is you load all the key values into the KV cache into memory. This memory is often HBM. The reason why HBM isn't so high in demand is that you want this big, as large as you can, essentially. Then pre-fill the weights. And then that's the compute-bound part. But then after that, you're just running the inference over and over and over. You submit a query, and then the model generates the results. So that's the decode portion. And that is much more memory bandwidth limited. I wanted to confirm that's the high-level summary. Is that correct?Val Bercovici: Exactly. And this is very much a kid buying our laptops today, right? We always want to buy as much DRAM if we wish to or memory on our laptops as possible. The same rules always apply to servers for making databases run faster. Really critical to this process of inference with GPUs. And there's three key tiers of memory we want to discuss here, Doug. There's the SRAM, which is really where the algorithms do the actual work with these tensor cores. These precious tensor cores on the video GPUs do the matrix multiplication.Then there's a high-bandwidth memory reference, which is essentially the working memory, where data is stored and retrieved from SRAM to facilitate the actual matrix multiplications. However, the challenge remains that we can never afford as much memory as we would like on our laptops or servers. It's the same challenge we see on GPUs. On the GPU package itself.There are all the GPU cores and the tensor cores, and we package this high-bandwidth memory; the real estate is just very finite. We can only get, you know, one, maybe 200 gigabytes now per GPU of high bandwidth memory. And because the working memory of these models has to also contend with the weights of these models, by the time you load the model weights for DeepSeek, by the time you begin to create this key value cache as working memory for the very first user you're almost out of that high bandwidth memory, right? Memory is just so precious right now, these models and the working memory are so large that there's this third tier of memory, which is the DRAM, regular dynamic RAM, DDR class five, you know, DDR five class memory right now. And that's a memory that's shared on the motherboard across all the GPUs, often eight GPUs on a single motherboard. That's the shared memory. And ⁓ that typically when you...Doug: Also, and then for the next generation, because you know the listeners are often semiconductor investors, so they are clued in, right? In the next generation, the grace portion of the Grace Blackwell CPU controller is often connected to banks and banks of DRAM, so that DRAM essentially is the third tier of memory. And so that's where the third tier kind of rolls inVal Bercovici: Grace Blackwell, yeah. That is certainly the case for GB 200s and even GHs, as well as Grace Hoppers, which have become a very standard computing unit in the world of AI training and AI inference.And the software, which we haven't mentioned, the software, which is key here in terms of how inference happens, is now very aware of that. So, the software consists of these inference servers. Historically, it would be NVIDIA's Triton TRT-LLM. For the past year or so, we've seen overwhelming market momentum and interest in the open-source VLLM inference server, as well as its thriving community. And we also see SG-LANG and other inference server models. We combine models and these inference servers into large kernels that we load into the GPUs and let them run. ⁓ so the software is now aware of these three tiers of memory. And there are these things called KV, Key Value Cache Managers, which manage the three tiers of working memory on behalf of these inference servers so that when you run out of high bandwidth memory, which is always, that you don't have to evict from KVCache. You don't have to say after five to 15 minutes, I'm just out of cache. I have to go back and re-prefill all that data, give you that 30-second delay, consume hundreds and hundreds of thousands of watts, and start the KV cache process all over again. Everyone's trying to minimize and avoid KV cache eviction as much as possible. We frequently observe KV transfers between GPUs and GPU servers to move memory around until it's exhausted, and then we have to re-prefill. One of the fundamental aspects here is that we want to minimize the time spent pre-filling by keeping everything in cache across an entire cluster of hundreds or thousands of GPU servers.Doug: So this is a, think, okay, let me try to summarize this back, because this is dense stuff. We're talking about how inference is done at a core level, right? We're discussing all the aspects that go into loading the KV cache into memory, including having banks of it, trying not to get evicted, and the software having an understanding and being able to KB manage so it can dynamically address where the memory weights are being stored or held. And that's kind of, I think, the state of the art till today. However, the reason I have Val on is not to explain it, but rather to explain how we infer at all on a GPU. It's the next step. The next step that I think is starting to become increasingly apparent. For instance, the foundational change is something called disaggregated, pre-fill, and decode. So, we've been discussing pre-fill as the loading of the KV cache onto the GPU, and then decode as the serving of the model, essentially running a query and then activating all the model weights, which then tells you the result. But importantly, there has been a significant shift, where we're starting to disaggregate it, meaning that it no longer has to be done on a single GPU. Because we have this KV manager, we can pretty much handle routing, and we could create a pooled access of resources to achieve better utilization of a GPU cluster. And that's disaggregated pre-filled decode at the highest level, but I wanted to give the mic over to Val to explain what, maybe in a little bit more depth, what exactly it is, why is it such a big deal, can you kind of even talk about the amortization of the KVCache, like how many users are being able to, the difference of being able to do one GPU on the pre-fill versus many on the decode, just kind of walk through the whole and what it will hopefully be able to enable. Val Bercovici: Absolutely. So it was DeepSeek, at least publicly introduced a new tier of pricing, that new class of pricing, cached input pricing, last year. They wrote a paper about it and disclosed how they do it, a few months ago, during their infamous six days of open-source disclosures. But it takes the concept, and we can go back to the early days of cloud and Snowflake. One of the reasons Snowflake became so popular is that they said, you know what, you don't have to have the same Amazon instance, cloud instance, to do both your data processing and your data storage. You could decouple, which was a term they would use back then, your storage from your processing, scale those differently, pay for those differently, consume those differently, as you have different ratios of, you know, storage work and processing analytics work. The same thing is now happening in the AI inference space, with disaggregated, rather than decoupled, pre-fill and decode. And so what that means now is the process of pre-filling, preparing the data to decode, is very GPU-intensive. That's where all those tensor cores kick in and operate full-time. And that is where it makes sense to have your best accelerator, your highest-performance GPU, focus on pre-filling your data and creating the KV working cache for five to 15 minutes or so, so that you can then begin to decode it. And that still is a serial process. You really can't decode until you create the KV cache, and creating the KV cache is very compute-bound. Now, once you've created that KV cache, the process of, again, getting to your reasoning and finally outputting tokens that you care about is decoding. It's very memory-intensive. It is going back because of the way these autoregressive transformer models work, which is a very Bayesian approach. All of the next token prediction is best performed by looking at all the prior tokens just up until the point in time you're creating that next token. So, you've to look back into your memory, into the context, over and over again in parallel millions of times, so that you can make that high-quality next token prediction. This is very memory-intensive, the decode. And you're stressing very different parts of a GPU server at that point, right? At decode, you're stressing those three tiers, particularly, you know, the high bandwidth memory and the DRAM portion of KV cache, because you're trying to pull during decode as much memory into the GPU, into the tensor core as fast as possible. Any delay there again just keeps that costly asset idle. Therefore, decoding again is a significantly different workload profile in the server compared to pre-fill. And what's making sense right now is dedicating at scale, particularly banks and banks of GPUs, just for pre-fill. That first part of the disaggregation, and then banks and banks of GPUs just for decode, and you can process optimize the GPUs for pre-fill, you should be processing optimizing differently than the GPUs for decode, because those are memory-centric and memory-focused. And what that means is you can bring new life into prior generations of accelerators by mixing Blackwell's, for example, for pre-fill, and hoppers for decode and each are performing optimally, each are certainly giving you a certain depreciated value, know, and current rate of return and you're not creating any artificial delays, you're keeping everything humming as efficiently as possible by optimizing what each class of processor and its memory ecosystem is best suited for.Doug: Yeah, so I want to reiterate this because I think it's really important to understand why this is such a big deal. Nvidia talked about it at GTC. Dynamo was essentially the implementation of some of the work that DeepSeek had already did. This is going to be the key story for inference serving for the rest of this year. I think, and we've got to think about the bigger picture here, I think it's really important because what this will enable, like we're talking about the inference and the token throughput per a single node, this will hopefully be able to add a lot more scaling out and meaning that you know it's the same GPU bottleneck we had will probably because of the increase. I don't know, and this is something that I think a lot of people are working on in terms of quantifying. think we are at SemiAnalysis as well, right? However, understanding what this unlocks is a really big deal. And essentially, I mean, it's kind of interesting. I don't think it will happen, but it would be effective within inference, right? Within this test time, compute scaling, we're almost unlocking two different types of compute. That's being done just for inference, if you remember roll back a year ago. Everyone's talking about doing or two years ago. Everyone's talking about accelerators focused on inference versus training, right? This is just within inference, two different types of computation being done here, can head, and then the point is that the heterogeneous Output is going to be a lot bigger than what one single node can do. I'm very excited because I think it's going to increase token throughput massively, I think it's going to improve memory utilization massively. Val Bercovici: Exactly.Doug: And then obviously that brings the cost of tokens down. Is there any other kind of things that I should be aware of that are like logical takeaways from that?Val Bercovici: So, you know, one of the first takeaways here is that ⁓ disaggregated prefill, disag prefill is introducing the notion of assembly lines to AI factories for the first time. because it's kind hard to imagine a factory without an assembly line today.But back in the olden days, when factories were first established, they had clusters of workers coming in and out of a particular work area, such as a car assembly line. The different specialists would go and work on the car, then leave, but the car never moved, and it was very inefficient to have to move the workers in and out. That's exactly how inference happens today before Disagg Prefill. Assembly lines, as the name implies, essentially mean that we keep the data flowing and keep them flowing through whatever specialty is necessary for processing. So prefill is a very different set of work than decode. And that's why moving data using an assembly line process, the way DeepSeek innovated, at least publicly, is very, very important right now. So that's one of the first takeaways: we're finally entering the era of assembly lines for AI factories on the inference side. Next, of course, is just the nature of the workloads themselves. Specifically, 2025 and a subsequent year, as an excellent proxy for this, have seen workloads shift from being individual chat session-based to agent-based, which means much more context.We're talking about processing large volumes of documents, extensive code bases, extensive transcripts, or large videos themselves. And more importantly, we're not just stopping at the first question and answer. We're asking a lot of repeated questions and receiving the same answers. It's called a multi-turn prompt. This notion of high-context, multi-turn communication is becoming the dominant workload of 2025. And that really presses, know, or stresses inference far, far more and creates this need to scale inference and decouple the two different natures of the workloads, the pre-fill and the decode, to get to the maximum efficiencies, tokens per second, tokens per watt, and ultimately be able to support more users simultaneously, larger batch sizes on the same GPU asset investment, the same memory deployment, and certainly just the same power budget that every AI data center operates under.Doug: So, I'm going to transition this to Weka and your potential solution. You know, we're working on some of the verification for all this stuff, but talking about how we scale the decode out to become a lot bigger, because in this process, we're talking about the assembly line and being able to split one portion of inference into one part and another portion of inference into the other. The thing that's specifically on these agentic really large multi-turn processes is that the KB cache just explodes because you know the entire context of your Claude code or your cursor session becomes it hits the context window every single time, so the decode and kind of disaggregating the decode can make hopefully I believe a much, much larger context windows. Can you help me discuss what is currently being done, which, to my understanding, involves DRAM caching —essentially, giant pools of DRAM cache? For example, if you have many users at OpenAI, what Weka is trying to do is address and scale out the decode context window. Val Bercovici: Absolutely. So, you know, let's take one quick step back, one short step back in an ideal world. Since we're essentially creating this working memory in prefill before we can actually use it for the outputs on the decode side, we don't want to have to recreate it repeatedly. A best-case scenario is that we create working memory for every model instance session, and then we just decode forever. So DeepSeek created a simple formula for this, xpyd, and the open source community in particular, Vllm, has adopted it as a discussed scaling inference. So, XPYD simply stands for X, how many times you have to pre-fill the data in a working session. So, if your working session extends for more than five or 15 minutes, and you have multiple simultaneous users on the same cluster, that five to 15 minutes gets compressed because you have to support more users on the same hardware and memory. So, you typically have to do many, many pre-fills, and that's where a lot of the waste, slowness, and inefficiency come in. However, you want to have a certain ratio between X pre-fill and Y number of decodes. However, the ideal scenario, of course, is one P and one pre-fill. You pre-fill the data, and then you decode forever. How do you get there? Well, how you get there is by having more or less infinite memory. And we've heard this term from Google, for example, infinite context windows. So, Google has been able to approximate this by using both their TPU, or Tensor Processing Unit, architectures, as well as a ring retention algorithm particular to Google and Gemini.And they've been able to take banks and banks of DRAM, network DRAM associated with their TPUs, and give you these million tokens, up to 10 million token context windows. They were first to market with that because they were able to optimize their infrastructure for that. However, the economics of doing so are very stressful, even for companies like Google. And not everyone is Google. Until recently, no one else had million-token context windows. We're just on the cusp of seeing that go mainstream with Facebook, with Llama 4, with Minimax, which was just released the other week, and various models that will soon support millions in token context windows, which everybody wants, by the way. So, the pressure is on. How do we take these three tiers of memory —SRAM, which is super expensive, super finite, and high-bandwidth memory —as your readers know all too well, also relatively expensive and completely finite —and DRAM, which is also theoretically expensive and finite? Well, when you look at the math, DRAM is the one now, and its cousin, NVMe, non-volatile memory extension, and flash devices, are now within striking distance of each other. know, DRAM in isolation is nanosecond class latency, NVMe in isolation is microsecond class latency, but you brought up the grace chip earlier on, and even, you know, the Hopper class, whether it's GH or Blackwell, actually doing the transfer between DRAM and HBM is on the order of microseconds, whereas in doing the transfer from HBM to SRAM is still on the order of nanoseconds on these GPU servers. So because now we're looking at a microsecond-level transfer between HBM and DRAM, optimally managing these NVMe NAND flash device as well is the key to making this all work. And I always like to joke there's no S in NV’ in NVMe, right? It's a non-volatile memory extension. So that's one of the things Weka was optimized for doing from day one is taking full advantage of all of these new converging standards, NVMe, the fact that a lot of people still haven't internalized the notion that the high performance compute networks in AI factories that GPUs are attached to are faster than the motherboards themselves. This is very counterintuitive to most people. So there are more PCI lanes accessible on the network than on any individual server motherboard, or GPU server motherboard. And so that means that if you can aggregate all that amazing network bandwidth to the GPUs across your high-performance computing networks and attach NVMe devices on one end and get the consumption by each GPU across that high-performance computing on the other end, you end up with more memory to the GPU, more DRAM particularly, to the GPU from the network than from the motherboard. And that's what WECA has released with augmented memory technology. We've been publishing our own benchmarks on this topic in our blogs since February. We recently had our first cloud partner, OSCI, and Oracle Cloud publicly publish their own benchmarks and validate these results, showing that you can extend the DRAM class of memory from the motherboard down to the compute network on these GPU clusters. And by leveraging that compute network, we'll dive into what that means in contrast to the storage network. You're able to have effectively limitless DRAM, which now means limitless KV cache, and ultimately, infinite context windows for everybody else outside of Google.Doug: So yeah, that's the engineering that will make the infinite context window. And to be clear, context windows are exploding already. And leading-edge models, for example, are often larger than the stated numbers available in the API, which is necessary for some fine-tuning, as well as for some of the system prompts that are incorporated into the models. think, yeah, that's kind of like the Weka solution here.I would actually like to use the rest of this time to transition to a brief history of storage, as Val has been around the block, and it would be a real shame for me not to discuss the other companies. One of the ones that everyone is probably aware of from here is like Pure Storage. They're publicly traded. They're, you know, the kings of network-attached storage. I would love to discuss this transition, possibly moving away from the disaggregated, pre-filled decode, which, to be clear, is a significant change. This is how we're going to scale to much bigger. Now I want to talk about how the network attack storage space itself has kind of transitioned from you know, kind of the history of the past to now, because I think that that would be a very interesting arc to sort of go through with my listeners. Val Bercovici: Absolutely. So let's go all the way back, even though storage predates network attached storage, but clearly network attached storage is a big deal. NetApp was born in the network-attached storage era.They kind of optimized storage not for the storage media, which back then was only what we call spinning rust, now hard drives, which have heads that platter that spin and heads that actually go back and forth across the platters to extract the data. NetApp was born in that era, optimized for that environment, and, you know, became the brand that it is today. Pure that you brought up earlier on was born of the flash era, of the NAND flash era. And so Pure realized, you one fundamental thing, which was, hey, all of a sudden this NAND flash media that we're having terabytes and terabytes of in our storage arrays is worth more than this CPU based controller that controls all of it and presents, you know, these protocols, these SAN and NAS and other protocols to the servers and other end users. So by just realizing that, you know, there was an advantage to engineering the NAND flash and the NAND flash shelves and then creating this really brilliant program, the Evergreen program, just to let people upgrade controllers because they were the lower-priced item compared to the media. Pure, you know, became the brand that it is today in the sand space and in the flash space, flash storage space, because they were born in this era, understood the supply chain, understood the value, the ultimate customer value that is fundamentally different than spinning media, the older hard drive generation of technology and emerged to be the leader they are today.Doug: So, then I want to, so I know Pure Storage is the leader, like in the legacy compute, let's call it that. I'm sorry to call it legacy compute. I know the CPU guys probably hate it, but let's be real, it's legacy computing, right? Let's discuss what that looks like in the future. Because I think that there have been a few other, DDN, VAST, Weka, started, these newer challengers who are entering this space specifically with the focus on the opportunity from the newer compute strap ⁓ world, which is mostly ⁓ a GPU-driven architecture. So let's just kind of talk about how, know, what are, you know, you can talk about your competitors, whatever you want to do. I just think a market ecosystem overview of, and then obviously with the caveat that that works for Weka.Val Bercovici: Yeah, you know, let's take a simple visual that I think a lot of us have seen. We've all encountered the concept of exponential data growth. And we've probably seen some of the crazy charts. I'm a visual guy, so looking at the hockey stick charts for exponential data growth, that's no longer a future state; that's a present state. We're in the steep part of this hockey stick curve, and exponential data growth means just that. There's just an enormous amount of data being generated synthetically now, as well as organically by the world and by all of our systems. The vast majority, 80- 90 % of it, is unstructured data. So the legacy market, the legacy compute and storage market that we talk about now jokingly, was not optimized for unstructured, explosive, exponential data growth. It was optimized for transactions, Oracle databases, DBT databases, SQL servers worldwide, structured data, and latency-sensitive data. And that was paired with more analytical systems, which provided more horsepower, more CPUs, and more memory, all brought to bear in the cloud. This was the dominant workload of the cloud, creating these analytical systems to extract meaning and value from all this transactional data. Enter GPUs. And again, GPUs just are fundamentally different than CPUs. The stress and workload on GPUs bear no resemblance to what a database transaction looks like. To process all of this massive, exponentially growing data in parallel, we need fundamentally different storage systems and they're not storage arrays at all. They have to scale out much, much more horizontally. And let me know if we're gonna have a new member in the group. That's all good. To address exponential data growth, you need all your storage systems to work in parallel. They have to be scaled out from the start. They have to be optimized for unstructured data.And unstructured data doesn't mean just one workload, one storage workload. It means large file reads and small file reads. It means both random access and sequential access. It means ⁓ focusing on ⁓ billions and billions of directories and trillions and trillions of files in those directories. It means object interfaces, S3 protocol interfaces to millions and trillions of buckets. It means so many different things.In parallel compared to what we've seen in the past for CPU-based storage. So the fundamental systems are just built differently. Weka, in particular, doesn't build storage arrays, never really has. Many people like to think that we build bigger storage arrays. That's not true. We've had this software-defined containerized system from day one. Again, we're born not just of the cloud era, but of the AI era, just as NetApp was for NAS and Flash was for Pure. We're AI-native; we're very GPU-native systems.So we call this a mesh now, just a mesh of containers that fundamentally take advantage of one reality, which is that the network in GPU computing and AI factories is faster than a motherboard. Weka is the first storage system and the first storage cluster designed for this reality. No one else is. Everyone else optimizes around.Either individual storage controllers or, like most of our competitors, clusters of storage controllers front-end the actual high-performance NAND flash and storage media. Weka doesn't work that way, right? One of the most radical examples of how Weka differs is something we call converged mode, where we deliver software-defined memory. That's an oxymoron for most people, but then again, software-defined networking was very heretical in the earliest days. Software-defined storage became somewhat less heretical but still novel when it emerged. Software-defined memory today is very heretical, but people will realize soon that when you're buying these banks and banks of GPU servers, they come, of course, with GPUs, they come, of course, with all three classes of memory we talked about. Still, they come with X NVMe drives, often eight NVMe drives per server.Since GPUs, particularly at inference time as well as training, often work in clusters themselves, when you have eight GPU servers, each with eight drives, or let's pick nine GPU servers to talk about NVL72s in particular, with eight GPUs per, often you get about eight drives, about 72 drives, NVMe devices, per NVL72.Installing Weka software on that instantly creates software-defined memory because we take those 72 NVMe devices and convert them into a DRAM class of memory. And these inference servers we spoke of, VLLM, TRTLLM, SG-LANG, now understand how to recognize the thousand times more density, the terabytes per device of memory, now to complement the very limited number of terabytes. Really the fractions of petabytes of NVMe capacity with the terabytes of DRAM capacity with the gigabytes of HBM capacity. And all of a sudden, we have a theory mechanism that gets you to this Nirvana ideal state of pre-filling that working memory that KV cache wants after you've loaded the model weights, never having to pre-fill again, never having to evict cache and decoding forever. And we can essentially fast-forward AI inference to just decoding.Everybody wins, particularly for these cursor-like, agentic workloads where it's always high context, it's always multi-turn. If we never pause and slow down to re-prefill every 10 or 15 minutes, but we hit the throttle, you know, we hit the throttle, we put the pedal to the metal, so to speak, nonstop during inference, we get optimal inference. then what I love is the token economics, the unit economics of this, all of a sudden make a lot of sense right now, because we can do, create more tokens per second in aggregate, generate larger batch sizes, which means supporting more users, and every one of those users gets the lowest latency time to first and last token. So it's a win-win all around when you can unlock that last final bottleneck of AI inference.Doug: Yeah, so yeah, I think that that's a really good ending on what weka does as well as like, know, Kind of wrapping it all together of the disaggregated PD. This is the example. Yeah, I'm pretty excited for what's to happen there. You know, there could be some, well, let's see, I'm not sure if everyone wins, right? Well, I guess the reality is that if we purchase more GPUs, right or no, if we do more tokens, we create more things. They'll still purchase more GPUs, like me, something that I'm not familiar with, but it doesn't concern me. However, that does sound like GPU utilization is about to reach its limit, right? But that's an aside, right? But I do appreciate, yeah.Val Bercovici: I would say just to wrap up technically, GPUs were underutilized before. They weren't utilized during the majority of inference, which was the decoding part. Now we truly drive up GPU utilization to its maximum potential.All of the NVMe, ⁓ sorry, all of the HBM capacity being manufactured is already allocated and being purchased. All the DRAM capacity, particularly the AI-friendly DRAM, is already being purchased. So this is very bullish for NVMe, the Flash SSD ecosystem, particularly the high-performance TLC class of NAND Flash. It's very bullish for that because it represents a new life on top of a pretty valuable market opportunity that NVMe has always had.Doug: To be clear, that's not the official opinion of fabricated knowledge of SemiAnalysis, but it's the official opinion of Val. Anyways, I just wanted to wrap it up here. Is there anything else you'd like to say to the listeners, or should we probably leave it at that?Val Bercovici: It's my opinion that I expect one fundamental change in the AI business, which is again funded by inference. And that is, we're going to see the three classes of pricing, which is input cashed input and output, collapse very soon to only two classes of pricing because people will realize when you don't have to only have five to 15 minutes of life for your input token pricing, it could be weeks or months of life or effectively infinite. You don't need to have a distinct tier of pricing. So this is gonna fundamentally shift. The business of AI and the unit economics of AI, simplify the pricing and give more value to users. And it's a question of which provider will lead the charge and define that new low class of pricing, and who will be the followers.Doug: I'm excited to see how it shakes out. Thanks for the time, Val. Appreciate having you on.Val Bercovici: Likewise, love the conversation.That’s it for this week! Thanks for reading! Get full access to Fabricated Knowledge at www.fabricatedknowledge.com/subscribe

Wes Cummins, founder and CEO of Applied Digital, shares his journey from tech investing to tech hardware. He discusses the company's significant pivot from blockchain to high-performance computing, emphasizing partnerships and innovations with NVIDIA. Wes dives into the importance of Power Utilization Efficiency in data centers, as well as the challenges of scaling power capacity amid supply chain issues. He sheds light on advanced cooling systems and their critical role in meeting the demands of hyperscale customers.

Dan Kim, part of the CHIPS Program Office, previously served as Chief Economist for SK Hynix, while Hassan Khan specializes in manufacturing incentives. They unpack the CHIPS Act, highlighting its bipartisan foundation and $39 billion funding aimed at boosting U.S. semiconductor manufacturing. The conversation tackles budget complexities, talent acquisition challenges in the industry, and the critical need for workforce development. They stress collaborative efforts to enhance project quality and explain how exciting the semiconductor field can attract new talent.

This week, Rajesh Vashist of SiTime came to talk about SiTime’s new product launch, Epoch. Rajesh has been on the newsletter/podcast before, and I loved our last interview. I hope you enjoy this interview as well. Doug O’Laughlin: Hey, Rajesh.  It's nice to have you back on the newsletter. It's been a bit since we last talked, but I thought we might rehash the story for people new to SiTime. So, tell me about yourself. Tell me about SiTime. Tell me what you guys are focused on.Rajesh Vashist: So good to talk to you again, Doug; it's been a while. SiTime is still the least understood story in semiconductors because timing is one of the least understood products or applications in semiconductors. Timing chips. They are the heartbeat of any system in any communications, processing, computing, or all of the above. Timing is necessary, and SiTime makes timing chips. Our name is Timing. Si-Time. And we have decided to focus only on this $8-10 billion market. We're the only company focused exclusively on timing, and we're the only company focused exclusively on all aspects of timing. That means that with the advent of more AI, more automated driving, the Internet of Things, 5G communications, and healthcare. Timing becomes increasingly important, and the market for our products is precision timing, which is, of course, very high-performance timing across many parameters. But the kicker is under tough environmental conditions, under conditions of heat, under conditions of airflow, under conditions of small size, low power, shock, and vibration. Performance with tough environmental conditions is where SiTime makes its mark, and we're the only company that is going for it. We invented precision timing.DO: Let’s talk about precision timing and the difference between you and some of your competitors. I think people new to this story might not appreciate the big difference between what you do and what the market does. It’s distinct because you guys are the only player in MEMS, and everyone else is using legacy quartz. So, maybe let’s talk about MEMS and the benefits of MEMS. And why you guys are so well positioned in that space?RV: Right. So, we have several differentiations. You alluded to one major one: quartz technology or the quartz crystal. We've heard those terms interchangeably for the last 70 years. Quartz crystal has been around for 70 years. Most of the $10 billion market is serviced by quartz. Quartz is the dominant solution. Quartz has put a man on the moon. It’s a good technology. SiTime focuses on semiconductors, not quartz. That's the main point. Whenever semiconductors enter an application that is not using semiconductors, semiconductors do a superlative job in quality, reliability, performance, size, and manufacturing. In other words, when you compare hard disk drives to flash drives, when you compare LEDs to incandescent light., semiconductors win. MEMS technology is solving hard timing problems. It's not necessarily taking market share away from quartz but going for applications that aren’t serviced well enough.So think that the world I mentioned, AI and 5G and beyond, all of these will need SiTime's products. There's a very distinct difference between the two technologies. There’s also a distinct difference: quartz only goes for one portion of the market, the resonator oscillator. There's another portion of the market called clocking, which is a semiconductor technology. SiTime does all three of them, and that's again another distinction.DO: That's a pretty good overview of the story. Maybe we can take a second here to talk about what's new. First, we can talk about market conditions, and then about your recent new product launch. Since I last talked to you, it was the first quarter of inventory guide downs in this pretty drastic semiconductor correction. You finally have guided to the first sequential revenue quarter growth next quarter. So it looks like the worst is over. How do you guys see the demand on the other side? The lack of visibility into steady state revenue growth is where investors are probably most anxious and worried about the SiTime story.RV: Right. So, I won't comment on Q3 because it's in our quiet period. But let me talk about the broader growth of SiTime. Our thesis is very simple. Our thesis is that if you believe in AI, ADAS, IoT, and 5G, then you've got to believe in precision timing and SiTime. That’s all fine, but people want to know where’s the evidence. The evidence is our pole position. Four factors show our pole position. One is that our ASPs are average selling prices. Despite this big revenue decline, we continue to hold or grow our share. The second is unlike the quartz-based solutions, with 40 players and providers of quartz solutions. These players from Japan, Taiwan, Korea, and China are all interchangeable; therefore, they're not single-sourced as a customer. SiTime has been an 80% single-source supplier for several years. In other words, it's a demonstration that our customers choose to use us even though we’re a single source to them, which is a great testament to the company's value proposition. The third is that our design wins, which are the heartbeat of any semiconductor company. As you know, our design wins indicate our future potential, and our design wins continue to grow. Generally speaking, our quote activity, our quote/quotation activity, is growing. Our SAM, which is a served market, has been growing. We went public in 2019 with a billion dollars of SAM. This year, we'll exit with 2.51 billion of SAM. And by the end of 2024, we'll have $4 billion of SAM. That's an astonishing position, A) of growth of SAM in four years, and B) of the fact that we have this much-served market that we can even access. That is an amazing statement. These four points, Doug, really point to the company's strength despite the slowdown. Despite the increased inventory at our customers primarily last year, we shipped more than they needed. This year, China is having some economic constraints. Meanwhile, networking, telecommunications, and data centers have been slow. So demand is somewhat down. So that's a 1-2 punch. The low from high inventory buildup and the lower demand. However, all that said, we expect that we get into positive territory in 2024.DO: Perfect. Remind people how long it takes for these design wins to flow from orders to revenue and how long these design wins can be. For example, some of your first-generation products are still being sold despite being probably inferior to what you’re selling today. I think that might be helpful for investors to appreciate the longevity of design wins and the certainty of these design wins leading to SiTime’s revenue trajectory.RV: So that's a good point. Another way of looking at the duration of design wins is the age of our products. SiTime, from the day it first introduced a product in 2008, has never end-of-lifed product. We've never had an end-of-life because we know there's always a use case for a product. It is inferior in performance, though it may be. So we have design wins that go in the consumer business, which typically take about a year to a year and a half to go from when the engineers start looking at our product to when they start shipping. On the other hand, the longest ones are in military aerospace defense, where it might be two and a half years for design wins and another two years for production. Conversely, in the consumer business, new models come every year, so the model for this year will ship next year. And that will be it.On the other hand, in the military, aerospace defense, the missile or the munitions or the space unit probably goes for ten years and keeps on shipping for ten years, and it's a point of stickiness. So, SiTime has a portfolio strategy because of the diverse nature of our business. We are diverse across applications, and we're diverse across use cases. So we can build a portfolio with both spurts of growth through the consumer piece and longevity of growth through the longer design win cycle and the longer life of the end product that we ship into.DO: Perfect. Now, I want to shift to the opportunities ahead. The last time we talked, you discussed how networking is the biggest opportunity. Especially given the explosion of AI and 5G and all these new markets. Do you still see networking as the biggest opportunity? How long is the networking Design cycle for design wins and then maybe last? Let's touch on your new product launch because you guys are very excited about that.RV: That's right. Networking telecommunications, enterprise, data centers, we lump that all in; we sometimes call it communication. Sometimes, we call it a networking business. Unfortunately, we don't quite have a great handle on what to call this. Still, it's essentially 5G, backhaul, long haul, mid haul, satellite communications, Microwave communications, and, of course, the Hyperscalers and AI. So out of all the markets we have, this is the biggest market by far. By the end of 2024, I said earlier that this would be a four billion dollar-sized market for SiTime. The networking, telecommunications, and enterprise data center market is $1.2 billion. The other great part is that the prices or ASPs are high. The gross margins are high. The stickiness or the longevity of the design win is high, and it's an architectural move by companies like Ciena, Cisco, and Ericsson or on the hyperscaler side like AWS, Google, or Meta. The fact is that we have resonators, oscillators, and clocks, and they come together in a solution for the customer, uniquely different solution from anybody else. Many others are in the pipeline, like our product Cascade, which goes with our elite product line. Or our Elite X line and the recently launched Epoch product line. This is a 1 + 1 = 3. A full system play into the market and gives us much customer traction. Because, again, the timing solutions are getting increasingly difficult, and SiTime loves to solve tough timing problems.DO: So let's talk a little about Epoch, because I think it's interesting how different it is from your single-timing solution like Cascade. In the press release, you talk a lot about holdovers and smaller sizes. I'm curious about holdover because, to my understanding, it is how long a frequency without another reference frequency can hold. And then, on size, SiTime has already been the leader in shrink. Is this a new, smaller form factor compared to other products that you've had? And then I'll probably have even more questions about Epoch. RV: I'm going to get a little geeky in using terminology very specific to timing. The lowest level of performance measured by the stability or holding a particular frequency at a particular frequency fidelity is called an XO.X stands for crystal, and O stands for oscillator, so it is a crystal oscillator. The second higher one is TCXO. In other words, it's still an XO crystal oscillator, but temperature compensated. The 3rd and final one is called an OCXO, which is also still a crystal oscillator, but now it's an oven control and more on that in a second. Now, this is SiTime’s second big foray into the OCXO. We've had a product we call Emerald in the past. We call it OCXO, but we are not crystal. We didn't want to change the terminology even though we're not crystal; we just adopted it. So, this is our first OCXO solution engineered from the ground up. It's been several years in the making. And we've given it an apt name. Epoch, it's the dawning of a new age. It's a dawning of a new product class in the OCXO timeframe. So, the O in OCXO refers to the oven creating a constant temperature. In the constant temperature, the oscillator gets to keep its particular frequency. So because of that, if you look at the size of the current OCXO, which is not from SiTime, they're quite huge. SiTime has always been in the business of trying to make it smaller. Because A) we're highly integrated semiconductors and B) we don't need much of the technology that has gone for the last 50 years into OCXO.So, we are dramatically smaller than the current OCXO. It's a new form factor in the OCXO product line. It's about a ninth smaller size, and we are dramatically smaller. The holdover concept is one in which a typical OCXO depends on a GPS source, which is an atomic clock source for connecting time to time and getting the most accurate time from the GPS signal. The point is that power that fuels the box is never 100% reliable, so there is no GPS signal in the absence of power. There is no outside reference, and therefore the system then relies upon the time that the OCXO keeps. At this time, the fidelity of that time is measured in hours; about two, maybe four hours is the state of the art. This area has a lot of hyperbole, but it's no more than two to four hours. SiTime is coming out within eight hours and, in some conditions, a twelve-hour hold-over period. So, it's a real change in the reliability of the end system if it can go twice as long or three times as long without power. The other part of this is that the OCXO supply chain is very messy. It's a very long supply chain. It's not meant for predictably producing products, you know in, in, in millions or hundreds of thousands year after year. Our solution? Given that's a 100% semiconductor solution, we simplify the supply using our innovative MEMs technology. So, these three things are absolutely important to the market. We'll keep talking about this product for a long time because it has many aspects in many different markets.DO: I have the I have the dimensions here. It looks like it's 9 millimeters by 7 millimeters by 3.73 millimeters. And just for context, you know, you look at OCXOs, we're talking products that cost hundreds of dollars, 20 millimeters in size. It's a much larger timer. This seems to be extremely competitive. Listeners might not appreciate it in semiconductor packaging; what we want to do is shrink it, to move all the parts closer together and with lower energy. RV: We are a sort of low-key company. We let the product talk, but in this case, it is unique. DO: So could you give some examples of how you've enabled customer success like people, you know, like lowering the time to market or being flexible in redesigns? The tangible results of how you enable customers often help make the story much better. RV: Doug, this theme runs through all SiTime’s products and back to the 80% sole source. You can imagine that customers have been buying from 40 vendors of crystal technology. You have to ask yourself why it is that in a 70-year-old business, there are still 40 players. There are 40 players because they're unable to satisfy the demands because they either cannot manufacture enough, or they manufacture different sizes, or their supply chains are not generally reliable. For SiTime to have 80% of its business come from a single source, it's a testament to the integrity of the supply chain. We have two different chips, and Epoch consists of these two different chips. One is, of course, our classic analog chips, made in 180 nanometers at TSMC, which is our sole foundry for these. The other is Bosch, a leader in different kinds of MEMS, in accelerometers and gyros. For the last 15 years, we have had an extremely close relationship with them in Germany, where they manufacture these parts solely for us using our proprietary technology.So SiTime’s proprietary technology in MEMS allows us to bring 100,000 die per one wafer, which is an enormous amount of die. Typically, chips have 5,000, 2,000, 10,000, and some of the AI chips had two, two, or three times per wafer. With 100,000, you can imagine, die per wafer, and we can do a billion units of MEMs-based products with only 10,000 wafers.That's a tremendous amount of scale. And on the TSMC side, on the analog side, it's a well-known story for analog companies that they can go and scale very quickly. So SiTime uses multiple chips in this product, analog and MEMS, and we have architected a better solution across eight different parameters. It's better in seven parameters. So, there are seven parameters that it's better at, not just with one competitor but with the entire industry. So, different companies are better at some of these things, like aging or startup time or size or height. But SiTime is bigger than the whole as if we took the best of everybody and compared ourselves to the best of every product. We came out seven out of eight better, and on one, the eighth one, we were equal in the stability over temperature. So, we're talking about holdover, footprint, height, low power, aging, warm-up time, and air flow-based stability. We're not better by 5% or 10%, but we are multiples better than our competitors. That is why we call this product Epoch.DO: That makes that makes sense. This has been multiple years in the making for SiTime, right?? But at the same time, there are also new products in the pipeline. There’s a big difference between the mature quartz ecosystem and SiTime. At the same time, Quartz has had 70 years to reach technological maturity. We know that the road map for MEMS is even longer. That's pretty compelling if you ask me. What other parts of the story do you think people following your story should focus on? RV: Some things we do are very much translatable into non-timing markets. SiTime is a signal integrity company, and signal integrity is used in many ways. It's used in Serdes. It's used in retimers. But many of those markets are ahead of us, maybe in four or five years. We think today we have a unique opportunity to focus on this one market where we're unique because nobody else is focused exclusively on timing. Even Quartz players, for example, Epson or Kyocera, do other things. SiTime is the only company to do clocking and frequency products, such as resonators and oscillators. And so that gives us a unique ability to be a full-line systems provider to our customers and solve their tough timing problems. And timing has gotten more difficult since the time we started doing this. It's become way more difficult with the advent of these use cases. I think it's a real service we provide our customers, which we didn’t use until 20 years ago. If you talk to a large company, they have a half dozen timing experts. In the case of 1 particular company that shall go unnamed but the name you'll recognize in the networking communication space, one of our experts talked to 10 people in their team for five hours over one metric called jitter. Because jitter alone is so complex. SiTime gets to help our customers educate our customers and then serve our customers to take this timing headache away from them and deliver them the best solution. DO: Part of this is because some customers don't have a historical semiconductor background, right? The big tech companies become much more vertical; for example, Tesla called out MEMS resonators for their Dojo chip. One of the frequencies of the cores was interfering with one of the frequencies of the timing solution. And so they had to redesign it. The benefit of MEMS is that you can press a button and change the frequency. That's very compelling, especially on the networking side, because networking is getting harder as we go from 800G to 1.6T. The precision is really important.RV: And it's moving everywhere. The other day, we came across an application with an air conditioning unit with 600 gigahertz radar to tell when the room was empty. SiTime’s solution is used there and can make it better at judging the occupancy, which would contribute to ameliorating the effects of global warming.DO: I think we should talk about power efficiency because when I first started learning about quartz versus MEMs, I wasn't completely sure if it was a power performance difference. Is there? RV: I think there are two examples. One is what you put your finger on, which is lower consumption by our chip versus another quartz-based chip. And there, for example in this example of Epoch, we're three times lower power. So that's the sheer consumption of our power. But you know that is in terms of milliwatts, right? Milliwatts are important, but Watts is even more important. So, let's talk about another use case where we are important in lower power consumption because our products can withstand heat, be placed anywhere on a board, and still have high-integrity clocks. Because of that, the larger chip we are next to could be a modem chip. It could be a power management IC, or an FPGA; that chip can go to sleep and save its power because it depends upon our clock; it depends upon our chip to wake it up and make and allow it to go to sleep more importantly when it's not being used. So, this is a bigger saving. Sometimes, it can be measured in watts and sometimes in battery power. So there are two benefits of this, and both of them are generally combined; we know that power consumption for anything, whether it's a watch that sits on your wrist or even a satellite that's going around in space, is very important. It's very important because the more heat that's generated inside the system. Many of these systems do not, cannot afford to have fans. They cannot afford to have an exhaustion system. Sometimes, next to the chip, the temperature can get up to 120 degrees C, even 150 for a fraction of a second. Quartz crystal will have a greater movement of frequency than SiTime. We have some examples of that on our website where we showcase that.DO: Perfect. That's a perfect explanation. It was focusing on the package's macro instead of the chip's micro. It makes the designing a lot easier. That's very helpful. Rajesh, I think this is wrapping it up. Thank you for introducing your new products. Anything else you want to say before we hit the road?RV: No, thank you so much. I think it's great that the story of SiTime is coming out. As I said at the beginning, some people look at it and ask, how could such a great story be? Where is the fly in the ointment? And my only answer is that people don't know yet about us. We're still a young company. We're comparing ourselves to a 70-year-old technology. It takes us a little while to get the message out. So, thank you for helping us and getting the message out because, in the end, it helps our customers.DO: Thanks for for coming on, Rajesh. I'll probably talk to you sometime in the next year. I'm looking forward to all the updates.RV: Thank you. Take care. Bye bye.DO: Take care. Bye!That’s it for this week. I’ll have a more detailed update when earnings come out. SiTime looks expensive, but its continued execution and design wins should support the company in the coming years. Until next time! If you enjoyed this, consider subscribing to the newsletter! Fabricated Knowledge is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Get full access to Fabricated Knowledge at www.fabricatedknowledge.com/subscribe

Today’s post will be an interview with Tony Pialis, founder and CEO of Alphawave. Alphawave recently got the all-clear from their KPMG auditor, and shares have been unsuspended. This interview is lightly edited for readability. I have some thoughts at the end as well. Doug O’Laughlin: Hey Tony, this is Doug, and I’m excited to have you hear on Fabricated Knowledge today to talk about Alphawave Semi. Alphawave is a company that I’ve talked about, and there are a lot of aspects to this story that has been confusing. So I’m happy to have you hear so you can tell your story to people and discuss the obstacles lately. Let’s first start it off with what’s the story of Alphawave and what’s your story, specifically Tony.Tony Pialis: Thanks for the great introduction, Doug. I have also been following you and am excited to be here. Thanks for giving me the opportunity to talk about Alphawave. It is the cumulation of my life’s work. So first, a little background about myself. I’ve been in the semiconductor industry for 25 years, exclusively focused on connectivity. Everyone talks about computer chips and how they are used to process data, but connectivity brings the data on and off the chips. It’s a lot like the human body; if the processor is the brain, connectivity is like the spine. It brings all the information from the limbs into the brain and back to the limbs to do work. That’s what I’ve been focused on, and that’s what Alphawave Semi is focused on. I’m also a little bit of a serial entrepreneur. Alphawave is the third business that I cofounded and funded and led. The previous two had successful exits, but Alphawave when I founded it I had a different ambition. We are building the next great semiconductor company in the industry. We are doing it focused on high-speed connectivity. We started focusing on high-speed connectivity. We started by delivering it in Silicon IP, which is IP that other semiconductor companies such as Intel and AMD, and networking companies like Broadcom or Marvell could integrate on their chips. After we took the company public in 2021, we expanded the product portfolio to include delivering our connectivity technology into silicon format, which means we're now a vertically integrated semiconductor. So we can deliver our solutions as IP that goes into someone else's chips as custom silicon where we build a custom computer chip to meet a hyper scaler-specific need. Let's say something like an AI chip.And we also sell networking chips currently today targeting longer-reach networking. So whether it's driving cables, whether it's driving optical communications, and so look, that vision will help Alphawave Semi continue to grow.We've guided the market that by 2027 we'll be doing over a billion dollars of revenue per year. You know, one thing I also want to highlight is just growing 100% year on year is difficult, especially when you get to our scale, which is expected to do.Almost 1/3 of a billion dollars, or over 1/3 of a billion dollars this year in revenue, we closed on a great Q1, even though everyone's worried about recessions where we booked more than $100 million of revenue. And look, we continue to win, and we continue to execute in the marketplace.Doug: I would want to ask and maybe have you rehash some of your acquisitions because I'm just gonna assume that not everyone has followed the story in depth, right?Tony: So if you’d like to talk about OpenFive and then Banias and the thought process and what each acquisition brings to Alphawave and how they transition Alphawave into a much bigger company. So probably the best analogy for investors to think of us from a vertically integrated business model was a company called Inphi. Doug, I’m sure you remember you tracked Inphi before being acquired by Marvel, I think, for almost 10 billion dollars. You know they delivered the same 3 product types. They had their networking silicon. Most of their revenue came from silicon that drove fiber optics, which is what we're doing today.And there are two types of fiber optics; one leverages what's called PAM4 in the industry, and the other leverages coherent. So PAM4 is used within the data center. Coherent fiber optics is used to connect data centers. And as AI continues to grow, we need more data centers. They continue to expand, and so there's increasing usage of PAM4, but also coherent because you got to have these multiple buildings look like one single infrastructure, especially when you're training over a trillion parameters of data for LLMs like ChatGPT. But then those same hyperscalers that use your networking equipment also are building their own devices. I'll give an example NVIDIA is the clear market leader on GPUs, right, and GPUs are the dominant computer chip and semiconductor used for AI. But guess what? Hyperscalers have specific data trends and specific data patterns. And to deploy more to get higher performance and lower power, they are all building their custom silicon.So you can imagine hyperscalers in the future will offer a combination of NVIDIA GPUs, probably some AMD and or Intel CPUs, and their own optimized GPUs. And so when they're doing custom silicon, they need help. And so when they're doing custom silicon, they need help. I mean, virtually every advanced device today is using chiplets.OK, it's the only way that we can get all the functionality that we need onto a single die. It reduces power consumption by 30 to 40% and reduces manufacturing costs by more than 40%. So it's a compelling strategy. We also offer the intellectual property and the capabilities to deliver those custom silicon solutions. And finally, anyone building their own chip, can use our same Intellectual property and IP within their own chip.So the acquisitions that you originally want me to talk about help expand the business from the core IP. So OpenFive was a custom silicon provider. They had over a dozen chiplet designs pipeline and in the production pipeline. When we brought them in, they brought that custom silicon capability and that forms the foundation of Alphawave’s, custom silicon business unit. They also brought a lot of complementary IP, which expanded our IP portfolio from over 100 IPs to now over 220 IPs.And finally, Banias Labs, which we acquired late last year, came in on the product side. We had PAM4 technology, which is the product that is used within the data center. But we had a gap in our portfolio which was for the coherent fiber optics, which is used to connect multiple data centers together.Right now only Marvell and Cisco have that technology in the world and so with that capability combined with our Pam4, we now offer a full suite of networking products that can service all forms of data center inside.Doug: I would actually love to have like a concrete example for people who follow this story because I've always found semiconductors within the B2B context very hard to follow and understand. But recently you announced a 3-nanometer win with a North American hyperscaler.That seems to be on the SerDes side of the business. Could you give us an example of how that works? How do the customers come to you? And then maybe being able to sell more products after an initial relationship? How do you transition from just connectivity IP into a full custom product? Tony: That was a second-generation design with that customer and you can guess that hyper scaler what chip are they building themselves. AI is probably a good guess, right? The chip is used for generative AI like ChatGPT and so there's so much investment happening with these hyperscalers. They're transforming their data centers to evolve from a CPU-focused strategy into an AI-first strategy. By moving to an AI-first data center all the connectivity within the data center has to change. People don’t understand that in order to train the current generation of ChatGPT, which is more than a trillion parameters. It takes 10s of thousands of GPUs, and three months to train it. That’s often more GPUs that can be housed in any one data center. It's 10s of data centers and so now the number of connections because all these GPUs gotta talk to each other change. And so what does that mean? That means that the AI itself needs faster connectivity in order to train. A trillion parameters when the last generation was about 100 million. You need far more connectivity and you need it at lower power. That design doubled the data rates used for the connectivity.And look, this customer (North American hyper scaler) is at the forefront of bringing AI to the public. So it's, you know, it's amazing for me to know that my products are helping my own kids do their homework every time they log in and use AI.Doug: So these acquisitions changed Alphawave from mostly a SerDes company into an end-to-end chip company. Buying OpenFive gave you design capability and IP, and Banias gave you a coherent DSP capability. This scales you to an end-to-end provider of connectivity. But the problem is when you scale into all these acquisitions it creates like you know the headcount has exploded, right? The complexity of running the company has risen, and recently this has really shown up with the audit problems you had this year. And I think that that's the biggest part of your story right now. Recently Alphawave Semi got halted for a few days because KPMG missed your audit. That’s a big deal, and investors have been concerned, now more so than just the short report in the past. This audit mess is now a new nightmare for investors. But when you look at it a little closer, many companies in the UK missed their audit. I would love to hear your side of the story.Tony: So look, just to put some specific numbers in terms of the rapid growth over the last 12 months or so, 12 months ago we were about 175 people. Now we're north of 800. When we wrapped up our 2021 audit, we wrapped it up, I think just shy of $90 million of revenue. 100% of it came from silicon. In 2022 we closed the year with revenue 100% higher at 186 million in revenue and the revenue came from a combination of both silicon IP as well as actual silicon that we were selling.So we're a vastly different company and there's not a lot of semiconductor auditors in the UK and certainly, our experience with the auditors was that they don't have a lot of other clients in our space to learn and understand our business. We knew it would take a lot of work to help them understand the new businesses we've added.So we kicked off the process in late November, early December with the auditor. We had a new audit partner last year because the previous year there were also delays with KPMG. So rather than switch auditors, which is a major step given it was our first year as a public company, we swapped the senior partners.My second learning was that we deployed this year as we also program-managed it ourselves. Normally when I work with someone like a contractor that builds a home for me, they have their own program managers, right? They give me weekly updates in terms of status. I've found over the years working with auditors that they don't do this.So I put in my program manager to manage my supplier. But there we were at the end of April; everything looked like we were on track, and then with less than 48 hours left in the audit, we found out that our auditors needed more time. They believe they needed one more week. We did everything possible in less than 48 hours to try to keep it on track. Unfortunately, it didn't happen. And Doug, as important as being on time, being accurate is even more important. So we aligned with KPMG to delay which we did. But the delay pushed us beyond the 4-month reporting cycle required by the LSE which means you're suspended. That’s because they want the market to have audited results to have an orderly market. So we were suspended.It was just under two weeks in order to give KPMG the time they needed to complete the audit and all of their internal approvals. And look, we weren't the only company, there were more than a dozen other companies on the LSE in the exact same boat as us. I also get asked all the time by investors why don't just switch Auditors, well-being a UK-listed company, all of the Big 4 are swamped, including KPMG. I think it’s important to have a Big 4 accounting firm to provide confidence to our investors. And right now it's very hard to have any of the Big 4 take on any new clients because they're already so oversubscribed.So now what. What are we doing in order to rectify this?We have an action plan with KPMG where even though we only list and report on a semi annual basis, you know we're starting to operate like a US based company with quarterly closures of our books that are audit-ready quality. Implementing this rigor and working more collaboratively throughout the year rather than just at the end of the reporting cycles To help keep them educated on the progress of the business.So we're not back down to basics trying to explain what an EDA tool is to our auditors when it comes to the year-end that we can focus on the specific numbers and even more importantly they can trust us because they're seeing the progression of our business throughout the year.Doug: That 48-hour window sounded like a nightmare. And there's only so much you can do with that. Now you just announced a CFO transition and search. What’s the story there?Tony: So Dan decided on his own it was time for a change. Dan Baroni, who was our CFO I think it was a year before the IPO, right up to late April or early May of this year, was a great CFO. He was lucrative for taking the company public and for helping guide us through the three acquisitions. Dan had more than two decades as an investment banker. He was the lead tech investment banker for Barclays, so he was intrinsically in tune with UK tech. That experience, that respect, and the trust that he brought to the Company while he was here was invaluable.But as we transition to a vertically integrated semiconductor company, as we transition from doing 10s of millions of revenue a year to 100s of millions of revenue, we're now at a point where I think and Dan believed that we need a more traditional CFO with experience running a semiconductor finance organization.We’re global at this point. We're not just in the UK, we're not just in North America, but we're in Asia. We're in Israel; we're in EMEA. China, you name it.Bringing in that more seasoned tenure as well as relationships with auditors is going to help us continue to drive growth. That is the motivation. So Christian Bausher, who is the former controller for ARM is our Interim CFO. He's been with us for more than a year. Looking forward, we're a public company and the best thing for our investors is to have the best team around me. So we are running a search for the next long-term CFO. Christian is obviously a candidate. But we're running an extended search and I'm not going to rush this. Why? Because of the goal of a billion dollars by 2027. What I need is the right CFO with the right experience and the right mentality that is aligned with how I want to drive this business.We've interviewed and met dozens of candidates. We're shortlisting today. When I find the right candidate, I'll be very excited to introduce him to you and to the rest of the investment community.Doug: Well we're very much looking forward to that. I think it's an important part of your story and part of Alphawave Semi will grow into a much bigger company than it is today. Speaking about these long-term ambitions and the 1 billion in revenue by 2027, there seems to be a lot of levers to get there. The one lever everyone wants to talk about today is AI. You guys recently did a webinar talking about how connectivity is important in AI. I would love to hear your thoughts.It does seem like every other company is trying to say we're an AI company. But SerDes and the bandwidth problems of AI models do seem very helpful for your company. How will AI is going to impact you? Because as everyone knows, NVIDIA owns Mellanox, so Mellanox does their SerDes in house. So what's the opportunity set for you?Tony: It's a really timely question. Yesterday Barclays released a research note on the top 50 companies that will help drive and benefit from this AI transformation and obviously TSMC, Microsoft and others were on there.For example, just to train ChatGPT 4 you need more than 20,000 servers which with each server having 8 GPUs on that server. So you add it all up that's hundreds and thousands of high bandwidth connections just to train ChatGPT 4.Microsoft is spending 4 billion dollars to deploy the next generation datacenter for ChatGPT4. If you extrapolate it for Google and its search capability, it could drive more than $100 billion of investment. There's a keynote I'm giving next week at Samsung in regard to AI and how connectivity fits into AI. There's really two parts to a leading-edge GPU or AI chip. One part is the processor, the rest is the connectivity to other GPUs and to the net. Our core IP delivers that connectivity. And as the hyperscalers bring their own AI chips we could benefit.. Then the second question is, how do you deploy this?Up until 5 and 4 nanometers, it was all on a single chip. But as the number of processors increases, the size of the chips can’t. So in order to get more computing power and lower power consumption, you must move to chiplets. That's where our custom silicon business also comes in. We're helping to build the chiplets and the silicon for the next generation. And we’re focused on AI. Doug: That tracks. I want to ask you about the Samsung relationship because that seems new. You recently had a press release with Samsung and being a foundry partner. You’re a foundry partner for TSMC. Are you guys deepening that relationship with Samsung? Will you deepen the relationship with IFS whenever that comes up? I'm assuming the goal here is to be foundry agnostic, correct? Bring IP to any customer who wants to work with you.Tony: You are correct. That is the goal. Have our IP and even our custom silicon capability available to everyone. Even chiplets are available to anyone building their processor at TSMC, Samsung, or IFS. We’ve been a long-term partner with Samsung.They're one of my oldest and certainly one of our top customers. It's hard to be able to get approval to list who your partners and customers are. Samsung has been great. Samsung has been a customer back since late 2020. Our business with them has continued to grow.TSMC is the juggernaut, but Samsung is still a key player. They're the only other player with advanced 3-nanometer manufacturing available today. They are certainly winning in the market and they need high-performance connectivity and they need a strong North American custom silicon partner. And so that's you know that's really the focus of our relationship. Making sure our IP is there and secondly giving them a custom silicon provider that is equivalent to a Broadcom or Marvell that can help their customers build custom silicon and leverage chiplets for their next generation products. On the IFS side. We made an announcement with Intel that we were joining their IP ecosystem. And I know Intel well; they acquired my second business. I spent years helping the previous version of Intel Foundry try to win and succeed. And I think the opportunity was not there at that time. In 2015 to 2017. But now North America and the world needs Intel to succeed now. TSMC is building an advanced manufacturing plant in Arizona, Samsung is building an advanced fab in New York State, but we need North American process development. North American engineers. Given the geopolitical situation with US and China.You know closely keeping an eye and posturing around Taiwan, these uncertainties just heighten why we need to bring advanced semiconductor. Advanced semiconductor manufacturing back to North America. The key transition for Intel is to take its CPU-focused semiconductor manufacturing and migrate it to general-purpose semiconductor manufacturing. It's a migration. It's not the same thing.But we need them to be successful. I'm committed to it. I'm working with them. It's why our IP will be available in their technology and why I'm excited about Intel joining the semiconductor manufacturing fray yet again.Doug: I would like to ask a question that's hard to answer. Transitioning from a SerDes IP business into a full-stack vertical solution provider is frustrating because the market has a hard time evaluating your wins.You can’t announce your wins publically. Is there a future where we will know what you ramp, or how can investors keep score of how Alphawave is doing?Tony: The KPIs we use at our company is design wins and billings quarterly. We likely cannot announce all of our wins, because take for example Apple, our customers are secretive and don’t want to disclose their supply chain. Investors will just have to become accustomed with gauging our success in terms of quarterly bookings, the number of customers, and the areas we are winning in. Sometimes we are able to disclose a customer after the product is in production. I do expect we will be able to make more specific announcements, but it won't be at the front end of the win. Investors should expect it'll be as the semiconductor parts are entering the the production cycle, which is typically about two years after design wins.Other key KPIs or revenue, all right, I view revenue far more important to us than earnings right now. Investors track earnings carefully, but we are intentionally investing our earnings back into product development because it's all about scale. We will remain cash flow positive, and we’ve been profitable since day one. We will continue to be profitable, but the second most important KPI is revenue.Internally, the third is headcount. We continue to disclose headcount because headcount is also an indicator of our scale. How much are we investing into R&D? Is that growing? Is that growth sufficient in order to support the scale of the revenue growth. Lastly it’s earnings and adjusted earnings. so you know these are the four metrics. These are the indicators I use to manage the business at a macro level. I think these are the best metrics for investors.Doug: Perfect. I wanted to ask about the business shift toward lower-margin businesses. It’s been not very clear compared to your initial projections because of the OpenFive acquisition.How much more granularity will you give into the business and margins by each segment? How will gross margins and EBITDA for each of your three main segments be in the long run? Do you think think you're able to scale your EBITDA margin higher? How can you grow your business into higher-value segments, and how will you get there? Tony: This is a discussion I regularly have with investors. I can help give some color on the margin side. When we were a pureplay silicon IP business (SerDes), gross margins were greater than 95%, EBITDA was around the 50% range. But it’s a $1 billion TAM growing to a $2 billion TAM. Not sufficient to meet the ambitions of the business. So we added the custom silicon business, which brings NRE as well as silicon revenue. The business was mostly focused on older nodes, 28 nanometers, 16 nanometers, and consumer electronics.The NRE gross margin was around 30%, and the silicon gross margins were between the 20 to 35% range. Now what am I doing with that business? I am winding down that business to improve gross margins to a minimum of 30-ish percent gross margin.All of the deals in my custom silicon pipeline are 4 nanometers and below. That means now NRE gross margins are 50% plus. That means that gross margins are moving from the 20-30% range to the 45-55% range. The transformation on the custom silicon side is as that older business winds the margin goes up.This new business is far more advanced leveraging our IP portfolio, and has higher revenues and better gross margins. NREs are on the order of 40 to 50 million and margins are higher. They're on the 45 eventually north of 50% gross margin. The final business that I've added is the networking silicon business (Banias labs). This type of silicon has gross margins in the 75%+ gross margin range for coherent type of products. So 65%+ for the networking products in this business. So now let’s fast forward to 2027, and a billion-dollar business. I would expect somewhere around $400 million of revenue to come from products. That's the 65 to 75% gross margin mix.I would expect about 40% of our business coming from custom silicon at that point. I'd expect 45 to 55% gross margins there. And then the last $200 million coming from IP at 95% gross margins. So for investors trying to model it, this business from where we are today, where we guided 2023 to be, is nominally around 350 million with a 25% EBITDA margin. With this blend of portfolio and margins, we can get you know to a high 30s EBITDA business, possibly even a low 40s EBITDA margin in that time frame. That’s top tier. Doug: That's really helpful. I want to ask a question about transitioning a team focused on designing 28 nanometer chips to 4 nanometers and below. What are the challenges there? Designing a chip at a much smaller scale takes a lot more people and experience. It’s harder one of the most confusing parts of the story.The custom silicon business is likely not an ARM kind of business, rather it’s more of a die to die business and focused on connectivity and other IP. Not the traditional logic businesses. Could you give us examples of things you are pursuing in the custom silicon business?Tony: Sure. First let me talk about the team. I brought in a team of a couple of 100 to 200 people purely focused on custom silicon. But I added to that team, and now it consists of probably north of 100 people located here in North America and also a chip-building team from Israel. So that team is responsible for building the custom silicon of our customers and as well as our internal customers. So they have experience designing on 3 or 4 nanometer products because that’s where our products sit at today. And my Israel team, my North America team are at the forefront of new nodes. We've worked on every TSMC node and tapped out on every TSMC first shuttle since the company was formed.As well as Samsung as well as Intel. I've aligned the organization so all of the chip-building teams use One methodology in three nanometers and below. And that methodology is driven by all three groups, but it's architected out of North America, where the hyperscalers are. So we know how they build chips and we make sure our methodology is step with them.Doug: Examples of the products that you’re doing on the custom silicon sideTony: Let me talk about one example specifically. When I acquired Banias, we focused on Coherent, a technology to connect datacenters over longer distances. That’s a product that we're bringing to market but its not being used for just data centers, it's being used in telecom.So you have the Major Telecom players like Nokia, Ericsson that build their own custom silicon in this space. It's these types of customers that have been coming to us now that we have this coherent capability and are asking us to help them build their own chips moving forward.Coherent connectivity is a combination of advanced analog as well high performance, high speed digital signal processing. You need experience with ARM and experience with RISC-V and experience on the leading nodes. Everyone is focused on driving down power consumption while bandwidth continues to double every two to three years in our base stations and datacenters. That’s one example beyond AI, which is burning super hot, which is probably where more than 60% of my pipeline resides today. But this highlights the incremental value that the Banias acquisition brought beyond our internal products and the hyper scalar wins for our internal products. It's also driving further custom silicon revenue at the most advanced nodes.Doug: That's that's an amazing answer. So wrapping this up, Alphawave has definitely been a bumpy road to get here. You’ve positioned yourselves a lot better to become a much bigger semiconductor company. So what are your longer term ambitions other than the revenue goal. How are you going to build this company bigger. I would just like to give you a platform to get your message out there to investors. Tony: Doug, that's amazing. It's not often I get to wrap up with that type of platform. Simply put, we will be the next great semiconductor company. Like a Broadcom or Marvell. And how are we going to get there?For the first decade, we'll be focused on connectivity. Beyond that we'll continue to expand. We're an engineering first company. And no one has ever challenged our technology and our technology leadership. The shorts and everyone else have tried throwing rocks at us have been on the peripheral, but never at the technology. That's because I'm an engineer by trade. All of the leadership team, or the vast majority of the leadership team are engineers. And if technology leadership is critical to our success, that means our engineers are just as critical because they're the ones building the tech. So we know who are customers are and we focus on them. Our engineers drive innovation, and my job as CEO is to set that north star and everytime I get with my people I remind them to not look at the day-to-day stock price, but to focus on building the next great semiconductor company of the industry. Follow the plan and success and ultimately the share price will reflect our success. And so I'm super excited to be where I am today.This has always been the goal, but to see the goal beginning to materialize and getting to scale is rewarding. But look I'm a young guy. I'm still in the middle of my 40s. I have a lot of energy and vigor left, and I'm going to use all of that energy and vigor to push this organization higher. I hope the innovation and passion has come across in this conversation. And from my perspective, we’re a steal today. I hope shareholders listening today know that I am the number 2 shareholder in the company behind the Sutardja family.I’m aligned with my shareholders. Every decision that's made by myself and my other C level executives are made with the same objective. And it’s not to drive share price in the next 24 hours, but to build a long-term and sustainable business. Doug: At the end of the day, the results will speak for themselves. You’re clearly an engineer. And you’ve already built this from almost nothing. It’s been a cool story from that perspective. Before I go I want to talk about the share dynamics and the inability of you to buy back shares. Can you speak on that? Because that’s been part of the story. The Sutardja family has been buying shares, but why can’t you buy more shares?Tony: This is one of the unique differences of the LSE. Me and my cofounders who funded this company, we didn't have traditional VC's at the time of the IPO. We owned about 48.5% of the company. The LSE and FCA who overseas the policy of the exchange has this takeover panel. And they view me and my cofounders as a concerted party that could operate together to takeover the company. So we are not allowed to buy a single share of the company that would push us to the majority holding position. We’ve gone to the FCA numerous times out of our own pockets to appeal this, and each time it’s been rejected.So we constantly get asked why aren't we buying more if we're so passionate about the growth and the value of the company and the simple answer is we're not allowed to. But the Sutardjas aren’t a concerted party so they can. They believe in the company. They believe in the leadership and certainly they know how to build successful companies because they founded and led Marvell for more than two decades. They're buying and they continue to buy.Doug: Thanks so much for the time. I appreciate you having the opportunity to tell your story and I'm excited to see the continued results and execution because that’s what the market is looking for the most.The shares reflects low confidence, with the short report to the suspension of shares. Investors obviously want to see results. And hopefully on a quarterly basis. Any other last words, Tony?Tony: Doug this has been great. I’ll be more than glad to come back regularly and periodically give you updates on the company. I appreciate the factual based writing that you have done in the company. It’s been a pleasure. Take care, Doug.Doug: Take care Tony. That’s it for today! I was excited for Tony to come to the newsletter. It’s a frustrating story, but the Q1 results have been solid, and their positioning in theory continues to be best in class. The shares trade like it’s a going concern, but eventually one day the market will not view it as such. Until then. Get full access to Fabricated Knowledge at www.fabricatedknowledge.com/subscribe

Today’s newsletter is an interview with Rajesh Vashist, CEO of SiTime. I’ve written a bit on SiTime, first an overview of the timing market and a piece on SiTime specifically. Today I got Rajesh to talk about SiTime’s history and opportunity. This interview is lightly edited for readability, but the audio file is above, and I’ll be publishing the interview on podcasting platforms and YouTube. I’ll write some thoughts at the end regarding valuation. But first, I hope you enjoy this interview because I certainly did.The Story of SiTimeDoug OLaughlin: Hey Rajesh, it’s good to have you on Fabricated Knowledge. I love the story that’s going on at SiTime, and more people should hear and know about it, so I’m happy to have you here to talk about it. So, what’s the SiTime story, what does SiTime do, and what are you trying to accomplish? Rajesh Vashist: Hi, Doug. I’m happy to be here. SiTime is an amazing story of determination, focus, and technical prowess, but it’s also a great story of identifying a market and focusing on it. A little-known timing market which has been on a quest for more accurate timing under tough environmental conditions. This has been ongoing for several hundred years, starting with the invention of the chronometer in the 1700s. SiTime takes that quest to a new semiconductor level and solves the world’s toughest timing problems with silicon. It’s an $8 billion market, and it’s been growing at 5% a year for the last 40-50 years. Ten years from now, it will be double that, so a $16-20 billion market, and SiTime has carved out a niche in the highest growth portion of the $8B market, with a cumulative growth rate of 30-50% a year, called precision timing.Precision timing is a category that SiTime invented. Precision timing is high-end timing in tough environmental conditions like 5g, mobile, data center, cloud, automated driving, and personal healthcare. We know these are becoming more processor-oriented and more connected, often demanding smaller, lighter, and lower power form factors. SiTime hits all these requirements by leading timing solutions for those markets.We went public in 2019. But we’ve been around since 2005. We are a public market cap company with a few billion in market cap, depending on analyst estimates, anywhere between $250 and $300 million in revenue. SiTime is highly profitable as well. Doug: Before we continue, I would love to step back to the beginning.SiTime was around in 2005, and important to this story is MEMS. In 2005, you were among the first MEMS startups, but you weren’t the only ones. Many competitors were trying to achieve the holy grail of MEMS timing. Why did SiTime win, and where are those competitors today? Rajesh: You’re right. I think there were six startups, including SiTime. Even more interesting is there were three large semiconductor companies. Silicon Labs $SLAB , IDT, and Maxim, now part of Analog Devices $ADI . So six companies together spent $400-500 million over an 8-10 year period, and only SiTime was the winner. We are now the only scale MEMS timing company. So let’s talk about MEMS technology before we go into why SiTime won.What is MEMS? MEMS or micro-electronic mechanical systems are exactly what it sounds like. You and I are used to talking about circuits. In my past life, I used circuits in semiconductors. But I had never used moving parts in semiconductors. In other words, MEMS uses semiconductor processes to create moving parts. It’s already being done in gyroscopes, filters, and various sensors. The founders of SiTime, Aaron Partridge and Marcus Lutz used this technology to build vibrating resonators to replace a 50-year-old technology, quartz crystals. That was in 2005, before my time at SiTime.Quartz crystals have been used in almost all electronic solutions before the advent of MEMS. By and large, most systems still use quartz crystals. The MEMS promise is that it is semiconductor-based, bringing scale, smaller size, lower power, manufacturing scale, and programmability. And, of course, it’s a semiconductor. We know that whenever semiconductors go up against non-semiconductors, such as transistors versus vacuum tubes, spinning hard disk drives versus solid state drives. Semiconductors always win.There’s never been a technology like that in the history of humans. Everyone knew that, but the tricky part was making it into a product, productizing it, building it, and making it work at scale at 3 billion units, not at 50-100k units. Making MEMS work with 95-98% yields at scale. So it turns out SiTime is the only company that did it. And the speculative reason I think we did it is that most semiconductor companies did not understand that MEMS requires its own process technology. Most people go to fabs like TSMC or Global Foundries and use their processes. MEMS requires the development of its own process. SiTime developed and owns its process. The second thing is that this is a mechanical system. It’s electrical, but it’s also mechanical. In other words, we are talking about material science, physics, and moving parts. Not a strength of a classical semiconductor company. Finally, there is the design element and the design methodology. In semiconductors, if you want to design something, you buy tools from Synopsys, Cadence, Mentor, or someone else. Here you have to build your own tools. SiTime heavily invested in our own tools for design. Putting it another way, we have three barriers to entry. We own our process, understand material science, and build our own tools. I don’t think the 8-10  companies that we competed with did all that or did it in a concentrated way. But SiTime did.I arrived at the company in 2007, when it was pre-revenue with 30 people, and we were one of six startups and three other large companies. And the net result of all this was by 2013-2014, cumulatively, the 10 of us had spent $500 million, and only SiTime remained. This happens often in MEMS. Generally, those in other MEMS technology, like filters, when they win, they are the only ones standing. Broadcom, old Avago, and old HP invented filters; they are the only ones standing 20 years later. Bosch invented gyroscopes and accelerometers, and they are the only ones standing 20 years later. TI invented DLP; they are the only ones standing 20 years later. It’s no surprise that after this much time and money spent, most companies ceded the market to SiTime. Doug: That’s the perfect explanation. I didn’t realize you had to make your own MEMS tools. That’s a big endeavor for almost any company, and that is why you guys are the last ones standing and the leader in MEMS resonators. I want to now ask about MEMS vs. quartz. Near term it’s about replacing older technology, but in the longer term, there is an opportunity to create your categories because MEMS is a new technology. MEMS allows for new form factors and precision, and the roadmap is better because of its silicon. So I would love to hear where you are making new categories and why some of your customers have intentionally gone all-in on MEMS very early. These companies could have used Quartz, but they chose MEMS, so why is that? Rajesh: To size it for you again, it’s an $8 billion market, and it was a $4 billion market a decade ago. It will be ~$16 billion market in ten-ish years. SiTime is only a ~$300 million revenue company today, give or take. Even if we were to grow rapidly to get to a couple of billion in revenue by the end of the decade, we would be a small percentage of the market. The point I’m making is that our goal is not to replace the legacy technology, but our goal is to recognize the needs of new markets like 5G, datacenter, automotive, healthcare, advanced military specifications, high-end industrial, and IoT. These market’s needs are not being fully addressed, so instead of returning to legacy use cases like PCs and Laptops, we are jumping ahead to new use cases. We found over 300 new use cases for our chips at the last count. So when we go to new use cases, we are in a world of 80% single source, in spite of the fact that there are 30-40 crystal companies and some semiconductor companies that are doing clocking. SiTime becomes the single source 80% of the time, and this is because we have discernable benefits to the customer because no one wants to be forced into a single source component. So if you look at the growth of timing, you can see what attracted me in 2007 to SiTime. I had a previous successful career building a company called Ikanos communications and exited in 2006 when it was half a billion in market cap and $150 million in revenue company. Ikanos also had a 95% market share against Broadcom, ST, and TI, but I left that business because the market was not scaling and was not big. My thesis is that the way to establish large companies is to grow into a very large established market and bring a dramatically different solution. If you look at urban transportation, which has existed for 150 years, Uber entered with a dramatically different solution and disrupts it. If you look at advertising, along comes Google with significantly better technology, which disrupts it. Ditto with retail and Amazon. The thesis is that if you have a large existing technology, you can pick the use cases to which you can bring value and expand from there. We are following the trends, and with that, you can build a great company. As I said before, my previous, Ikanos did not have that luxury. While we were a 95% market share, that was the whole market, and the market wasn’t growing. That’s why I exited from taking it public from 0. Meanwhile, when I came to SiTime, that was the biggest benefit I saw apart from MEMS and the great team. It was our great strategy and opportunity. Since then, it has taken us the first six years of existence (2007-2013) to move away from a me-too product against quartz and truly understand the market. Those were tough years. 2008 and 2009 were challenging for a company with no revenue. We were a company backed by NEC and Greylock, but we were on our own to raise money. Luckily I was able to raise ~$100 million through the life of the companies to 2013. And 2013 was our first new category product launch. Before that, we were me-too; our generation 5/6 products, which still ship, were me-too products. But after 2013, every product has been a new category or dramatically better than the competition. At that point, it was a 32-kilohertz product, but a 32-kilohertz TCXO. Meaning it was 10x more accurate, had 90% less power, and was 1/20th the size. It was an exceptionally differentiated product picked up by our largest customer today. It was a watch at that time, and the world had not seen a smartwatch, and I’m happy to say we have been in every subsequent watch since.The lesson here is you can go into a large market, but you only win with dramatically differentiated products. Since then, we have a product called Elite, which is differentiated, and we have been able to win customers as diverse as Nokia, Google, SpaceX, Tesla, and so on. Since that 2013 time frame, we have introduced a dozen new products, out of which half have been category creators, and the others have been dramatically better than competitors. Our central thesis is “it’s the product stupid.” Get a product that is fantastically differentiated and that customers care about. Customers frankly cannot wait to get it from you; they don’t care if you’re a small company or not; they want that product. If you go back to 2013, we were a small VC-backed and money-losing product, and one of the world's largest companies adopted our product as a way to differentiate their first category-defining product. It’s been a real template for success, and we have tried to replicate it, and we have done tremendously since then.Doug: Perfect! I want to break apart a few of those statements because, during that period, some interesting things happened at SiTime. The large customer was instrumental to your growth, but also, at this time, you were acquired by Megachips. Was that a platform for reinvestment in the business? Because let’s say the eras of SiTime, 2007 to 2013, were scrapping by and trying to become better than just a me-too product. At the same time, Megachips was from 2014 to 2019, a completely different era of your journey where you began to take off. You had the 2013 watch product, and then you were acquired by Megachips. What was it like at Megachips? Rajesh: In 2014, many of the large semiconductor companies started to recognize that SiTime had made the final breakthrough in MEMS technology. And we were still small, around 20 million in revenue, but most felt it would be a nice bite-sized acquisition. We got multiple offers that recognized the value of SiTime, and they were 10x revenue. That was an astonishingly high multiple of sales for a company that wasn’t making money, and it was too good to pass. We thought about our investors and our team, and we decided to sell. Picking Megachips was, in retrospect, the best decision. When I met with the people there, I realized that they were real partners and they had their eye on the long-term future. Which sometimes in the United States doesn’t always happen. We chose them because we thought they would leave us alone. They would understand that we understood this market and technology and didn’t need help. I have seen many times large companies come into small companies and think they can do a better job, and instead of helping it, they make it worse, and everyone leaves. Megachips was brilliant in leaving us alone.Because they left us alone, our team thrived. We had no one leave the company. There was no strategic help because they could not help us with technology, and our channels were different, but because we were part of a larger company, we were better funded for the first year or so. That was also when we got into a very popular phone, and our sales went from $20 million to $100 million in three years. Consqeuentaily Megachips benefited. Their stock went from $10 to $40 as a consequence, which had not happened in a long time.We learned then the benefit of a highly differentiated product. We accelerated our move into differentiated categories and used the Megachips funding to accelerate the timeline. We also came to the conclusion that while the Megachips platform was a good one, it still wasn’t big enough to contain where we should grow. We felt that SiTime should be a public company, and we proposed that to the Megachips chairman, and to their deep credit, again, they showed wisdom and let us go. Consequently, SiTime went public in 2019, and in the first two years, they took out $600 million on their $200 million investment, and they still own 25% of SiTime today (~$625 million). That’s a great investment. They took 200 million dollars to a little over a billion dollars. And it’s been great for SiTime employees because we can attract the best of the best, and we have been able to retain our team. And our investment rate has quadrupled in our business. At this point, the amount of investment and R&D in markets and channels we have been able to make is several orders of magnitude more than anyone else solely focused on this market. It’s been an amazing journey. We don’t know many companies that sell themselves, thrive as a subsidiary, and then get to go public. It’s kind of a fun journey for the company and me personally.Doug: Very few companies get acquired, flourish, and go back out. I can’t think of another example when you put it that way.During that time period, you got into a very popular phone, and there was a bit of a crisis at SiTime. To get technical, the way it’s packaged, it’s hermetically sealed, but small molecule gases can get inside. There was this whole “helium-gate” at your customer, which wasn’t easy to go through as a supplier. How did you mitigate that crisis, and did you adapt? Because that’s a hard problem, impacting the majority of your revenue. Rajesh: That’s another wonderful story. A wonderful story now but it wasn’t a wonderful story then. Because it ended well, I can laugh and talk about it now. We were in a popular phone, and as you pointed out, our MEMS chip is encased in a perfect vacuum with one exception, the smallest molecule in the universe, helium. We did not think to make it impervious to Helium, so when Helium ingresses, the gas pressure stops the MEMS device from vibrating from inertia and friction, and when Helium leaks out of the cavity it goes back to action. We did it for every other molecule and not Helium because Helium is not naturally found on earth. Helium comes to the earth from the stars, it’s a planetary acquisition, it’s not a native gas and there’s very little of it on earth. We should have, but we didn’t. We asked our customer if it was important, and we didn’t do a good job of interpreting the answer, and it created a problem for our large phone customer down the road. We were not given the chance to fix it in a timely manner.The tragedy is that we could have fixed it in a very quick time. It’s not a difficult problem to solve for SiTime, but we solved it too late for the generation of that phone. The reason that we get to laugh about it is that it helped us learn about depending on any one large customer, no matter how wonderful or desirable they are. Since then we have branched out completely, and from 2018 when this happened, our revenue was ~$85 million and heavily dependent (70% of revenue) on this one customer. In 2019 we were exactly the same amount of revenue, but the large customer went from 70% to 35% of revenue. We learned painfully that we needed to depend on a broader customer base, and kudos to everyone at SiTime for executing it so quickly. Today that large customer is a smaller part of our revenue. At the same time, our revenue has grown to $250-300 million (depending on which estimate); this large customer is less than 30% of revenue, despite all that growth. We are in a wonderful place, and our products are now helium insensitive, so it’s a good story all around.Doug: I presume you won the phone back?Rajesh: We did get one generation, one SKU, but we have so much of the business in networking and telecommunications that is our number one market; we are much more focused on that today.Doug: I was going to ask about that. Now that we have told the story of how SiTime came to be, I am curious about the future of SiTime. There are so many interesting niches and products, and when I visited SiTime and talked to people who worked there, they gave many examples of how you’re differentiating.But I would love to hear about what your biggest opportunity sets are. It sounds like one of the biggest problems is choosing. You have all these slivers of potential markets, but you must focus on each one at a time. What is your biggest opportunity set in the future, how do you make that market yours, and what are the challenges to get there? Rajesh: Our servable addressable market that we serve when we went public in 2019 was about $700 million. Today is double that at $1.5 billion, and by the end of next year, 2024, it will be $4 billion. By delivering new products faster, we can capture more of the served market than we can today. We expect that 10 years from now, that $4 billion SAM might be as high as $10 billion. In other words, the world is our oyster, and we can go after different portions of the market with different products and different value sets and grab those. Out of the $4 billion to come, we have identified networking and telecommunications as the largest market. We think that will be ~$1.4 billion dollars by the end of 2024, as big as our current servable addressable market today. That market is a wonderful place to be because we have new opportunities for 150 different kinds of chips. We are in long-haul, back-haul, front-haul, radio, RRU, the 5G base station, the small cell, the data center, and the server; we are in all of that. And that’s sensitive to high performance. As you know, latency and high performance are erupting; that is where Microsoft and Amazon play. All of the cloud areas are critical. 5G is not only being used by telecommunication companies, but large enterprises are creating their own native 5G networks. So this is the market for us because the average selling prices are high. Prices are between $3 to $30. The gross margins are high. Our corporate gross margin is ~65%, and in those markets, our gross margins are even higher. The business is very sticky, and lasts around for 2 to 4 years, unlike consumer or mobile which changes every year. And we bring exceptional differentiation to the market. So now we have a cluster of customers, anyone you could think of in this market, who are our customers.We play in industrial, military, automotive, mobile, and consumer, but if I had to zero in on one market to win, telecommunications is our happy hunting ground. I would say our focus is on that market. And 80% of our new products are focused on this opportunity. Doug: Well put. In this inventory cycle, there’s a bumpier ride than the last few quarters, or rather the next few quarters. But I am trying to take a bigger picture step back. How large are we talking about these markets could be for timing? Within telecommunications, 5G is a smaller part of the whole but growing faster. And within timing, SiTime is a smaller part of that market, but growing faster. So is it reasonable to say you could sustain very high growth rates for a decade? Could you do north of 20-30% for a decade as you pursue this massive multi-billion opportunity? When looking at SiTime, you can see the past growth, but you look at the future and reckon SiTime thing can grow for a long time. Do you think when you hit the $10 billion dollar SAM you’ll be a meaningful percent of that market?If yes, this is billions of dollars of revenue, so I would love to hear how you get from $200-300 million dollars today to a billion or even the next billion. Rajesh: Good question. When we went public in November 2019, we told the world we would grow at 20-25% annually. Then COVID hit, and the world grew upside, and we grew at 65% and 88% in 2020 and 2021. And even in 2022, a choppy year in the second half, we will likely grow at 30-35%. What we have told the world and investors at large is that excluding the inventory correction, which will take 2-3 quarters, SiTime expects a 30% growth rate year on year for the next five years. This is doable and well within our reach.Some people tell us we are being too modest. Perhaps we are. But we will know more when we get through the second half of this year. But at this point 30% growth rate seems like a very achievable rate for us. One thing people always ask us is if you’re so differentiated, why are not 40%+ growth rates possible then? And the reason is I really value profitable growth along the way. When we went public we were at a 46% gross margin, today we are a ~65% gross margin business. Our gross margin rocketed 20% higher, and when we went public, we were not profitable, but we have told the world we have a corporate view (longer-term) of 30% net income. So that makes us very profitable company. As you know, today, we have $600 million in cash with no debt. I like that profitable almost analog-like model; I am not willing to trade profitability for growth. The world is a choppy place, and we have seen recently what happens to those companies that say growth at all costs. However, I can say that it’s possible when the SAM is $10 billion that, SiTime could be a couple billion in revenue. Those are enormous numbers and enormous growth. And that’s why SiTime could be like a Texas Instruments or Analog devices story, a 50-year company from here on out. Because timing was first invented in 1720, which was 300 years ago, in that sense, this is an early story. And that’s what I love about SiTime, SiTime sings one beautiful song, and we sing it really well in various ways, and that song is timing. We specialize in timing and solving the world’s tough timing problems. We have no boundaries to this ambition.We want to be in the future with timing. With atomic and quantum timing. Already our smallest chips are 1.5 x .8 millimeters, but we would love to be selling chips that are .5 x .5 millimeters. We would like to bring a $4000 atomic clock and bring down the price to 20-30 bucks. We would love to get into the software side of synchronous networks. We think that timing is such a beautiful big gold mine that we have built barriers to entry around, that our job is not to go wandering outside of the goldmine but to keep digging because there’s a bunch of gold in this mine. Doug: That’s a wonderful analogy. You have a huge opportunity to execute. Another question I want to ask about is the pricing aspect.One of the big fears is because of the AKM fire and the price shocks in quartz that there was a one-time shift in MEMS that will come back when prices go back down. Yet you are 80% single-sourced. Do you think you could lose some of that business when pricing normalizes? Rajesh: One of the innovations in our business models that we have focused more on in the last couple of years is that we tell our customers we are a premium-priced product. We approach our customers telling them that we are selling them a BMW or a Porsche; we are not selling them a Kia. And Kias are perfectly fine cars, but you can go buy that if you’re not interested in a premium product. Quartz is a perfectly good fine product for many things, and out of an $8 billion market, we are only selling $250-$300 million in revenue today. So by definition, most of the market is quartz, and what we sell is differentiated by a higher price and higher value. That’s why customers seek us out for a reason. Because we typically sit in their higher-value products which we can differentiate.And when you have a very high-value product, like a satellite or a digital processing part, or a very high-value consumer or medical product, you need a high-value component like SiTime in there. We are comfortable being a premium single-source product, and this is not a market share gain story. Those who think about SiTime taking market share from quartz are not quite thinking about timing as we do. We are servicing the hard-to-do markets; sometimes, that’s taking market share, but sometimes that means creating a new product category. Remember I told you about our twelve new products; how six of them didn’t exist as a category before. A TXCO that works like an OCXO didn’t exist, an oscillator that looks and feels like a resonator didn’t exist. A TXCO in 32 kilohertz did not exist. These are categories that we have created, and as you know, category creators get to harvest more value. Doug: I’m going to ask a final question along the profitable growth premium product vein. Something so interesting about MEMS is that you worked to become better than a me-too product, but the MEMS roadmap still has a long way to go. In theory, as the technology matures, do you think you could address lower price products that are not as profitable today and that will expand the SAM by addressing larger parts of the market more profitably because of the silicon roadmap? That’s interesting to me because you can have your cake and be profitable today and, as you scale, still have high gross margin products as you address the rest of the market. Rajesh: Exactly. First, we sell products at 10s of cents. It’s premium priced in its category. Using hypothetical prices, if there were products sold at 20 cents, we would sell at 25 cents. If it were sold at 25 cents, we would be at 40 cents. We are selling premium priced products.But we see a day where we play in resonators, a market that sells for 10s of cents. That is a $3 billion odd market we don’t address today. SiTime can see a way of replacing the need for resonators that are better than quartz, smaller, more scalable, and more reliable, and we would again do it at a premium. So if 10 cents, we would sell for 20 cents.But the good part of that story is in the volume story, about 40 billion units of resonators are being sold a year. And that number is even growing at 5 percent a year. There are still a lot of markets to address. Another example is clocking chips, which until two years ago, we had no clocking chips. These are not oscillators or resonators; semiconductors companies make them. SiTime introduced our first product a year and a half ago, and at the end of last year (2022), we gained 200 customers in that market. Where ever we show up, we win. That is a $1.5 billion market, where we have multiple product introductions coming. “Where ever we show up, we win.”So yes, we see a lot of potential untapped markets like quantum timing and synchronous timing software. Or positioning and navigation precision timing. We see all of that as part of the market we are addressing.We are in the very early innings of establishing a 50-year-plus-long company, and as someone who has been at SiTime for 15 years, it’s been gratifying to see the company's growth. Not only the growth but our culture. From the time that I arrived in 2007, there were 30 people. Of those 30, 15 of those people are still at the company. A tenure of 10 years is very common at SiTime, which is shocking given the churn in the tech industry.Doug: Well, I’m excited to see you execute that opportunity. There is a special energy when I went and visited SiTime. The story, the culture, and the opportunity are all special. I can’t wait to keep watching. Thank you, Rajesh.Rajesh: Thanks. Take care. Wow, that was quite the interview. If you want some thoughts on my site visit and model changes, I’ll have a quick follow-up email for paid subscribers only. Thanks for reading. Please share this, as that would massively help both SiTime and me. Thank you!Refer to these posts for the whole writeup, model, and thesis.If you found this helpful - also consider subscribing! Your support gives me the ability to offer better quality products like this. The more support, the better access. Get full access to Fabricated Knowledge at www.fabricatedknowledge.com/subscribe