<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[tensorViz]]></title><description><![CDATA[tensorViz]]></description><link>https://tensorviz.hashnode.dev</link><image><url>https://cdn.hashnode.com/uploads/logos/6acb2fa3e1160e0cd93805a4/d58165bb-b912-4c30-bd7f-8b6f078913ef.png</url><title>tensorViz</title><link>https://tensorviz.hashnode.dev</link></image><generator>RSS for Node</generator><lastBuildDate>Fri, 16 Oct 2026 15:35:00 GMT</lastBuildDate><atom:link href="https://tensorviz.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Read a large PyTorch model one block at a time]]></title><description><![CDATA[A graph with thousands of nodes is hard to read when every operation is open at once. Start with a smaller question: what does this block take in, and what does it pass to the next block?
This walkthr]]></description><link>https://tensorviz.hashnode.dev/read-a-large-pytorch-model-one-block-at-a-time</link><guid isPermaLink="true">https://tensorviz.hashnode.dev/read-a-large-pytorch-model-one-block-at-a-time</guid><category><![CDATA[pytorch]]></category><category><![CDATA[Machine Learning]]></category><dc:creator><![CDATA[tensorViz]]></dc:creator><pubDate>Sun, 11 Oct 2026 06:49:50 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6acb2fa3e1160e0cd93805a4/7e13c2ca-3985-4054-becc-cb03d64bd4d6.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>A graph with thousands of nodes is hard to read when every operation is open at once. Start with a smaller question: what does this block take in, and what does it pass to the next block?</p>
<p>This walkthrough uses TorchVision's Swin-B in tensorViz, a free VS Code extension we build. The recorded capture has 2,964 graph nodes. It follows one route through the model: feature stages → a transformer block → its MLP → a linear operation.</p>
<p><a href="https://www.youtube.com/watch?v=u-lx58ascmw">Watch the recorded walkthrough</a>.</p>
<h2>Keep an overview you can return to</h2>
<p>Begin at the stage level. Identify the sequence of major modules before opening their internals. On Swin-B, the transformer stages contain 2, 2, 18 and 2 blocks. The long stage is exactly where an overview helps: you can find one block without reading the operations inside all 18.</p>
<p>This is a useful habit in source code too. <code>model.children()</code> gives direct child modules. <code>model.named_modules()</code> walks nested modules recursively. Starting with the direct children keeps the first pass manageable; inspect a particular nested module when you have a reason to go deeper. PyTorch's <a href="https://docs.pytorch.org/docs/stable/notes/modules.html">module guide</a> documents the distinction between immediate and recursive traversal.</p>
<p>Registration order alone does not prove runtime data flow. Read <code>forward</code> and use a capture for the inputs you care about when that distinction matters.</p>
<h2>Follow one path, keeping the module names</h2>
<p>In the recording, the route ends at <code>features.5.0.mlp.0</code>. The path carries useful context: an operation belongs to an MLP, inside one block, inside a feature stage. Keep that context visible while you inspect the details.</p>
<p>In tensorViz, open the relevant stage, then one transformer block. Open its MLP only when you want to inspect that part of the computation. The breadcrumbs give you a route back to the parent. This lets you answer a local question without losing where the module sits in the larger model.</p>
<p>A practical stopping rule: if the current view already answers your question, stop expanding. For a block-boundary question, the block's inputs and outputs may be enough. For a projection question, descend into the MLP and select the linear operation.</p>
<img src="https://cdn.hashnode.com/uploads/covers/6acb2fa3e1160e0cd93805a4/e743e56f-cefc-4650-ad81-688e074f6d6c.png" alt="Swin-B transformer block in tensorViz, with module breadcrumbs and captured tensor shapes" style="display:block;margin:0 auto" />

<p><em>One transformer block, with its place in the model kept visible.</em></p>
<h2>Read tensor shapes in the context of the capture</h2>
<p>Once you reach the operation, inspect its captured inputs and outputs. Keep the example input beside your notes so a shape is not separated from the run that produced it.</p>
<p>This capture used CPU evaluation, random weights and one float32 image tensor of shape <code>[1, 3, 224, 224]</code>. No pretrained checkpoint was downloaded. The graph helps inspect this execution; it is not a claim about model accuracy, training behavior, every input size or every possible branch.</p>
<p>The walkthrough also uses previously visited scopes with restored camera views. It demonstrates navigation and context, not a benchmark of first-open camera placement or performance.</p>
<h2>Use the graph to decide what code to read next</h2>
<p>After finding the module, inspect its Python implementation and the operations in <code>forward</code>. The graph gives you a concrete place to start. The source explains the computation, including details that a compact diagram may leave out.</p>
<p>For a different model, the same sequence works as an inspection method:</p>
<ol>
<li><p>Name the question and the example input.</p>
</li>
<li><p>Find the major stage containing the relevant module.</p>
</li>
<li><p>Open one block, keeping its parent path visible.</p>
</li>
<li><p>Compare the captured input and output shapes.</p>
</li>
<li><p>Read the corresponding source, then return to the overview.</p>
</li>
</ol>
<p>Whether a particular model can be captured depends on its implementation and setup. Keep unsupported paths or missing context visible instead of treating a diagram as a complete proof of behavior.</p>
<p>You can <a href="https://tensorviz.dev/explore/?utm_source=hashnode&amp;utm_medium=organic_social&amp;utm_campaign=launch_202610&amp;utm_content=one_block_blog_v1">explore browser examples</a> before installing anything. For local models, <a href="https://marketplace.visualstudio.com/items?itemName=tensorviz.tensorviz-vscode">tensorViz is on the VS Code Marketplace</a>.</p>
<p>This topic was informed by a <a href="https://www.reddit.com/r/pytorch/comments/1kou26e/comment/mtesdgf/">public r/pytorch discussion about graph size</a>. Those comments concerned another tool in 2025; they are not tensorViz testimonials or a comparison with that tool's current behavior.</p>
<p>Disclosure: published by the tensorViz team, with AI assistance. The workflow and capture details were checked against the recorded demonstration and its source manifest. The recording was made with a development build.</p>
]]></content:encoded></item></channel></rss>