Zola 2023-07-28T00:00:00+00:00 https://suchprogramming.com/atom.xml 2023-07-28T00:00:00+00:00 2023-07-28T00:00:00+00:00 https://suchprogramming.com/mad-computer-science-part-6/ <p>Trying to teach myself hardware design was a monumental pain in the ass compared to learning software engineering, but today I want to tell you about 4 projects that are paving the road for anyone to go beyond telling computers what to do and to start telling them what to be!</p> <h1 id="hardware-built-for-anyone-and-everyone">Hardware Built For Anyone and Everyone</h1> <p>Let's start off with the idea that we don't want to pay a couple hundred dollars to buy an FPGA board that we have no clue how to use. The board I used to recommend was the Arty A7-35T, which has great publicly available documentation and a pretty nice set of on-board peripherals for about $150 USD. That model has been discontinued though and only the higher trim model is still available at almost double that price. I still like my Arty quite a lot but deprecated already describes way too much of my experience with logic design for FPGAs.</p> <p>A couple months ago I saw the <a href="https://pico-ice.tinyvision.ai/">pico-ice by tinyVision.ai</a> and... well it's an open source board with an RP2040 and an iCE40 FPGA for $35, that sounds awesome! I was thinking that the RP2040 sounded like an awesome peripheral for the FPGA... or was the FPGA the peripheral? I bet I get to choose and either way I am all for it</p> <p><img src="/images/mad-computer-science/6/pico_ice_front.jpg" alt="The Pico-Ice" /></p> <p>It didn't take very long to arrive, and I was excited to get started. I love the Raspbery Pi Pico but this is my first time working with a Lattice FPGA. I'm mostly familiar with Vivado so I wanted to try out the Lattice tool chain out to check it out. I downloaded and installed a few gigs of software but...</p> <p><img src="/images/mad-computer-science/6/licenses.png" alt="Bah!" /></p> <p>I dunno, I'm still waiting to find out</p> <h1 id="open-source-you-re-my-only-hope">Open Source, You're my Only Hope</h1> <p>That was a bit disappointing, but fortunately I have other options.</p> <p><a href="https://github.com/YosysHQ/oss-cad-suite-build/"><code>oss-cad-suite</code></a> is a nice binary bundle of hardware designs tools. It has HDL synthesizers, simulators and many, many other tools that are useful for logic design. It was incredibly convenient to just download and extract a directory, include it in my <code>PATH</code> and I'm just about ready to go.</p> <p>I worked through some documentation for my pico-ice, got a little help from the tinyVision Discord and setup my project. When I finally tried to build the design I was shocked that it took literally 1 second, I assumed something must have gone wrong.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ time fusesoc run --target pico_ice fusesoc:utils:blinky:1.1.1 </span><span>INFO: Preparing fusesoc:utils:blinky:1.1.1 </span><span>INFO: Setting up project </span><span>INFO: Building </span><span>INFO: Running </span><span>INFO: Running post_run script iceprog </span><span> </span><span>Build was completed </span><span> </span><span>To program the board run: </span><span>iceprog /tmp/blinky/build/fusesoc_utils_blinky_1.1.1/pico_ice-icestorm/fusesoc_utils_blinky_*.bin </span><span> </span><span>real 0m1.030s </span><span>user 0m0.927s </span><span>sys 0m0.104s </span></code></pre> <p>Nothing went wrong though! I'm used to it taking minutes to synthesize even a trivial design. That's definitely something I could get used to and I'm eager to see what other goodies oss-cad-suite has in store, but I've got a couple other great projects to share.</p> <h1 id="what-i-was-missing">What I was missing</h1> <p>My engineering nightmare is a world without package managers, and that of course leads me to <a href="https://github.com/olofk/fusesoc">FuseSoC</a>, a tool for making your logic design experience so much better.</p> <p>When I first came across FuseSoC I thought it was a high level generator focused on MCU or SoC development and I overlooked its broader capability. It's the critical tool that I was missing to enjoy logic design much more. I dabbled a bit with packaging some of my own cores and I'll be looking into making many more and trying to contribute to the project as I can.</p> <iframe width="560" height="315" src="https://www.youtube.com/embed/7eWRAOK9mns" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe> <p>I think FuseSoC is awesome and deserves a lot more attention and interest. The code reuse and knowledge share that can be built around this type of package manager can really help people jump into logic design when there are a lot of good building blocks to work with. We need folks of all backgrounds to mess around and share ideas. I can't wait to see what the critical mass on this looks like, it'll be absolutely amazing.</p> <h1 id="oh-and-you-can-make-asics-too">Oh, and you can make ASICs too</h1> <p>This to me is the most mind-explodey open source hardware project that has me excited. Through <a href="https://tinytapeout.com/">Tiny Tapeout</a> you can make your own open source ASIC for like $100. Yes... YOU! If you're willing to try</p> <iframe width="560" height="315" src="https://www.youtube.com/embed/f4w1QOpHzOo" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe> <p>Until this project I had pretty much assumed that for me to make an actual chip design was impossible. I'd never be able to do that... Well now I feel like I have to!</p> <p>Without even installing anything you could build a small custom chip using open, browser-based schematic design tools or via HDL. I highly recommend checking it out.</p> <p>My pico-ice is going to be quite busy with some prototyping!</p> 2023-06-10T00:00:00+00:00 2023-06-10T00:00:00+00:00 https://suchprogramming.com/mad-computer-science-part-5/ <p>A huge element of software and hardware design is figuring out how various components are going to fit together and talk to each other. This could be interfaces within a codebase like function names and signatures, networked components communicating via REST or gRPC, or the PCI-Express lanes between your processor and GPU. In today's episode, I'll be experimenting with designing a SystemVerilog module that can work with the classic TTL serial protocol.</p> <h1 id="talking-to-my-fpga">Talking to my FPGA</h1> <p>In some of my planned projects, I'm going to want a way to communicate between a digital design I've loaded onto my FPGA and my laptop. Here's the design that I'm working towards here:</p> <p><img src="/images/mad-computer-science/uart-elaborated.png" alt="UART Design" /></p> <p>For my target board, the Arty A-7, the easiest available option for me is serial since this board has a USB controller with <code>rx</code>/<code>tx</code> pins routed to the FPGA.</p> <p><img src="/images/mad-computer-science/arty-uart.png" alt="Arty UART" /></p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset> <p>To kick things off, I'm going to start a <a href="https://github.com/KennethWilke/wiz">Wiz</a> project and map some IO pins. For this design I'll have the pins: <code>rx</code> and <code>tx</code> for serial, <code>clk</code> as my input clock using the 100mhz clock from the Arty board and <code>reset</code> to help make sure things initialize into a good state.</p> <pre data-lang="toml" style="background-color:#2b303b;color:#c0c5ce;" class="language-toml "><code class="language-toml" data-lang="toml"><span style="color:#bf616a;">package_name </span><span>= &#39;</span><span style="color:#a3be8c;">uart</span><span>&#39; </span><span style="color:#bf616a;">builder </span><span>= &#39;</span><span style="color:#a3be8c;">vivado</span><span>&#39; </span><span style="color:#bf616a;">programmer </span><span>= &#39;</span><span style="color:#a3be8c;">vivado</span><span>&#39; </span><span style="color:#bf616a;">target_part </span><span>= &#39;</span><span style="color:#a3be8c;">xc7a35ticsg324-1L</span><span>&#39; </span><span style="color:#bf616a;">target_device </span><span>= &#39;</span><span style="color:#a3be8c;">xc7a35t_0</span><span>&#39; </span><span style="color:#bf616a;">source_files </span><span>= [ </span><span> &#39;</span><span style="color:#a3be8c;">top.sv</span><span>&#39;, </span><span>] </span><span style="color:#bf616a;">bitstream_path </span><span>= &#39;</span><span style="color:#a3be8c;">uart.bit</span><span>&#39; </span><span> </span><span>[pins] </span><span style="color:#bf616a;">tx </span><span>= &#39;</span><span style="color:#a3be8c;">D10</span><span>&#39; </span><span style="color:#bf616a;">rx </span><span>= &#39;</span><span style="color:#a3be8c;">A9</span><span>&#39; </span><span style="color:#bf616a;">clk </span><span>= &#39;</span><span style="color:#a3be8c;">E3</span><span>&#39; </span><span style="color:#bf616a;">reset </span><span>= &#39;</span><span style="color:#a3be8c;">C2</span><span>&#39; </span></code></pre> <p>I find it really easy to swap <code>rx</code> and <code>tx</code> by accident, especially in cases like this where the diagram above says <code>TXD</code> goes to <code>A9</code>. From the perspective of my module that is actually the <code>rx</code> line since that is the line I'll be receiving from, so I'll map those as such to help me keep their usage straight in my design.</p> <p>I want to verify that my pins are working as expected, so before I start trying to build some kind of UART I'm going to just wire <code>rx</code> to <code>tx</code> to create a stupid-simple echo implementation for the serial communication.</p> <pre data-lang="sv" style="background-color:#2b303b;color:#c0c5ce;" class="language-sv "><code class="language-sv" data-lang="sv"><span style="color:#b48ead;">module </span><span>top ( </span><span> input </span><span style="color:#b48ead;">wire</span><span> clk, </span><span> input </span><span style="color:#b48ead;">wire</span><span> reset, </span><span> input </span><span style="color:#b48ead;">wire</span><span> rx, </span><span> output </span><span style="color:#b48ead;">wire</span><span> tx </span><span>); </span><span> </span><span> </span><span style="color:#b48ead;">assign</span><span> tx = rx; </span><span> </span><span style="color:#b48ead;">endmodule </span></code></pre> <p>I'll give this a <code>wiz build</code> to build it and a <code>wiz program</code> to slap it on my Arty-A7. I'll write a short Python script to use the serial port.</p> <pre data-lang="python" style="background-color:#2b303b;color:#c0c5ce;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#b48ead;">import </span><span>serial </span><span> </span><span>ttl = serial.</span><span style="color:#bf616a;">Serial</span><span>(&#39;</span><span style="color:#a3be8c;">/dev/ttyUSB1</span><span>&#39;, </span><span style="color:#d08770;">115200</span><span>) </span><span>ttl.</span><span style="color:#bf616a;">write</span><span>(</span><span style="color:#b48ead;">b</span><span>&quot;</span><span style="color:#a3be8c;">test!</span><span>&quot;) </span><span style="color:#96b5b4;">print</span><span>(ttl.</span><span style="color:#bf616a;">read</span><span>(</span><span style="color:#d08770;">5</span><span>).</span><span style="color:#bf616a;">decode</span><span>(&#39;</span><span style="color:#a3be8c;">utf-8</span><span>&#39;)) </span></code></pre> <p>Lastly, I'll run that to make sure it works.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ python test.py </span><span>test! </span></code></pre> <p>Perfect! I now have a working &quot;echo&quot; serial device, though so far I'd probably just call it a complicated way of jumping two pins together.</p> <h1 id="building-a-uart">Building a UART</h1> <p>I'm not familiar with UART design techniques, but through trying to get this project to work I settled on a simple design that's based on some stateful shift registers. I'll have one register working on the RX side receiving data and one on the TX side sending data.</p> <p><img src="/images/mad-computer-science/uart-design.png" alt="UART design" /></p> <p>One of the big considerations for how I want this UART to work is going to be its baud rate. On this board I have a 100mhz input clock to work with, so I can very reliably time a 115200 baud communication. The approach is going to be very similar to a software bitbanging approach, but kind of simpler and very accurate by comparison.</p> <p>For this post I won't detail the TTL Serial protocol in full, as I think <a href="https://learn.sparkfun.com/tutorials/serial-communication/all">this Sparkfun tutorial</a> covers that very nicely and it continues to be my go-to resource for UART protocol details. The most important part is the general transmission format, which in my case here will be with 3.3V TTL serial.</p> <p><img src="/images/mad-computer-science/uart-bitsies.png" alt="UART bits" /></p> <p>I'll build my design as a state machine. It will start at an initial state where we're waiting for the RX line to drop low to indicate the <code>IDLE</code> bit. At that detection, I'll reset and start a counter of <code>clk</code> transitions and at the right number of clock cycles I'll sample the RX line for data, easy enough!</p> <p>I'll start this part of the design by build the core guts of my RX register:</p> <pre data-lang="sv" style="background-color:#2b303b;color:#c0c5ce;" class="language-sv "><code class="language-sv" data-lang="sv"><span style="color:#b48ead;">module </span><span>UartRxRegister ( </span><span> input clk, </span><span> input reset, </span><span> input rx, </span><span> output </span><span style="color:#b48ead;">logic</span><span> available, </span><span> output </span><span style="color:#b48ead;">logic</span><span> [</span><span style="color:#d08770;">7</span><span>:</span><span style="color:#d08770;">0</span><span>] data </span><span>); </span><span> </span><span> </span><span style="color:#b48ead;">typedef enum </span><span>{ </span><span> </span><span style="color:#d08770;">IDLE</span><span>, </span><span> </span><span style="color:#d08770;">BIT</span><span>[</span><span style="color:#d08770;">8</span><span>], </span><span> </span><span style="color:#d08770;">STOP </span><span> } RxState; </span><span> </span><span> </span><span style="color:#b48ead;">RxState </span><span>state; </span><span style="color:#b48ead;"> logic</span><span> [</span><span style="color:#d08770;">13</span><span>:</span><span style="color:#d08770;">0</span><span>] counter; </span><span> </span><span> </span><span style="color:#b48ead;">always </span><span>@ (</span><span style="color:#b48ead;">posedge</span><span> clk or </span><span style="color:#b48ead;">negedge</span><span> reset) </span><span style="color:#b48ead;">begin </span><span> </span><span style="color:#b48ead;">if</span><span> (!reset) </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">IDLE</span><span>; </span><span> available &lt;= </span><span style="color:#d08770;">0</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">endmodule </span></code></pre> <p>So far this module has it's inputs and outputs, an <code>enum</code> defining the <code>RxState</code> type along with the 10 states for this system (<code>IDLE</code>, <code>BIT0</code> through <code>BIT7</code> and <code>STOP</code>). I've also started added the reset logic to default <code>state</code> to <code>IDLE</code> and to set the <code>available</code> output flag low.</p> <p>Next I'll start to shape out the state machine itself.</p> <pre data-lang="sv" style="background-color:#2b303b;color:#c0c5ce;" class="language-sv "><code class="language-sv" data-lang="sv"><span style="color:#b48ead;">case</span><span> (state) </span><span> </span><span style="color:#d08770;">IDLE</span><span>: </span><span style="color:#b48ead;">begin </span><span> </span><span style="color:#b48ead;">if</span><span> (!rx) </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT0</span><span>; </span><span> available &lt;= </span><span style="color:#d08770;">0</span><span>; </span><span> counter &lt;= </span><span style="color:#d08770;">0</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT0</span><span>: </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT1</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT1</span><span>: </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT2</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT2</span><span>: </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT3</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT3</span><span>: </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT4</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT4</span><span>: </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT5</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT5</span><span>: </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT6</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT6</span><span>: </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT7</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT7</span><span>: </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">STOP</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">STOP</span><span>: </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">IDLE</span><span>; </span><span> available &lt;= </span><span style="color:#d08770;">1</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span style="color:#b48ead;">endcase </span></code></pre> <p>With this state machine, the system should wait in <code>IDLE</code> until the <code>rx</code> serial line is brought low for the starting bit. Once the starting bit arrives, the system should begin transitioning through the other states in order until the <code>STOP</code> bit prepares the system to begin again.</p> <h1 id="testing-it-live">Testing it Live</h1> <p>At this point I want to see how my state machine behaves when presented with some serial traffic, so I'm going to generate an ILA (integrated logic analyzer) core to debug my design with.</p> <p>After giving my project a fresh <code>wiz build</code>, I'll open up the generated Vivado project and instantiate a new ILA core with 2 inputs: a 1-bit input I'll use for <code>rx</code> and a 4-bit value to monitor how the <code>STATE</code> register reacts to it and a 14-bit register for the <code>counter</code>.</p> <p><img src="/images/mad-computer-science/uart-ila.png" alt="UART ILA" /></p> <p>I'll have vivado kick off the generator it uses to build the ILA, and add an instantiation of this new module in my rx module:</p> <pre data-lang="sv" style="background-color:#2b303b;color:#c0c5ce;" class="language-sv "><code class="language-sv" data-lang="sv"><span style="color:#b48ead;">ila_uart </span><span>debugger ( </span><span> .</span><span style="color:#96b5b4;">clk</span><span>(clk), </span><span> .</span><span style="color:#96b5b4;">probe0</span><span>(rx), </span><span> .</span><span style="color:#96b5b4;">probe1</span><span>(state), </span><span> .</span><span style="color:#96b5b4;">probe2</span><span>(counter) </span><span>); </span></code></pre> <p>Then I'll build the bitstream in Vivado and run it along with the debugging stuff it generated. When I deploy the design via the Hardware Manager in Vivado it'll bring up the ILA's waveform viewer, where I'll see the design is stably in the <code>IDLE</code> state until <code>rx</code> goes low to indicate the start of a transmission.</p> <p><img src="/images/mad-computer-science/uart-ila2.png" alt="UART ILA" /></p> <h1 id="reading-time">Reading Time</h1> <p>With the state transitions working, I'll now work on triggering those transitions as close to the middle point of each bit as I can with my given input clock of 100Mhz. I'll refer again to the diagram from SparkFun.</p> <p><img src="/images/mad-computer-science/uart-bitsies.png" alt="UART bits" /></p> <p>If I want to read the value of the least significant bit, the ideal time to poll at &quot;1.5 baud&quot;. Each next bit is 1 baud away until the stop bit, which we're not going to be using so the final transition from STOP to IDLE should happen at 0.5 baud after we read the most significant bit.</p> <p>To determine how many of the 100Mhz clock cycles we should be waiting for at each transition, I whipped up a python script that spits out the numbers I'm looking for.</p> <pre data-lang="python" style="background-color:#2b303b;color:#c0c5ce;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#65737e;">#!/usr/bin/env python3 </span><span>input_clock = </span><span style="color:#d08770;">100000000.0 </span><span>baud = </span><span style="color:#d08770;">115200.0 </span><span> </span><span>bauds_per_clock = input_clock / baud </span><span> </span><span style="color:#b48ead;">for </span><span>i </span><span style="color:#b48ead;">in </span><span style="color:#96b5b4;">range</span><span>(</span><span style="color:#d08770;">0</span><span>, </span><span style="color:#d08770;">8</span><span>): </span><span> next_step = </span><span style="color:#bf616a;">int</span><span>((</span><span style="color:#d08770;">1.5 </span><span>+ i) * bauds_per_clock) </span><span> </span><span style="color:#96b5b4;">print</span><span>(</span><span style="color:#b48ead;">f</span><span>&quot;</span><span style="color:#a3be8c;">BIT</span><span>{i}</span><span style="color:#a3be8c;">: </span><span>{next_step}&quot;) </span><span> </span><span>last_step = </span><span style="color:#bf616a;">int</span><span>(</span><span style="color:#d08770;">9 </span><span>* bauds_per_clock) </span><span style="color:#96b5b4;">print</span><span>(</span><span style="color:#b48ead;">f</span><span>&quot;</span><span style="color:#a3be8c;">STOP: </span><span>{last_step}&quot;) </span></code></pre> <p>When ran this gives me:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>BIT0: 1302 </span><span>BIT1: 2170 </span><span>BIT2: 3038 </span><span>BIT3: 3906 </span><span>BIT4: 4774 </span><span>BIT5: 5642 </span><span>BIT6: 6510 </span><span>BIT7: 7378 </span><span>STOP: 7812 </span></code></pre> <p>I'll now add checks to each state so that when the <code>counter</code> hits that number it transitions to the next state, I'll also go ahead and read <code>rx</code> to set the appropriate bit in my output <code>data</code> register.</p> <pre data-lang="sv" style="background-color:#2b303b;color:#c0c5ce;" class="language-sv "><code class="language-sv" data-lang="sv"><span style="color:#b48ead;">case</span><span> (state) </span><span> </span><span style="color:#d08770;">BIT0</span><span>: </span><span style="color:#b48ead;">begin </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">1302</span><span>) </span><span style="color:#b48ead;">begin </span><span> data[</span><span style="color:#d08770;">0</span><span>] &lt;= rx; </span><span> available &lt;= </span><span style="color:#d08770;">0</span><span>; </span><span> state &lt;= </span><span style="color:#d08770;">BIT1</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT1</span><span>: </span><span style="color:#b48ead;">begin </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">2170</span><span>) </span><span style="color:#b48ead;">begin </span><span> data[</span><span style="color:#d08770;">1</span><span>] &lt;= rx; </span><span> state &lt;= </span><span style="color:#d08770;">BIT2</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT2</span><span>: </span><span style="color:#b48ead;">begin </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">3038</span><span>) </span><span style="color:#b48ead;">begin </span><span> data[</span><span style="color:#d08770;">2</span><span>] &lt;= rx; </span><span> state &lt;= </span><span style="color:#d08770;">BIT3</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT3</span><span>: </span><span style="color:#b48ead;">begin </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">3906</span><span>) </span><span style="color:#b48ead;">begin </span><span> data[</span><span style="color:#d08770;">3</span><span>] &lt;= rx; </span><span> state &lt;= </span><span style="color:#d08770;">BIT4</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT4</span><span>: </span><span style="color:#b48ead;">begin </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">4774</span><span>) </span><span style="color:#b48ead;">begin </span><span> data[</span><span style="color:#d08770;">4</span><span>] &lt;= rx; </span><span> state &lt;= </span><span style="color:#d08770;">BIT5</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT5</span><span>: </span><span style="color:#b48ead;">begin </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">5642</span><span>) </span><span style="color:#b48ead;">begin </span><span> data[</span><span style="color:#d08770;">5</span><span>] &lt;= rx; </span><span> state &lt;= </span><span style="color:#d08770;">BIT6</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT6</span><span>: </span><span style="color:#b48ead;">begin </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">6510</span><span>) </span><span style="color:#b48ead;">begin </span><span> data[</span><span style="color:#d08770;">6</span><span>] &lt;= rx; </span><span> state &lt;= </span><span style="color:#d08770;">BIT7</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT7</span><span>: </span><span style="color:#b48ead;">begin </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">7378</span><span>) </span><span style="color:#b48ead;">begin </span><span> data[</span><span style="color:#d08770;">7</span><span>] &lt;= rx; </span><span> state &lt;= </span><span style="color:#d08770;">STOP</span><span>; </span><span> available &lt;= </span><span style="color:#d08770;">1</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">STOP</span><span>: </span><span style="color:#b48ead;">begin </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">7812</span><span>) </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">IDLE</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span style="color:#b48ead;">endcase </span></code></pre> <p><code>BIT7</code> gets a little special treatment, as that's our last bit so at this point our transferred byte is available and I'll indicate that with the <code>available</code> output.</p> <p>The reading side should be good to go now I think... I'll recustomize the ILA core to add probes for <code>data</code> and <code>available</code> so that I can see how they are changing as well.</p> <p><img src="/images/mad-computer-science/uart-ila-reading.png" alt="ILA reading" /></p> <p>Perfect, the transitions are happening at the right times, and <code>data</code> holds the appropriate values when <code>available</code> is raised!</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="writing-back">Writing Back</h1> <p>To wrap up this echoing UART design I'll implement the TX register in roughly the same way as the RX register, but basically in reverse. It'll take in <code>clk</code> and <code>reset</code> just like RX register, but the TX register will also have the inputs <code>send</code> and <code>data</code> for writing a byte out to its one output: <code>tx</code>.</p> <pre data-lang="sv" style="background-color:#2b303b;color:#c0c5ce;" class="language-sv "><code class="language-sv" data-lang="sv"><span style="color:#b48ead;">module </span><span>UartTxRegister ( </span><span> input clk, </span><span> input reset, </span><span> input send, </span><span> input [</span><span style="color:#d08770;">7</span><span>:</span><span style="color:#d08770;">0</span><span>] data, </span><span> output </span><span style="color:#b48ead;">logic</span><span> tx </span><span>); </span></code></pre> <p>The state enum is a little different as well. We'll still start in <code>IDLE</code>, but we'll need to bring <code>tx</code> low for 1 baud as the <code>START</code> bit. After that, we'll send out the data bits.</p> <pre data-lang="sv" style="background-color:#2b303b;color:#c0c5ce;" class="language-sv "><code class="language-sv" data-lang="sv"><span style="color:#b48ead;">typedef enum </span><span>{ </span><span> </span><span style="color:#d08770;">IDLE</span><span>, </span><span> </span><span style="color:#d08770;">START</span><span>, </span><span> </span><span style="color:#d08770;">BIT</span><span>[</span><span style="color:#d08770;">8</span><span>] </span><span>} TxState; </span></code></pre> <p>Within the module I have a <code>state</code> and <code>counter</code> just like before, but this time I'll also add a <code>send_data</code> register that will store the input <code>data</code> when <code>send</code> goes high.</p> <pre data-lang="sv" style="background-color:#2b303b;color:#c0c5ce;" class="language-sv "><code class="language-sv" data-lang="sv"><span style="color:#b48ead;">TxState </span><span>state; </span><span style="color:#b48ead;">logic</span><span> [</span><span style="color:#d08770;">13</span><span>:</span><span style="color:#d08770;">0</span><span>] counter; </span><span style="color:#b48ead;">logic</span><span> [</span><span style="color:#d08770;">7</span><span>:</span><span style="color:#d08770;">0</span><span>] send_data; </span></code></pre> <p>Now to implement the logic for sending. I'll add some additional outputs to my Python script to give me the clock times I'll need.</p> <pre data-lang="python" style="background-color:#2b303b;color:#c0c5ce;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#96b5b4;">print</span><span>(&quot;</span><span style="color:#a3be8c;">Writing times:</span><span>&quot;) </span><span>first_step = </span><span style="color:#bf616a;">int</span><span>(bauds_per_clock) </span><span style="color:#96b5b4;">print</span><span>(</span><span style="color:#b48ead;">f</span><span>&quot;</span><span style="color:#a3be8c;">START: </span><span>{first_step}&quot;) </span><span> </span><span style="color:#b48ead;">for </span><span>i </span><span style="color:#b48ead;">in </span><span style="color:#96b5b4;">range</span><span>(</span><span style="color:#d08770;">0</span><span>, </span><span style="color:#d08770;">8</span><span>): </span><span> next_step = </span><span style="color:#bf616a;">int</span><span>((</span><span style="color:#d08770;">2 </span><span>+ i) * bauds_per_clock) </span><span> </span><span style="color:#96b5b4;">print</span><span>(</span><span style="color:#b48ead;">f</span><span>&quot;</span><span style="color:#a3be8c;">BIT</span><span>{i}</span><span style="color:#a3be8c;">: </span><span>{next_step}&quot;) </span></code></pre> <p>Similar to before, this prints out my timing for when each state should transition to the next.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>START: 868 </span><span>BIT0: 1736 </span><span>BIT1: 2604 </span><span>BIT2: 3472 </span><span>BIT3: 4340 </span><span>BIT4: 5208 </span><span>BIT5: 6076 </span><span>BIT6: 6944 </span><span>BIT7: 7812 </span></code></pre> <p>Now to implement the <code>always</code> block. It'll start off with the reset logic followed by the state machine. It'll hold the <code>counter</code> to <code>0</code> and <code>tx</code> to <code>1</code> while reset or idle. When <code>send</code> is raised the value will be stored in <code>send_data</code>, the <code>state</code> transitions to <code>START</code> and the counter starts counting.</p> <pre data-lang="sv" style="background-color:#2b303b;color:#c0c5ce;" class="language-sv "><code class="language-sv" data-lang="sv"><span style="color:#b48ead;">always </span><span>@ (</span><span style="color:#b48ead;">posedge</span><span> clk) </span><span style="color:#b48ead;">begin </span><span> </span><span style="color:#b48ead;">if</span><span> (!reset) </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">IDLE</span><span>; </span><span> tx &lt;= </span><span style="color:#d08770;">1</span><span>; </span><span> counter &lt;= </span><span style="color:#d08770;">0</span><span>; </span><span> </span><span style="color:#b48ead;">end else begin </span><span> </span><span style="color:#b48ead;">if</span><span> (state == </span><span style="color:#d08770;">IDLE</span><span>) </span><span style="color:#b48ead;">begin </span><span> counter &lt;= </span><span style="color:#d08770;">0</span><span>; </span><span> tx &lt;= </span><span style="color:#d08770;">1</span><span>; </span><span> </span><span style="color:#b48ead;">if</span><span> (send) </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">START</span><span>; </span><span> send_data &lt;= data; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end else begin </span><span> counter &lt;= counter + </span><span style="color:#d08770;">1</span><span>; </span><span> </span><span style="color:#b48ead;">case</span><span> (state) </span><span> </span><span style="color:#d08770;">START</span><span>: </span><span style="color:#b48ead;">begin </span><span> tx &lt;= </span><span style="color:#d08770;">0</span><span>; </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">868</span><span>) </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT0</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT0</span><span>: </span><span style="color:#b48ead;">begin </span><span> tx &lt;= send_data[</span><span style="color:#d08770;">0</span><span>]; </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">1736</span><span>) </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT1</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT1</span><span>: </span><span style="color:#b48ead;">begin </span><span> tx &lt;= send_data[</span><span style="color:#d08770;">1</span><span>]; </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">2604</span><span>) </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT2</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT2</span><span>: </span><span style="color:#b48ead;">begin </span><span> tx &lt;= send_data[</span><span style="color:#d08770;">2</span><span>]; </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">3472</span><span>) </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT3</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT3</span><span>: </span><span style="color:#b48ead;">begin </span><span> tx &lt;= send_data[</span><span style="color:#d08770;">3</span><span>]; </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">4340</span><span>) </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT4</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT4</span><span>: </span><span style="color:#b48ead;">begin </span><span> tx &lt;= send_data[</span><span style="color:#d08770;">4</span><span>]; </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">5208</span><span>) </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT5</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT5</span><span>: </span><span style="color:#b48ead;">begin </span><span> tx &lt;= send_data[</span><span style="color:#d08770;">5</span><span>]; </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">6076</span><span>) </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT6</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT6</span><span>: </span><span style="color:#b48ead;">begin </span><span> tx &lt;= send_data[</span><span style="color:#d08770;">6</span><span>]; </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">6944</span><span>) </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">BIT7</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#d08770;">BIT7</span><span>: </span><span style="color:#b48ead;">begin </span><span> tx &lt;= send_data[</span><span style="color:#d08770;">7</span><span>]; </span><span> </span><span style="color:#b48ead;">if</span><span> (counter == </span><span style="color:#d08770;">7812</span><span>) </span><span style="color:#b48ead;">begin </span><span> state &lt;= </span><span style="color:#d08770;">IDLE</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">endcase </span><span> </span><span style="color:#b48ead;">end </span><span> </span><span style="color:#b48ead;">end </span><span style="color:#b48ead;">end </span></code></pre> <p>That should take care of the TX register. The last bit left to do is to wire it up on the <code>top</code> module.</p> <pre data-lang="sv" style="background-color:#2b303b;color:#c0c5ce;" class="language-sv "><code class="language-sv" data-lang="sv"><span style="color:#b48ead;">module </span><span>top ( </span><span> input </span><span style="color:#b48ead;">wire</span><span> clk, </span><span> input </span><span style="color:#b48ead;">wire</span><span> reset, </span><span> input </span><span style="color:#b48ead;">wire</span><span> rx, </span><span> output </span><span style="color:#b48ead;">wire</span><span> tx </span><span>); </span><span> </span><span style="color:#b48ead;"> logic</span><span> available; </span><span style="color:#b48ead;"> logic</span><span> [</span><span style="color:#d08770;">7</span><span>:</span><span style="color:#d08770;">0</span><span>] data; </span><span> </span><span> </span><span style="color:#b48ead;">UartRxRegister </span><span>RxRegister(clk, reset, rx, available, data); </span><span> </span><span style="color:#b48ead;">UartTxRegister </span><span>TxRegister(clk, reset, available, data, tx); </span><span> </span><span style="color:#b48ead;">endmodule </span></code></pre> <p>Now to hope it all works! I'll instantiate another ILA instance and watch all the good stuff in the <code>top</code> module.</p> <p><img src="/images/mad-computer-science/uart-ila-working.png" alt="Working echo" /></p> <p>There it is, the UART successfully reads and echos back! This was a pretty large post, but if you made it to the end here I appreciate it and hope you enjoyed the post.</p> 2023-01-29T00:00:00+00:00 2023-01-29T00:00:00+00:00 https://suchprogramming.com/preview/6502-reasons-to-learn-assembly/ <p>Learning the language of a processor can be a daunting task, but <a href="https://nostarch.com/writegreatcode1_2e">understanding the machine</a> you're working with makes it easier to write great code for it. Modern systems are so complex that directly writing assembly is laborious. However, modern open source tooling allows us to travel back in time 40 years and explore foundational software and hardware concepts in ways unimaginable during the lifetime of those systems.</p> <p>One of the systems I plan to explore quite a bit in this blog series is the NES, my all time favorite retrocomputer. In this post I want to warm you up to some of the shenanigans to come and show you that tinkering with assembly on the NES may be <em>a lot</em> easier than you think!</p> <p><img src="/images/mad-computer-science/nintendopowerglove.jpg" alt="" /></p> <h1 id="simpler-times">Simpler Times</h1> <p>Compared to writing assembly for a desktop computer, the NES has no operating sytem, no <a href="https://en.wikipedia.org/wiki/Memory_management_unit">MMU</a> and no <a href="https://en.wikipedia.org/wiki/Memory_protection">memory protection</a>. These complex systems are valuable to understand but are also more difficult to appreciate without a context. The lack of these modern features means the code you write for a 6502-based machine runs in a very raw and simple environment that is very close to hardware.</p> <p>A lot of what I'd like to share about why and how to get started with 6502 assembly development is already well covered in <a href="https://skilldrick.github.io/easy6502/">Easy 6502</a>, which offers a nice getting started tutorial and provides an in-browser assembler and simple debugger. The main aspect I want to focus on for this post are the &quot;APIs&quot; of the processor.</p> <p>There <a href="http://www.6502.org/tutorials/6502opcodes.html">are only 56 operations</a> the 6502 can perform, most of which are trivial to understand. These are the building blocks used to make great stuff and are the internal API for your software.</p> <p>Via the pins connecting the 6502 to whatever system it's placed in, the CPU has a 16-bit address bus, an 8-bit data bus and a control bus that lets the processor interact with the outside world. This is the external interface that the CPU will use to interact with RAMs, ROMs and devices.</p> <p><img src="/images/fpga-meets-6502/W65C02S-40-pin-dip-color.gif" alt="" /></p> <h1 id="pointers-for-the-hardware">Pointers for the Hardware</h1> <p>Before exploring the internal API more, let's look at the external side of things. Since the 6502 has a 16-bit address bus and an 8-bit data bus, it can read/write 8 bits of data from/to <code>2^16</code> different addresses. The CPU controls what address is being referred to on this address bus and uses the control bus to notify other connected devices if the data bus should be used for a read or a write operation.</p> <p>To compare this to something like JavaScript, imagine an array of 65536 <code>[Object]</code>s, some which may support a <code>.read()</code> and/or <code>.write()</code> operation. What type of device exists at each address varies greatly by system, though for 6502 you can comfortably rely on the lowest addresses being system RAM. For the NES, here's how <a href="https://www.nesdev.org/wiki/CPU_memory_map">nesdev.org</a> describes the CPU memory map.</p> <p><img src="/images/mad-computer-science/nesdev-memmap.png" alt="" /></p> <p>Would you like to read the first byte of RAM? that's effectively someting like this:</p> <pre data-lang="javascript" style="background-color:#2b303b;color:#c0c5ce;" class="language-javascript "><code class="language-javascript" data-lang="javascript"><span style="color:#b48ead;">var </span><span style="color:#bf616a;">first_byte </span><span>= </span><span style="color:#bf616a;">memory_map</span><span>[</span><span style="color:#d08770;">0</span><span>].</span><span style="color:#8fa1b3;">read</span><span>(); </span></code></pre> <p>What about something that sounds more difficult, like reading the status of the <a href="https://www.nesdev.org/wiki/PPU">Picture Processing Unit</a>? Well that's pretty much the same thing but instead of reading from <code>0</code> we'd want to read from <a href="https://www.nesdev.org/wiki/PPU_registers#Status_($2002)_%3C_read"><code>$2002</code></a></p> <pre data-lang="javascript" style="background-color:#2b303b;color:#c0c5ce;" class="language-javascript "><code class="language-javascript" data-lang="javascript"><span style="color:#b48ead;">var </span><span style="color:#bf616a;">ppu_status </span><span>= </span><span style="color:#bf616a;">memory_map</span><span>[</span><span style="color:#d08770;">0x2002</span><span>].</span><span style="color:#8fa1b3;">read</span><span>(); </span></code></pre> <p>We could even use our old friend, the pointer!</p> <pre data-lang="javascript" style="background-color:#2b303b;color:#c0c5ce;" class="language-javascript "><code class="language-javascript" data-lang="javascript"><span style="color:#b48ead;">const </span><span style="color:#bf616a;">PPU_STATUS </span><span>= </span><span style="color:#d08770;">0x2002</span><span>; </span><span> </span><span style="color:#b48ead;">var </span><span style="color:#bf616a;">pointer </span><span>= </span><span style="color:#bf616a;">PPU_STATUS</span><span>; </span><span style="color:#b48ead;">var </span><span style="color:#bf616a;">ppu_status </span><span>= </span><span style="color:#bf616a;">memory_map</span><span>[</span><span style="color:#bf616a;">pointer</span><span>].</span><span style="color:#8fa1b3;">read</span><span>(); </span></code></pre> <p>This is really the main concept to understand for the hardware-facing side of the interface. There are interesting details in the implementation and the devices you would communicate to but hopefully this give a good functional base for the exploration of the internal API, coming up after the break.</p> <h1 id="operating-the-6502">Operating the 6502</h1> <p>This section summarizes the controls available within the code you run on a 6502. Don't worry if some or many of these terms sound entirely unfamiliar, they are concepts learned best by tinkering with in their own experimentation.</p> <p>As I mentioned before there are only 56 Opcodes for the 6502. Many of these have similarities and can be grouped together. The other big element to be aware of is that the 6502 has 3 primary registers: the accumulator (<code>A</code>) and 2 index registers (<code>X</code> and <code>Y</code>). Internally, there is also a Program Counter <code>PC</code> that points to the memory address of the next location that should be fetched from and executed as code, as well as a Stack Pointer (<code>SP</code>) that is used for stack based operations. There are also a few processor status flags: <code>Carry</code>, <code>Zero</code>, <code>Interrupt</code>, <code>Decimal</code>, <code>Overflow</code> and <code>Negative</code> encoded into a Status register.</p> <p>I would broadly categorize these instructions into 4 groups:</p> <ul> <li> <p>Load/Store/Transfer Instructions</p> <ul> <li><code>LDA</code> - Load A Register</li> <li><code>LDX</code> - Load X Register</li> <li><code>LDY</code> - Load Y Register</li> <li><code>PHA</code> - Push Accumulator</li> <li><code>PHP</code> - Push processor status</li> <li><code>PLA</code> - Pull Accumulator</li> <li><code>PLP</code> - Pull processor status</li> <li><code>STA</code> - Store A Register</li> <li><code>STX</code> - Store X Register</li> <li><code>STY</code> - Store Y Register</li> <li><code>TAX</code> - Transfer A to X</li> <li><code>TAY</code> - Transfer A to Y</li> <li><code>TSX</code> - Transfer stack pointer to X</li> <li><code>TXA</code> - Transfer X to A</li> <li><code>TXS</code> - Transfer X to stack pointer</li> <li><code>TYA</code> - Transfer Y to A</li> </ul> </li> <li> <p>Control Flow</p> <ul> <li><code>BCC</code> - Branch &quot;Carry&quot; clear</li> <li><code>BCS</code> - Branch &quot;Carry&quot; set</li> <li><code>BEQ</code> - Branch equal</li> <li><code>BMI</code> - Branch &quot;minus&quot;</li> <li><code>BNE</code> - Branch not equal</li> <li><code>BPL</code> - Branch &quot;plus&quot;</li> <li><code>BRK</code> - Break</li> <li><code>BVC</code> - Branch &quot;Overflow&quot; clear</li> <li><code>BVS</code> - Branch &quot;Overflow&quot; set</li> <li><code>CMP</code> - Compare A Register</li> <li><code>CPX</code> - Compare X Register</li> <li><code>CPY</code> - Compary Y Register</li> <li><code>JMP</code> - Jump</li> <li><code>JSR</code> - Jump to subroutine</li> <li><code>NOP</code> - No-operation</li> <li><code>RTI</code> - Return from interrupt</li> <li><code>RTS</code> - Return from subroutine</li> </ul> </li> <li> <p>Math/Logic</p> <ul> <li><code>ADC</code> - Add with Carry</li> <li><code>AND</code> - Bitwise AND</li> <li><code>ASL</code> - Arithmetic shift left</li> <li><code>DEC</code> - Decrement A register</li> <li><code>DEX</code> - Decrement X register</li> <li><code>DEY</code> - Decrement Y register</li> <li><code>EOR</code> - Exclusive OR</li> <li><code>INC</code> - Increment A register</li> <li><code>INX</code> - Increment X register</li> <li><code>INY</code> - Increment Y register</li> <li><code>LSR</code> - Logical shift right</li> <li><code>ORA</code> - Bitwise OR</li> <li><code>ROL</code> - Rotate Left</li> <li><code>ROR</code> - Rotate Right</li> <li><code>SBC</code> - Subtract with Carry</li> </ul> </li> <li> <p>Flags</p> <ul> <li><code>BIT</code> - Test BITs</li> <li><code>CLC</code> - Clear &quot;Carry&quot; Flag</li> <li><code>CLD</code> - Clear &quot;Decimal&quot; Flag</li> <li><code>CLI</code> - Clear &quot;Interrupt&quot; Flag</li> <li><code>CLV</code> - Clear &quot;Overflow&quot; Flag</li> <li><code>SEC</code> - Set &quot;Carry&quot; Flag</li> <li><code>SED</code> - Set &quot;Decimal&quot; Flag</li> <li><code>SEI</code> - Set &quot;Instruction&quot; Flag</li> </ul> </li> </ul> <p>This is not a exactly a small amount of instructions, but there are 987 in x86_64 so by comparison it is quite tiny!</p> <h1 id="the-learning-journey">The Learning Journey</h1> <p>Generally speaking I think it's fair to say that understanding 6502 assembly is easy. What is much less easy is seeing and understanding how you can put these tools together to do something useful, like letting me jump on the head of a Koopa.</p> <p>Generally speaking this is a situation that get</p> 2023-01-16T00:00:00+00:00 2023-01-16T00:00:00+00:00 https://suchprogramming.com/mad-computer-science-part-4/ <p>I like tinkering with schematic based logic design, but I can easily find myself distracted away from the technical problem at hand by trying to make my schematic nice and neat. Using code instead of a schematic will let me abstract away this problem while also providing means of making my designs more scalable and easier to understand.</p> <p>In this post I'll export a schematic based design from Digital to run on a real FPGA and I'll also look at how custom blocks in Verilog can be added into a schematic design in Digital.</p> <p><img src="/images/mad-computer-science/about-damn-time.jpg" alt="About damn time" /></p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="digital-goes-irl">Digital Goes IRL</h1> <p>So far in this series all of the circuit tinkery has been virtual, but the HDL exporting features in <a href="https://github.com/hneemann/Digital">Digital</a> will let us bring these designs to life.</p> <p>I'll first come up with a rough plan. The <a href="https://digilent.com/reference/programmable-logic/arty-a7/reference-manual">Arty-A7 board</a> I will use has enough lights, buttons and switches for me to play around with, so that is not a big concern.</p> <p><img src="/images/mad-computer-science/arty-a7.png" alt="Arty A7" /></p> <p>The schematic has one big issue though: the telnet block is <em>actually</em> powered my magic (or simulator software) so it can't be exported to HDL. That's not a dealbreaker for today's goal though, I'll just... simulate the emulated hardware with actual hardware, obviously!</p> <p>Since the telnet block is going away, so does the communication protocol from before and so I'll simplify the schematic a bit before getting deeper into this.</p> <p><video controls="" src="/videos/mad-computer-science/irl-cleanup.mp4"></video></p> <p>That cleans up a good bit of clutter. The register now has floating inputs so I'll need to route some new signals to it. This is also a good time to replace the switches and LEDs in the design as those physical devices are also not part of the HDL model of the digital world. Everything will get simplified down to inputs an outputs at the top level.</p> <p><video controls="" src="/videos/mad-computer-science/irl-inputify.mp4"></video></p> <p>I'm not sure if <code>Clk</code> needs to be changed, I guess we'll find out when I try to export it!</p> <p><video controls="" src="/videos/mad-computer-science/irl-export.mp4"></video></p> <p>So <code>Clk</code> wasn't a problem, but all of my other pins being unlabeled was. After slapping some names on them, Digital is ready to export it to Verilog for me.</p> <p>Let's see what it generated...</p> <pre data-lang="systemverilog" style="background-color:#2b303b;color:#c0c5ce;" class="language-systemverilog "><code class="language-systemverilog" data-lang="systemverilog"><span style="color:#65737e;">/* </span><span style="color:#65737e;"> * Generated by Digital. Don&#39;t modify this file! </span><span style="color:#65737e;"> * Any changes will be lost if this file is regenerated. </span><span style="color:#65737e;"> */ </span><span> </span><span style="color:#b48ead;">module </span><span>DIG_Register_BUS #( </span><span style="color:#b48ead;"> parameter</span><span> Bits = </span><span style="color:#d08770;">1 </span><span>) </span><span>( </span><span> input </span><span style="color:#d08770;">C</span><span>, </span><span> input en, </span><span> input [(Bits - </span><span style="color:#d08770;">1</span><span>):</span><span style="color:#d08770;">0</span><span>]</span><span style="color:#d08770;">D</span><span>, </span><span> output [(Bits - </span><span style="color:#d08770;">1</span><span>):</span><span style="color:#d08770;">0</span><span>]</span><span style="color:#d08770;">Q </span><span>); </span><span> </span><span style="color:#b48ead;"> reg</span><span> [(Bits - </span><span style="color:#d08770;">1</span><span>):</span><span style="color:#d08770;">0</span><span>] state = </span><span style="color:#d08770;">&#39;h0</span><span>; </span><span> </span><span> </span><span style="color:#b48ead;">assign </span><span style="color:#d08770;">Q </span><span>= state; </span><span> </span><span> </span><span style="color:#b48ead;">always </span><span>@ (</span><span style="color:#b48ead;">posedge </span><span style="color:#d08770;">C</span><span>) </span><span style="color:#b48ead;">begin </span><span> </span><span style="color:#b48ead;">if</span><span> (en) </span><span> state &lt;= </span><span style="color:#d08770;">D</span><span>; </span><span> </span><span style="color:#b48ead;">end </span><span style="color:#b48ead;">endmodule </span><span> </span><span style="color:#b48ead;">module </span><span>top ( </span><span> input Clk, </span><span> input r1, </span><span> input r2, </span><span> input t, </span><span> input s1, </span><span> input s2, </span><span> output l1, </span><span> output l2 </span><span>); </span><span> </span><span style="color:#b48ead;">wire</span><span> [</span><span style="color:#d08770;">1</span><span>:</span><span style="color:#d08770;">0</span><span>] s0; </span><span> </span><span style="color:#b48ead;">wire</span><span> [</span><span style="color:#d08770;">1</span><span>:</span><span style="color:#d08770;">0</span><span>] s3; </span><span> </span><span style="color:#b48ead;">assign</span><span> s0[</span><span style="color:#d08770;">0</span><span>] = r1; </span><span> </span><span style="color:#b48ead;">assign</span><span> s0[</span><span style="color:#d08770;">1</span><span>] = r2; </span><span> </span><span style="color:#b48ead;">DIG_Register_BUS </span><span>#( </span><span> .</span><span style="color:#96b5b4;">Bits</span><span>(</span><span style="color:#d08770;">2</span><span>) </span><span> ) </span><span> DIG_Register_BUS_i0 ( </span><span> .</span><span style="color:#96b5b4;">D</span><span>( s0 ), </span><span> .</span><span style="color:#96b5b4;">C</span><span>( Clk ), </span><span> .</span><span style="color:#96b5b4;">en</span><span>( t ), </span><span> .</span><span style="color:#96b5b4;">Q</span><span>( s3 ) </span><span> ); </span><span> </span><span style="color:#b48ead;">assign</span><span> l1 = (s1 ^ s3[</span><span style="color:#d08770;">0</span><span>]); </span><span> </span><span style="color:#b48ead;">assign</span><span> l2 = (s2 ^ s3[</span><span style="color:#d08770;">1</span><span>]); </span><span style="color:#b48ead;">endmodule </span></code></pre> <p>It's interesting to see what kind of HDL it generated for me. There's a module digital made to represent it's version of a register and the <code>top</code> module which is my design. I named the file <code>top.v</code> so that the module would be named <code>top</code> and that is conventionally the name for the top level module in an FPGA project.</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="some-fpga-wizardry">Some FPGA Wizardry</h1> <p>One of my side projects, which will hopefully be receiving some more love in the near future, is a tool to aide in this kind of digital tinkery called <a href="https://github.com/KennethWilke/wiz">Wiz</a>. My aim for this tool is to make playing with FPGAs a lot easier and more fun, especially for curious software developers.</p> <p>As of this writing the tool is very young and in rough shape, but it's made for this exact type of exploratory usage so here we go!</p> <p>I'll start by creating a new folder, and within that folder I'll make a <code>wiz.toml</code> file, this is a manifest file used by Wiz that tells it how I want to build and program my logic design project.</p> <pre data-lang="toml" style="background-color:#2b303b;color:#c0c5ce;" class="language-toml "><code class="language-toml" data-lang="toml"><span style="color:#bf616a;">package_name </span><span>= &#39;</span><span style="color:#a3be8c;">irl</span><span>&#39; </span><span style="color:#bf616a;">builder </span><span>= &#39;</span><span style="color:#a3be8c;">vivado</span><span>&#39; </span><span style="color:#bf616a;">programmer </span><span>= &#39;</span><span style="color:#a3be8c;">vivado</span><span>&#39; </span><span style="color:#bf616a;">target_part </span><span>= &#39;</span><span style="color:#a3be8c;">xc7a35ticsg324-1L</span><span>&#39; </span><span style="color:#bf616a;">target_device </span><span>= &#39;</span><span style="color:#a3be8c;">xc7a35t_0</span><span>&#39; </span><span style="color:#bf616a;">source_files </span><span>= [&#39;</span><span style="color:#a3be8c;">top.v</span><span>&#39;] </span><span style="color:#bf616a;">bitstream_path </span><span>= &#39;</span><span style="color:#a3be8c;">irl.bit</span><span>&#39; </span><span> </span><span>[pins] </span></code></pre> <p>This file format is very likely to change, but the important parts are that <code>vivado</code> is what I'll be using for building and programming the device and it also specifies some details about my target Artix-A35T FPGA part/device.</p> <p>I'll also need to add more detail to the <code>[pins]</code> section to map out the FPGA pins to the inputs and outputs of my <code>top</code> module. This is mostly going to be a matter of referring to the <a href="https://digilent.com/reference/programmable-logic/arty-a7/reference-manual">board reference material</a> and selecting from that. </p> <p><img src="https://digilent.com/reference/_media/arty/arty_gpio.png" alt="" /></p> <p>In this case, I'll use the Arty's <code>LD7</code> and <code>LD6</code> LEDs as LED1 (<code>l1</code>) and LED2 (<code>l2</code>), which are mapped to pins <code>T10</code> and <code>T9</code> respectively. For the physical switch inputs (<code>s1</code> and <code>s2</code>), I'll use the left two dip switches (<code>SW3</code> and <code>SW2</code>) and for the register inputs (<code>r1</code> and <code>r2</code>) I'll use the right two (<code>SW1</code> and <code>SW0</code>). For <code>t</code>, my trigger input to save the new register state, I'll use the <code>BTN0</code> button. That just leaves <code>Clk</code>, which I'll hook up to the 100mhz clock on FPGA pin <code>E3</code>.</p> <pre data-lang="toml" style="background-color:#2b303b;color:#c0c5ce;" class="language-toml "><code class="language-toml" data-lang="toml"><span>[pins] </span><span style="color:#bf616a;">Clk </span><span>= &#39;</span><span style="color:#a3be8c;">E3</span><span>&#39; </span><span style="color:#bf616a;">r1 </span><span>= &#39;</span><span style="color:#a3be8c;">C11</span><span>&#39; </span><span style="color:#bf616a;">r2 </span><span>= &#39;</span><span style="color:#a3be8c;">A8</span><span>&#39; </span><span style="color:#bf616a;">t </span><span>= &#39;</span><span style="color:#a3be8c;">D9</span><span>&#39; </span><span style="color:#bf616a;">s1 </span><span>= &#39;</span><span style="color:#a3be8c;">A10</span><span>&#39; </span><span style="color:#bf616a;">s2 </span><span>= &#39;</span><span style="color:#a3be8c;">C10</span><span>&#39; </span><span style="color:#bf616a;">l1 </span><span>= &#39;</span><span style="color:#a3be8c;">T10</span><span>&#39; </span><span style="color:#bf616a;">l2 </span><span>= &#39;</span><span style="color:#a3be8c;">T9</span><span>&#39; </span></code></pre> <p>Picking and mapping the pins can be a little tedious, but professional FPGA tools often make it much more work than this!</p> <p>Next I'll build the project with <code>wiz build</code></p> <p><video controls="" src="/videos/mad-computer-science/irl-build.mp4"></video></p> <p>It takes a good bit to get through all its steps, but at the end I have my bitfile ready to flash. I'll plug in my Arty board and load it up with <code>wiz program</code>.</p> <p><video controls="" src="/videos/mad-computer-science/irl-program.mp4"></video></p> <p>Huzzah, my design is now implemented and working in real hardware!</p> <p><video controls="" src="/videos/mad-computer-science/irl-arty.mp4"></video></p> <p>The design itself isn't very interesting, but I'm pretty happy with the flow my Wiz tool is starting to enable.</p> <h1 id="returning-to-schematics">Returning to Schematics</h1> <p>Now what I'll do is a little bit of the opposite direction, just to prove that these tools can be mixed and matched. I'll make a new schematic and include a new Verilog block that includes the design. This is the exported Verilog from earlier, but this time imported into Digital as a module.</p> <p><video controls="" src="/videos/mad-computer-science/irl-back-to-digital.mp4"></video></p> <p>Everything needs to match up, including the one <code>Clk</code> signal I capitalized for no reason and the <code>top</code> module name for the block. I now have a pretty decent grip on a flow for how to mix and match HDL and schematics for logic design. That'll be all for this post!</p> 2023-01-03T00:00:00+00:00 2023-01-03T00:00:00+00:00 https://suchprogramming.com/mad-computer-science-part-3/ <p>The little Telnet block is humorous, but to make it much more interesting I want to wrap more useful functionality around it over time. So far I've only used it for sending data to the simulated design, but now I want to support sending data back as well.</p> <h1 id="a-virtual-device-project">A Virtual Device Project</h1> <p>I want to make a little simulated device to prove out the basic concepts of interacting between simulated hardware and running software. The device will be a little LED controller, with 2 lights and 2 switches; toggling the switches will toggle if the associated LED is on or off.</p> <p>Via the socket interface, software can read the status of the lights and switches and can also toggle the lights. The software can't toggle the switches, since those are &quot;physical&quot;.</p> <p>I'll start off my modeling the basic hardware bits in <a href="https://github.com/hneemann/Digital">Digital</a>.</p> <p><video controls="" src="/videos/mad-computer-science/basic-togglies.mp4"></video></p> <p>Before going too much further, I want to consider the communcation model and where this device is placed when integrated into a complete design. This device will be driven by a remote connection, so it'll act kind of a like server in a client-server model as far as the communication protocol is concerned.</p> <p>The Telnet block in Digital reads and writes a byte at a time, but I'll only need a few bits for this protocol. For read requests I'll just send a <code>0</code>, and encode the switch and light states into 4 bits of the reply. For writes, I'll have 1 bit to indicate the write operation and the 2 other bits will be settings for the LEDs.</p> <p>Now to the building!</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="initial-bits-and-bobs">Initial Bits and Bobs</h1> <p>The first change I'll make is to add some <code>XOR</code> gates for the LEDs, and a second set of switches to use for some early testing.</p> <p><video controls="" src="/videos/mad-computer-science/lc-xorify.mp4"></video></p> <p>With that in place, the bottom two switches can effecively override the top two switches. What I <em>really</em> want here is a 2-bit register to store these values, so I'll wire that up instead.</p> <p><video controls="" src="/videos/mad-computer-science/lc-regify.mp4"></video></p> <p>One more easy thing to add will be a status bus for reading the states of the switches and LEDs. I'll just merge the relevant wires onto a bus for that. I ultimately know this will be going to an 8-bit input to the Telnet block so I'll size it as such from the start.</p> <p><video controls="" src="/videos/mad-computer-science/lc-readbus.mp4"></video></p> <p>Lastly, I'll slap my Telnet block into the design and loosely wire it. That'll be the next area of focus.</p> <p><video controls="" src="/videos/mad-computer-science/lc-addtelnet.mp4"></video></p> <p>That takes care of the easy bits.</p> <h1 id="now-the-tricky-part">Now the Tricky Part</h1> <p>How I would wire up the rest of this design was not obvious to me. I thought I'd first try an approach that felt like it would simple if it worked... though I had strong doubts that it would work at all. I learn the most by breaking stuff so we'll call this an experiment!</p> <p><video controls="" src="/videos/mad-computer-science/lc-notquiteright.mp4"></video></p> <p>It starts off looking well enough, until the hardware version of undefined behavior (high impedance) says otherwise. The problem is that the <code>D</code> and <code>en</code> inputs to the register are not being driven high or low, though on each clock cycle the register still does its best to use those garbage inputs.</p> <p>The first little change I'll make to try to fix this is to use an <code>AND</code> gate for the register's <code>en</code> input. This will trigger the storage of the two register bits when <code>out[0]</code> and <code>av</code> from the telnet block are <code>1</code>.</p> <p><video controls="" src="/videos/mad-computer-science/lc-andify-en.mp4"></video></p> <p>I believe this circuit is going to have some timing issues because the register and telnet blocks are both triggering at the same rising edge of <code>Clk</code>. I'll test this out to see what happens with a small Python script.</p> <pre data-lang="python" style="background-color:#2b303b;color:#c0c5ce;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#65737e;">#!/usr/bin/env python3 </span><span style="color:#b48ead;">from </span><span>socket </span><span style="color:#b48ead;">import </span><span>create_connection </span><span style="color:#b48ead;">import </span><span>time </span><span> </span><span style="color:#bf616a;">ADDRESS </span><span>= (&#39;</span><span style="color:#a3be8c;">127.0.0.1</span><span>&#39;, </span><span style="color:#d08770;">1337</span><span>) </span><span> </span><span>socket = </span><span style="color:#bf616a;">create_connection</span><span>(</span><span style="color:#bf616a;">ADDRESS</span><span>) </span><span>socket.</span><span style="color:#bf616a;">sendall</span><span>(</span><span style="color:#b48ead;">b</span><span>&quot;</span><span style="color:#96b5b4;">\x01</span><span>&quot;) </span><span>time.</span><span style="color:#bf616a;">sleep</span><span>(</span><span style="color:#d08770;">1</span><span>) </span><span>socket.</span><span style="color:#bf616a;">sendall</span><span>(</span><span style="color:#b48ead;">b</span><span>&quot;</span><span style="color:#96b5b4;">\x03</span><span>&quot;) </span><span>time.</span><span style="color:#bf616a;">sleep</span><span>(</span><span style="color:#d08770;">1</span><span>) </span><span>socket.</span><span style="color:#bf616a;">sendall</span><span>(</span><span style="color:#b48ead;">b</span><span>&quot;</span><span style="color:#96b5b4;">\x05</span><span>&quot;) </span><span>time.</span><span style="color:#bf616a;">sleep</span><span>(</span><span style="color:#d08770;">1</span><span>) </span><span>socket.</span><span style="color:#bf616a;">sendall</span><span>(</span><span style="color:#b48ead;">b</span><span>&quot;</span><span style="color:#96b5b4;">\x07</span><span>&quot;) </span></code></pre> <p>This will send 4 operations that save to the 2 bit register. The binary form of these operations is something like <code>?????ABW</code> where <code>W</code> is the bit that indicates a set operation, <code>A</code> and <code>B</code> are the values to save and the <code>?</code> bits are ignored. This little script cycles through all 4 states.</p> <p><video controls="" src="/videos/mad-computer-science/lc-mightwork.mp4"></video></p> <p>I was a little surprised this looked stable. My thought was that on the rising edge of <code>Clk</code> the telnet block would check its internal status and then set <code>av</code> if incoming data was available. If that was the case, what would the <code>rd</code> input be when <code>Clk</code> rises? Not knowing why something doesn't work can be a pain, but not knowing why it <em>does</em> work is a little more frustrating!</p> <p>Then I noticed the <code>av</code> output may actually be asynchronous, which would explain the stability. I thought I'd slow that down and inspect it a bit more.</p> <p><video controls="" src="/videos/mad-computer-science/lc-async-av.mp4"></video></p> <p>Ah! That confirms it. For the sake of science though... what if I made it synchronous? I'll add an <code>AND</code> gate to test and see!</p> <p><video controls="" src="/videos/mad-computer-science/lc-very-broken.mp4"></video></p> <p>Oh my, that is entirely broken. Good to know! </p> <p>On the rising edge of <code>Clk</code>, the telnet block and the <code>AND</code> gate below it recieve this transition at approximately the same time. The <code>AND</code> gate gets it's true condition and shifts its output to a <code>1</code>, but the telnet block also reads <code>rd</code> as the clock transition happened which is probably before the <code>AND</code> gate had time to update its output. The tiny delay is just enough guarantee the telnet block is never read from and breaks everything.</p> <p>I'll remove that <code>AND</code> gate, it was definitely better before that.</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="works-on-my-box">Works On My Box</h1> <p>This is probably something worth fixing, or at least investigating more, for a real project. In this case I am not feeling too worried about it. If <code>av</code> goes high before or after <code>Clk</code> there will be no issues. When I manually caused a timing issue this prevented a read, but in a design where <code>av</code> is allowed to stay synchronous that would simply add a minor delay.</p> <p>At the end of the day, I'm using the telnet block so this design can only really ever run in Digital. As a single threaded software application, I'd bet the behavior is certainly deterministic enough for my silly needs.</p> <p>I'm on to finishing up some final bits of this project to handle read commands as well. I'll just need to trigger the <code>wr</code> input when <code>av</code> is set and <code>out[0]</code> is not.</p> <p><video controls="" src="/videos/mad-computer-science/lc-readify.mp4"></video></p> <p>I think that may do it, but I'll need a bit more Python for testing.</p> <p>In this case, I'm going to make a little script that I'll include into my terminal to use as a cheap terminal interface and use via <code>ipython</code>. It'll just define a few lazily written functions to perform the two types of interactions.</p> <pre data-lang="python" style="background-color:#2b303b;color:#c0c5ce;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#65737e;">#!/usr/bin/env python </span><span style="color:#b48ead;">from </span><span>socket </span><span style="color:#b48ead;">import </span><span>create_connection </span><span style="color:#b48ead;">import </span><span>time </span><span> </span><span style="color:#bf616a;">ADDRESS </span><span>= (&#39;</span><span style="color:#a3be8c;">127.0.0.1</span><span>&#39;, </span><span style="color:#d08770;">1337</span><span>) </span><span> </span><span>socket = </span><span style="color:#bf616a;">create_connection</span><span>(</span><span style="color:#bf616a;">ADDRESS</span><span>) </span><span> </span><span> </span><span style="color:#b48ead;">def </span><span style="color:#8fa1b3;">set_lights</span><span>(</span><span style="color:#bf616a;">light1</span><span>, </span><span style="color:#bf616a;">light2</span><span>): </span><span> </span><span style="color:#b48ead;">if </span><span>light1: </span><span> </span><span style="color:#b48ead;">if </span><span>light2: </span><span> socket.</span><span style="color:#bf616a;">sendall</span><span>(</span><span style="color:#b48ead;">b</span><span>&quot;</span><span style="color:#96b5b4;">\x07</span><span>&quot;) </span><span> </span><span style="color:#b48ead;">else</span><span>: </span><span> socket.</span><span style="color:#bf616a;">sendall</span><span>(</span><span style="color:#b48ead;">b</span><span>&quot;</span><span style="color:#96b5b4;">\x03</span><span>&quot;) </span><span> </span><span style="color:#b48ead;">else</span><span>: </span><span> </span><span style="color:#b48ead;">if </span><span>light2: </span><span> socket.</span><span style="color:#bf616a;">sendall</span><span>(</span><span style="color:#b48ead;">b</span><span>&quot;</span><span style="color:#96b5b4;">\x05</span><span>&quot;) </span><span> </span><span style="color:#b48ead;">else</span><span>: </span><span> socket.</span><span style="color:#bf616a;">sendall</span><span>(</span><span style="color:#b48ead;">b</span><span>&quot;</span><span style="color:#96b5b4;">\x01</span><span>&quot;) </span><span> </span><span> </span><span style="color:#b48ead;">def </span><span style="color:#8fa1b3;">read_status</span><span>(): </span><span> socket.</span><span style="color:#bf616a;">sendall</span><span>(</span><span style="color:#b48ead;">b</span><span>&quot;</span><span style="color:#96b5b4;">\x00</span><span>&quot;) </span><span> x = socket.</span><span style="color:#bf616a;">recv</span><span>(</span><span style="color:#d08770;">1</span><span>)[</span><span style="color:#d08770;">0</span><span>] </span><span> </span><span style="color:#96b5b4;">print</span><span>(&quot;</span><span style="color:#d08770;">{:04b}</span><span>&quot;.</span><span style="color:#bf616a;">format</span><span>(x)) </span><span> </span></code></pre> <p>Now for the fun part, does it work!?</p> <p><video controls="" src="/videos/mad-computer-science/lc-worky-work.mp4"></video></p> <p>Eh... almost! There were some crossed wires.</p> <p>These were quick enough to fix, I apparently tied the same LED to two bits of my read bus and the bit I thought was my second switch was actually the output of my first register bit. Once I spotted those though, everything was pretty simple to fix and worked like a charm!</p> <p>There are certainly a variety of ways to improve this design, but I'm pretty content with where it's at for this post.</p> <p>See you in the next one!</p> 2022-10-28T00:00:00+00:00 2022-10-28T00:00:00+00:00 https://suchprogramming.com/mad-computer-science-part-2/ <p>In the previous post, I implemented a single decimal digit display that used &quot;hardware&quot; pretty minimally. The digital logic side of the design simply split each bit of a byte to allow a software controlled means of controlling each segment of the display.</p> <p><img src="/images/mad-computer-science/telnet-seggy.png" alt="Telnet Seggy" /></p> <p>In this post I'll expand on this design and explore a few more core concepts in hardware tinkery.</p> <p>The primary goal for this post is to use the 7-segment displays to view the hexademical values for the binary data we're delivering through the Telnet block. When I send an <a href="https://upload.wikimedia.org/wikipedia/commons/1/1b/ASCII-Table-wide.svg">ASCII</a> <code>K</code> to the design, I want to see the <code>4B</code> displayed in the simulated displays.</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h2 id="what-the-hex">What the Hex</h2> <p>In the last post I had a table for which segments should be lit for <code>0-9</code>. I'll need <code>A-F</code> as well, so I'll add them to that table.</p> <table><thead><tr><th align="center">Value </th><th align="center"> Segments </th><th align="center"> Hexadecimal</th></tr></thead><tbody> <tr><td align="center"><code>0</code></td><td align="center"><code>--543210</code></td><td align="center"><code>3F</code></td></tr> <tr><td align="center"><code>1</code></td><td align="center"><code>-----21-</code></td><td align="center"><code>06</code></td></tr> <tr><td align="center"><code>2</code></td><td align="center"><code>-6-432-0</code></td><td align="center"><code>5D</code></td></tr> <tr><td align="center"><code>3</code></td><td align="center"><code>-6--3210</code></td><td align="center"><code>4F</code></td></tr> <tr><td align="center"><code>4</code></td><td align="center"><code>-65--21-</code></td><td align="center"><code>66</code></td></tr> <tr><td align="center"><code>5</code></td><td align="center"><code>-65-3-10</code></td><td align="center"><code>6B</code></td></tr> <tr><td align="center"><code>6</code></td><td align="center"><code>-6543-10</code></td><td align="center"><code>7B</code></td></tr> <tr><td align="center"><code>7</code></td><td align="center"><code>----321-</code></td><td align="center"><code>0E</code></td></tr> <tr><td align="center"><code>8</code></td><td align="center"><code>-6543210</code></td><td align="center"><code>7F</code></td></tr> <tr><td align="center"><code>9</code></td><td align="center"><code>-65-3210</code></td><td align="center"><code>6F</code></td></tr> <tr><td align="center"><code>A</code></td><td align="center"><code>-654321-</code></td><td align="center"><code>7E</code></td></tr> <tr><td align="center"><code>B</code></td><td align="center"><code>-654--10</code></td><td align="center"><code>73</code></td></tr> <tr><td align="center"><code>C</code></td><td align="center"><code>--543--0</code></td><td align="center"><code>39</code></td></tr> <tr><td align="center"><code>D</code></td><td align="center"><code>-6-4-210</code></td><td align="center"><code>57</code></td></tr> <tr><td align="center"><code>E</code></td><td align="center"><code>-6543--0</code></td><td align="center"><code>79</code></td></tr> <tr><td align="center"><code>F</code></td><td align="center"><code>-6543---</code></td><td align="center"><code>78</code></td></tr> </tbody></table> <p>My plan is to make a couple small components to organize my implementation for this. I'll start with a single 7-segment driver that takes a 4-bit (nibble) input and drives 1 segment. Once I have that, I'll combine two of them into 1 block that can drive 2 segments.</p> <p>I'll start by planning out my driver design, first dropping a display in the schematic and making an input bus.</p> <p><video controls="" src="/videos/mad-computer-science/segment-addr-bus.mp4"></video></p> <p>So now I need to light up the right segments based on the values of 4 inputs. I thought of a few different ways for how this could be done.</p> <h2 id="logic-gates-logically">Logic Gates, Logically</h2> <p>I decided to see what it'd take to light up segment 6 for the right inputs. I started by looking at which numbers should and should not be lit for this segment.</p> <p><img src="/images/mad-computer-science/bit-numbery.png" alt="Segmented bits" /></p> <p>Should be lit:</p> <table><thead><tr><th align="center">Hexadecimal</th><th align="center">  Binary</th></tr></thead><tbody> <tr><td align="center"><code>2</code></td><td align="center"><code>0010</code></td></tr> <tr><td align="center"><code>3</code></td><td align="center"><code>0011</code></td></tr> <tr><td align="center"><code>4</code></td><td align="center"><code>0100</code></td></tr> <tr><td align="center"><code>5</code></td><td align="center"><code>0101</code></td></tr> <tr><td align="center"><code>6</code></td><td align="center"><code>0110</code></td></tr> <tr><td align="center"><code>8</code></td><td align="center"><code>1000</code></td></tr> <tr><td align="center"><code>9</code></td><td align="center"><code>1001</code></td></tr> <tr><td align="center"><code>A</code></td><td align="center"><code>1010</code></td></tr> <tr><td align="center"><code>B</code></td><td align="center"><code>1011</code></td></tr> <tr><td align="center"><code>D</code></td><td align="center"><code>1101</code></td></tr> <tr><td align="center"><code>E</code></td><td align="center"><code>1110</code></td></tr> <tr><td align="center"><code>F</code></td><td align="center"><code>1111</code></td></tr> </tbody></table> <p>Should not be lit</p> <table><thead><tr><th align="center">Hexadecimal</th><th align="center">  Binary</th></tr></thead><tbody> <tr><td align="center"><code>0</code></td><td align="center"><code>0000</code></td></tr> <tr><td align="center"><code>1</code></td><td align="center"><code>0001</code></td></tr> <tr><td align="center"><code>7</code></td><td align="center"><code>0111</code></td></tr> <tr><td align="center"><code>C</code></td><td align="center"><code>1100</code></td></tr> </tbody></table> <p>I don't see any obvious simple pattern for implementing this with logic gates. Since the display is lit 3/4 of the time, my quick implemention idea is to use 4-input <code>AND</code> gates with some inverted inputs to match each pattern that should not be lit, and input all of those into a 4-input <code>NOR</code> gate.</p> <p>I'll roughly layout the logic gates and route <code>0</code> to the remaining segments for now.</p> <p><video controls="" src="/videos/mad-computer-science/seggy-and-wiring.mp4"></video></p> <p>Each gate will match a condition where the segment shouldn't be lit. For when a bit value I'm looking for is <code>0</code>, I'll flip that invert on its way into the gate. I'll struggle a little bit to wire things up properly then give it a little bit of manual testing.</p> <p><video controls="" src="/videos/mad-computer-science/logic-6.mp4"></video></p> <h2 id="proper-engineering">&quot;Proper&quot; engineering</h2> <p>While this solution only has 5 gates in my schematic, 4-input gates are basically each made of 3 gates. So this design would take at least 15 gates, before including the inverted bits... which is pretty inefficient. There are better, boolean-mathical ways of addressing this, and the simulator has a bit of tooling to assist in avoiding the maths.</p> <p>If I add and name an output, and name the inputs, I can ask the analysis tool to do it's thing. It'll generate a truth table for me. At the bottom of that table I can find a simplified function that'll generate the same output. I can also bring up the <a href="https://en.wikipedia.org/wiki/Karnaugh_map">Karnaugh map</a>, though I don't find it particularly helpful here.</p> <p><video controls="" src="/videos/mad-computer-science/seggy-analyze.mp4"></video></p> <p>That's still... not exactly small is it? </p> <p><img src="/images/mad-computer-science/seggy-6-optimized.png" alt="Seggy Optimized" /></p> <p>I'm not particularly interesting in doing that 6 more times, so I'm going to change my approach.</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h2 id="cheat-codes">Cheat Codes</h2> <p>Instead of going through all that, I'm going to use a LUT (Look-Up Table) to design this a bit more like the original software-oriented route. I'll toss all of this implementation out the door and wire a new LUT block into the design.</p> <p><video controls="" src="/videos/mad-computer-science/seggy-redo.mp4"></video></p> <p>Next, I'll load up a file that contains the hex values of the digits.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>v2.0 raw </span><span>3f </span><span>06 </span><span>5d </span><span>4f </span><span>66 </span><span>6b </span><span>7b </span><span>0e </span><span>7f </span><span>6f </span><span>7e </span><span>73 </span><span>39 </span><span>57 </span><span>79 </span><span>78 </span></code></pre> <p><video controls="" src="/videos/mad-computer-science/seggy-loadtable.mp4"></video></p> <p>Well! That was a heck of a lot quicker and easier. I'll just tweak the inputs and outputs so that I can save this design and use it elsewhere.</p> <p><video controls="" src="/videos/mad-computer-science/seggy-display.mp4"></video></p> <p>Then I'll quickly make a dual-display variant, that simply takes in an 8-bit bus and outputs two segment busses. If I open straight to a new file, the simulator doesn't pick up my other files in the <code>Components</code> menu. If I choose a <code>New embedded Circuit</code>, it will find those files.</p> <p><video controls="" src="/videos/mad-computer-science/seggy-dual.mp4"></video></p> <p>Finally, I'll go back into my Telnet schematic and wire up everything to use my new block. To keep things looking relatively tidy, I'll pre-wire the segments to space things out then collapse those individual signals into 4-bit busses. I'll then split the 8-bit segments out into top and bottom busses. I used the horizontal offset of the <code>8:4*2</code> splitters to help me visually remember that S1 is on the left and S0 on the right.</p> <p><video controls="" src="/videos/mad-computer-science/seggy-buildemup.mp4"></video></p> <h2 id="that-darn-impedence">That Darn Impedence</h2> <p>With everything wired up, things do work, but there's a little bit of undesired behavior.</p> <p><video controls="" src="/videos/mad-computer-science/seggy-ub.mp4"></video></p> <p>Before and after I send my sequence through the Telnet block, the displays will show a random value due to the inputs that aren't being driven to a valid state for reading.</p> <p>I need somewhere to latch onto just the valid inputs at just the right times, which is exactly what flip-flops are here for!</p> <blockquote> <p>Latches and flip-flops are circuits for storing bits of information. <a href="https://www.youtube.com/playlist?list=PLTd6ceoshpreKyY55hA4vpzAUv9hSut1H">This playlist on YouTube</a> covers the standard variants</p> </blockquote> <p>I generally prefer the D-type flip-flops, as they are simple and good for registers. I just feed my data into <code>D</code>, and when <code>C</code> transitions from <code>0</code> to <code>1</code> the value will be stored and output on <code>Q</code> until the next rising clock (the opposite values will be output on the inverted <code>Q</code>, unused here)</p> <p><video controls="" src="/videos/mad-computer-science/seggy-flippy-flop.mp4"></video></p> <p>I also added the <code>AND</code> gate here so that the flip-flop will only latch to the value when the <code>av</code> and clock signals are high. This still leaves me an unexpected result at the end of my sequence.</p> <p>The problem here is that both the Telnet block and the flip-flop latch when the clock transitions to <code>1</code>. Does the Telnet block update fast enough to allow this? Probably not.</p> <p>Though, if I invert the clock the latching will happen one half-cycle after the Telnet block has updated its value. Giving the Telnet block a little time to change its output before storing it.</p> <p><video controls="" src="/videos/mad-computer-science/seggy-clockinvert.mp4"></video></p> <p>There we have it, a decently working hexadecimal display! With the goal complete I'll call it a wrap for this post.</p> <p>I hope you enjoyed this post, stay tuned for the next one!</p> <h6 id="bonus-material">Bonus material</h6> <p>While you wait for the next post, here are a couple good videos to check out:</p> <ul> <li><a href="https://www.youtube.com/watch?v=vxSExMj9lV8">Combination Circuit Vs Sequential Circuit</a> by <a href="https://www.youtube.com/c/ChinmayaASV">Chinmaya A.S.V</a></li> <li><a href="https://www.youtube.com/watch?v=7zffjsXqATg">Designing a 7-segment hex decoder</a> by <a href="https://www.youtube.com/c/BenEater">Ben Eater</a></li> <li><a href="https://www.youtube.com/watch?v=VkVe_wNU6RI">5.2.4 D Register Timing</a> by <a href="https://www.youtube.com/c/mitocw">MIT OpenCourseWare</a></li> </ul> 2022-08-16T00:00:00+00:00 2022-08-16T00:00:00+00:00 https://suchprogramming.com/mad-computer-science-part-1/ <p>Programmable logic is my favorite concept in computing, integrated circuits that let you define hardware behavior. This is an excessively powerful concept that is just waiting for the open source community to do amazing things with.</p> <p>For many hobby electronics projects, you can use a software-focused approach and use an Arduino or Raspberry Pi to accomplish a wide variety of tasks. I've enjoyed that approach but, I want to go a bit more off road than mere software allows. Programmable logic lets me, someone who should not [yet] be paid to design custom hardware, do it anyway!</p> <p><img src="/images/mad-computer-science/we-dont-need-roads.gif" alt="Where we're goin'" /></p> <p>I'd like to share a path that I think makes it a lot easier and more fun to get into. It'll start with tinkering in a virtualized hardware environment that lets you explore a bit more in the comfort of your own programming language.</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h2 id="virtual-tinkering">Virtual Tinkering</h2> <p>I want to bring more software programmers into the world of tinkering with programmable logic, but jumping head first into HDL is kinda hard and it helps to have a lot more context. I found it was really helpful to play with some of schematic based logic design tools first, but what I could accomplish with them felt pretty quickly constrained.</p> <p>I recently came across a digital logic designer and simulator tool called <a href="https://github.com/hneemann/Digital">Digital</a>, and it is pretty awesome. It's an open source, Java-based program that runs pretty simply on Linux, MacOS and Windows.</p> <p>This tool has all the features I want in a simulator, plus a few extras that surprised and excited me. It has the core stuff like logic gates, buttons and LEDs, and a few nice higher level components like flip-flops, multiplexers and 7-segment displays and even an LED matrix! A few components I'd never seen in a free simulator before, like blocks for keyboard input, VGA, MIDI and blocks for VHDL and Verilog.</p> <p>A couple blocks feel particularly ready for some mad computer science...</p> <h2 id="setting-the-stage">Setting the Stage</h2> <p>Digital is super easy to obtain and run, <a href="https://github.com/hneemann/Digital#download-and-installation">you can check here to see just how easy</a>. When you first launch the program it'll offer a short tutorial that helps quickly acclimate to placing components in the schematic, wiring them together and simulating the design.</p> <p>Start your new schematic and go to Components -&gt; IO -&gt; Peripherals and add a Telnet component.</p> <p><img src="/images/mad-computer-science/telnet-block.png" alt="Did I say Telnet?" /></p> <p>I chuckled when I saw the name Telnet, but I am definitely going to use this component because it sounds awesome. How does it work? I'll just hit play</p> <p><img src="/images/mad-computer-science/error-in-input.png" alt="Error number 1" /></p> <p>Alright, it's mad at me for not feeding it. It needs some inputs so I'll wire a button into all the inputs.</p> <p><img src="/images/mad-computer-science/button-up.png" alt="Button up" /></p> <p>This will definitely work</p> <p><img src="/images/mad-computer-science/error-in-input2.png" alt="Error number 2" /></p> <p>Okay fine, maybe some docs would be useful. If I right click on this component I get the following prompt.</p> <p><img src="/images/mad-computer-science/telnet-menu1.png" alt="Telnet menu defaults" /></p> <p>I'm going to change a few settings while I'm here, like turning <code>Telnet Mode</code> off so that it's a simple TCP server and I'll pick <code>1337</code> since that's a not a <a href="https://www.w3.org/Daemon/User/Installation/PrivilegedPorts.html">privileged port</a>.</p> <p><img src="/images/mad-computer-science/telnet-menu2.png" alt="My settings" /></p> <p>And look a handy help button!</p> <p><img src="/images/mad-computer-science/telnet-help.png" alt="Telnet help" /></p> <p>So based on what I'm seeing here, I'll wire up my inputs a bit more sanely. I don't plan on writing data out yet, so I'll set <code>in</code> and <code>wr</code> to 0, feed the output of a clock to <code>C</code>, and a basic input for <code>rd</code></p> <p><video controls="" src="/videos/mad-computer-science/wire-up-telnet.mp4"></video></p> <p>Next, I'll wire up a Terminal block as the output. I'll add a new button to <code>en</code>able the terminal, route the <code>out</code> from the Telnet block to the <code>D</code>ata input, and plumb the same clock signal into <code>C</code></p> <p><video controls="" src="/videos/mad-computer-science/wire-out-telnet.mp4"></video></p> <p>I think we're looking about ready to go!</p> <h2 id="hello-digital-world">Hello Digital World</h2> <p>Now for the fun part, when I simulate this design I'll be able to connect to the Telnet block on <code>127.0.0.1:1337</code> and data I pass into that socket connection should emit from the block. I like that this lets me hook into my design, relatively simply, with any language of my choice.</p> <p>My go-to language for quick hackyness is Python, so I'll write a little script to send some data:</p> <pre data-lang="python" style="background-color:#2b303b;color:#c0c5ce;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#65737e;">#!/usr/bin/env python </span><span style="color:#b48ead;">from </span><span>socket </span><span style="color:#b48ead;">import </span><span>create_connection </span><span> </span><span style="color:#bf616a;">ADDRESS </span><span>= (&#39;</span><span style="color:#a3be8c;">127.0.0.1</span><span>&#39;, </span><span style="color:#d08770;">1337</span><span>) </span><span> </span><span>socket = </span><span style="color:#bf616a;">create_connection</span><span>(</span><span style="color:#bf616a;">ADDRESS</span><span>) </span><span>socket.</span><span style="color:#bf616a;">sendall</span><span>(</span><span style="color:#b48ead;">b</span><span>&quot;</span><span style="color:#a3be8c;">Hello, Digital World!</span><span style="color:#96b5b4;">\n</span><span>&quot;) </span></code></pre> <p>There's a slight problem before running this though. Before I even send input, activating the terminal will spit out some junk data. I'll enable my clock and show you want I mean:</p> <p><video controls="" src="/videos/mad-computer-science/terminal-garbage.mp4"></video></p> <p>If you notice, before I let the terminal body-block the schematic, the data in the line is <code>Z</code>. This state is called &quot;high impedance&quot;, which basically means nothing is driving this signal and is also referred to as &quot;floating&quot;. Effectively this means we're going to be getting some undefined behavior!</p> <p>To address this, and make things a little bit more automatic, I'm going to route the <code>av</code>ailable signal to both the <code>en</code>able input for the Terminal block and the <code>rd</code> (read) input of the Telnet block.</p> <p><video controls="" src="/videos/mad-computer-science/improved-telnet.mp4"></video></p> <p>This should work a bit nicer. I'll now run my python script while the simulation is also going. The 1hz clock signal I sent before was painfully slow, so I'm going to crank that up to a whopping 10hz!</p> <p><video controls="" src="/videos/mad-computer-science/hello-digital-world.mp4"></video></p> <p>It works! I know it's a bit silly, but now I have this interface that I can use to interact with little simulated hardware designs. I think that's actually pretty nifty.</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h2 id="more-softwarification">More Softwarification</h2> <p>Before wrapping this post, I want to use this pattern to do something a bit more hardware oriented. Certainly <code>cat</code> or <code>nc</code> can offer pretty similar functionality to this without all the fuss!</p> <p>Considering the output is 8-bit, the first thing that comes to mind for me is the 7-segment display, which is more of an 8-segment display really! I can use each bit of a byte sent to the Telnet block to map directly to a display segment.</p> <p>In the video below, I remove the Terminal bits and replace it with the 7-segment display. To open up the 8-bit <code>out</code> bus I'll use the <code>Splitter/Merger</code> component and take the 8-bit input and split it into 8 1-bit outputs, the <code>1*8</code> expression helps me specify this. Then I route the wires to the display</p> <p><video controls="" src="/videos/mad-computer-science/telnet-to-segment.mp4"></video></p> <p>I definitely didn't practice this like a half dozen times to create the facade that I know what I'm doing...</p> <p>I'll modify the output of my python script to light up 1 bit at a time. I've played enough with <a href="https://www.youtube.com/watch?v=thrx3SBEpL8">binary numbers</a> to have picked up a few tricks so I'll just use some hexadecimal encoded characters for this.</p> <pre data-lang="python" style="background-color:#2b303b;color:#c0c5ce;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#65737e;">#!/usr/bin/env python </span><span style="color:#b48ead;">from </span><span>socket </span><span style="color:#b48ead;">import </span><span>create_connection </span><span> </span><span style="color:#bf616a;">ADDRESS </span><span>= (&#39;</span><span style="color:#a3be8c;">127.0.0.1</span><span>&#39;, </span><span style="color:#d08770;">1337</span><span>) </span><span> </span><span>socket = </span><span style="color:#bf616a;">create_connection</span><span>(</span><span style="color:#bf616a;">ADDRESS</span><span>) </span><span>socket.</span><span style="color:#bf616a;">sendall</span><span>(</span><span style="color:#b48ead;">b</span><span>&quot;</span><span style="color:#96b5b4;">\x00\x01\x02\x04\x08\x10\x20\x40\x80</span><span>&quot;) </span></code></pre> <p>Running this while simulating, I can visually identify which bits goes to which segment.</p> <p><video controls="" src="/videos/mad-computer-science/what-bits.mp4"></video></p> <p>It was a little handy to have a little video clip of this running too, so I could take a screenshot and label my semi-arbitrary segment numbering!</p> <p><img src="/images/mad-computer-science/bit-numbery.png" alt="Segmented bits" /></p> <p>With this, I'll make one more reference for what segments I want lit for each number.</p> <table><thead><tr><th align="center">Number </th><th align="center"> Segments </th><th align="center"> Hexadecimal</th></tr></thead><tbody> <tr><td align="center"><code>0</code></td><td align="center"><code>--543210</code></td><td align="center"><code>3F</code></td></tr> <tr><td align="center"><code>1</code></td><td align="center"><code>-----21-</code></td><td align="center"><code>06</code></td></tr> <tr><td align="center"><code>2</code></td><td align="center"><code>-6-432-0</code></td><td align="center"><code>5D</code></td></tr> <tr><td align="center"><code>3</code></td><td align="center"><code>-6--3210</code></td><td align="center"><code>4F</code></td></tr> <tr><td align="center"><code>4</code></td><td align="center"><code>-65--21-</code></td><td align="center"><code>66</code></td></tr> <tr><td align="center"><code>5</code></td><td align="center"><code>-65-3-10</code></td><td align="center"><code>6B</code></td></tr> <tr><td align="center"><code>6</code></td><td align="center"><code>-6543-10</code></td><td align="center"><code>7B</code></td></tr> <tr><td align="center"><code>7</code></td><td align="center"><code>----321-</code></td><td align="center"><code>0E</code></td></tr> <tr><td align="center"><code>8</code></td><td align="center"><code>-6543210</code></td><td align="center"><code>7F</code></td></tr> <tr><td align="center"><code>9</code></td><td align="center"><code>-65-3210</code></td><td align="center"><code>6F</code></td></tr> </tbody></table> <p>For my final trick, I'll make a countdown. I'll tick down from 9 to 0, then flash the whole display a couple times with <code>0x00</code> and <code>0xFF</code>.</p> <pre data-lang="python" style="background-color:#2b303b;color:#c0c5ce;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#65737e;">#!/usr/bin/env python </span><span style="color:#b48ead;">from </span><span>socket </span><span style="color:#b48ead;">import </span><span>create_connection </span><span> </span><span style="color:#bf616a;">ADDRESS </span><span>= (&#39;</span><span style="color:#a3be8c;">127.0.0.1</span><span>&#39;, </span><span style="color:#d08770;">1337</span><span>) </span><span> </span><span>socket = </span><span style="color:#bf616a;">create_connection</span><span>(</span><span style="color:#bf616a;">ADDRESS</span><span>) </span><span>socket.</span><span style="color:#bf616a;">sendall</span><span>(</span><span style="color:#b48ead;">b</span><span>&quot;</span><span style="color:#96b5b4;">\x00\x6f\x7f\x0e\x7b\x6b\x66\x4f\x5d\x06\x3f</span><span>&quot;) </span><span>socket.</span><span style="color:#bf616a;">sendall</span><span>(</span><span style="color:#b48ead;">b</span><span>&quot;</span><span style="color:#96b5b4;">\x00\xff\x00\xff\x00\xff</span><span>&quot;) </span></code></pre> <p>I'm hoping that'll work!</p> <p><video controls="" src="/videos/mad-computer-science/countdown.mp4"></video></p> <p>It does!</p> <p>With that, we have our little software foot in the door for some hardware tinkering. The Telnet block certainly won't easily translate to a real world design, but this pattern will be very useful for the tinkering to come. If nothing else, it allows for some experimentation through a mix of software and hardware!</p> <p>I hope you enjoyed this post, stay tuned for the next one!</p> <h6 id="bonus-material">Bonus material</h6> <p>While you wait for the next post, here are a couple good videos to check out:</p> <ul> <li><a href="https://www.youtube.com/watch?v=gI-qXk7XojA">Boolean Logic &amp; Logic Gates: Crash Course Computer Science #3</a> by <a href="https://www.youtube.com/c/crashcourse">Crash Course</a></li> <li><a href="https://www.youtube.com/playlist?list=PLowKtXNTBypGqImE405J2565dvjafglHU">Building an 8-bit breadboard computer</a> by <a href="https://www.youtube.com/c/BenEater">Ben Eater</a></li> <li><a href="https://www.youtube.com/playlist?list=PLilenfQGj6CEG6iZ4TQJ10PI7pCWsy1AO">Building A CPU From Scratch</a> by <a href="https://www.youtube.com/c/rj45Creates">rj45 Creates</a></li> </ul> 2022-08-07T00:00:00+00:00 2022-08-07T00:00:00+00:00 https://suchprogramming.com/mad-computer-science-part-0/ <p>I strongly believe we're approaching a revolution in computer technology, and I'd like you to join me in making that happen!</p> <p>The performance growth of computers is stalling out. The smart folks that build our modern CPUs, GPUs and other hardware peripherals still have some tricks up their sleeve, but we're continuing to approach some very difficult limitations. CPU <a href="https://www.maketecheasier.com/why-cpu-clock-speed-isnt-increasing/">clock speeds have reached a plateau</a>, and we’re starting to build <a href="https://newsroom.ibm.com/2021-05-06-IBM-Unveils-Worlds-First-2-Nanometer-Chip-Technology,-Opening-a-New-Frontier-for-Semiconductors">transistors that are just 4 silicon atoms wide</a>. We’re headed to a technology era where the mostly centralized model of computing is reaching its peak.</p> <p>The computer engineering field is heavily software focused, with about <a href="https://www.bls.gov/ooh/computer-and-information-technology/software-developers.htm">29 software engineers</a> for every <a href="https://www.bls.gov/ooh/architecture-and-engineering/computer-hardware-engineers.htm#tab-1">1 hardware engineer</a> in the US. This ratio is increasing too, with software jobs expected to grow by 22% over the next ten years compared to 2% for hardware. The Open Source Software movement made dramatic improvements in the accessibility of software design, and while there have been significant strides in Open Source Hardware, we still haven't democratized hardware design to nearly the same extent.</p> <p>The hardware side of computing needs more software engineers to dive deeper into the hardware world. It didn't used to be reasonably possible to tinker with hardware design as an amateur due to prohibitively expensive and complex design tools. This dam is finally starting to break, with amazing efforts such as the <a href="https://f4pga.org/">F4PGA</a> project opening up FPGA based logic design and the <a href="https://efabless.com/open_shuttle_program">Efabless Open MPW Program</a> offering anyone the chance to make a small ASIC with Open Source design tools. I believe these types of projects will be invaluable over the long term.</p> <p>It's intimidating to get into hardware and logic design, but it is getting easier, more open and more approachable every year. I think the main element we're missing for a <a href="https://iscaconf.org/isca2018/turing_lecture.html">paradigm shift in computing</a> is more communities of hackers and tinkerers experimenting with hardware designs in willy-nilly fashion, bringing fresh eyes and novel ideas to this area of design that's been traditionally infeasible for most of us.</p> <p>In the aspiration of contributing to this change, I'm hoping to share the fun of experimenting with mad computer science. I'm not a professionally trained hardware engineer, and you don't have to be either! In my journey, I've found that many aspects of hardware or logic design aren't nearly as difficult as they seem to be to tinker with as a noob, but there's definitely a lot to learn with fewer entry level resources available.</p> <p>In this post series I'll dig into various computer engineering topics and demonstrate simple and inexpensive ways to tinker with these concepts. I hope curious folks will check it out and begin their own experimentation that may lead to incredible outcomes in the future!</p> 2022-05-21T00:00:00+00:00 2022-05-21T00:00:00+00:00 https://suchprogramming.com/rusty-macros/ <p>I've been learning and loving Rust for something like 9 months now, but I haven't yet gotten a good grip on its various macro systems. So in this post I'll be exploring and sharing what I learn about writing macros in Rust. I'll start with a brief exploration of the different types of macros Rust supports, a few random anecdotes and then get into some declarative macro tinkering!</p> <h1 id="what-are-we-even-looking-at">What are we even looking at?</h1> <p>Macros in Rust can provide a lot of utility and they can help simplify the process of writing code. Built-in macros like <code>print!</code>, <code>write!</code>, <code>vec!</code> and others are sprinkled throughout my code. Many of the dependencies I often use provide handy macros as well like <a href="https://docs.rs/tokio/latest/tokio/attr.main.html"><code>#[tokio::main]</code></a>, <a href="https://docs.serde.rs/serde_json/macro.json.html"><code>json!</code> from serde</a> or the <a href="https://docs.rs/thiserror/latest/thiserror/index.html">derive and attribute macros provided by <code>thiserror</code></a>. I've looked with curiosity at how some of the macros I've used are implemented, but for the most part my experience with that has given me the same feelings I get when I look at my old Perl code... &quot;wtf I am I even looking at?&quot;</p> <p>I've dabbled with enough complex systems to know that I learn better with a bottom-up approach, so I'll start with the very basic building blocks and work my way up to increasingly more useful and complex designs. I've read that there are different types of macros, so enumerating those types feels like the best place to start.</p> <p>Looking at <a href="https://doc.rust-lang.org/book/ch19-06-macros.html">The Book</a> and <a href="https://blog.logrocket.com/macros-in-rust-a-tutorial-with-examples/#typesofmacrosinrust">this post by LogRocket</a>. These are the types of macros in Rust:</p> <ul> <li>Declarative macros via <code>macro_rules!</code></li> <li>Procedural macros <ul> <li>Derive macros</li> <li>Attribute-like macros</li> <li>Function-like procedural macros</li> </ul> </li> </ul> <p>Generally, macros are tools for code generation. Between declarative and procedural macros, the former seems to be more common and the latter more advanced, so I'll explore the declarative macro systems first.</p> <p>Just before jumping into declarative macros, I also want to call out <a href="https://veykril.github.io/tlborm/introduction.html">The Little Book of Rust Macros</a> as another great resource to have on hand. Often when I see or hear mentions of this book, it's followed by a warning that the information is outdated. <a href="https://github.com/veykril/tlborm">This version is a fork</a> that's actively maintained though, so it is very much worth consideration.</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="i-declare">I declare!</h1> <p>When I look at the example macros in the docs, references and posts, my first reaction has usually been a strong aversion to the syntax. Right from the start, declarative macros begin with <code>macro_rules!</code>, which certainly comes off as declarative expression to me but... is this really the syntax/keyword for this? I was glad to find <a href="https://github.com/rust-lang/rust/issues/8224">I'm not alone</a> in thinking this should be something more like <code>macro!</code>, and <a href="https://github.com/rust-lang/rust/issues/39412">I'm hoping someday that'll be what makes this post outdated</a>. (See also: <a href="https://github.com/rust-lang/rfcs/issues/440">Rust RFC #440</a> and <a href="https://github.com/rust-lang/rfcs/pull/1584">Rust RFC #1584</a>)</p> <p><img src="/images/rusty-macros/odoyle-rules.gif" alt="Macro Rules!" /></p> <p>It's partially comforting to learn that this is a result of <a href="https://en.wikibooks.org/wiki/Scheme_Programming/Macros">Scheme's influence on Rust</a> and not just an odd choice (though it certainly foreshadows the <a href="https://i.redd.it/9gwghrz9rcs11.jpg">Lispy business</a> to come). I know there are people that have a strong adoration for Lisp and they definitely have good reasons to feel that way, but my personal opinion is that Lisp dialects are usually more academic than they are practical. I really like when non-programmers can look at my code and follow along and that just doesn't seem feasible to me with a Lisp dialect. When I see the 7 closing parenthesis in a row in a tiny example I feel like I'm looking at a syntax made for machines rather than human beings.</p> <pre data-lang="scm" style="background-color:#2b303b;color:#c0c5ce;" class="language-scm "><code class="language-scm" data-lang="scm"><span>(define-syntax </span><span style="color:#b48ead;">while </span><span> (syntax-rules () </span><span> ((_ condition . body) </span><span> (</span><span style="color:#b48ead;">let loop </span><span>() </span><span> (</span><span style="color:#b48ead;">cond </span><span>(condition </span><span> (begin . body) </span><span> (</span><span style="color:#b48ead;">loop</span><span>))))))) </span></code></pre> <p>15% parenthesis by volume! (or 20% if you remove non-essential whitespace)</p> <p><img src="/images/rusty-macros/yo-lisp.jpg" alt="Yo Lisp!" /></p> <p>With that context in place, and the cathartic vent, I already feel more prepared to enter <a href="https://esolangs.org/wiki/Parenthesis_Hell">Parenthesis Hell</a>. Don't fret though! I'm still talking about Rust, so at least we're going to <a href="https://en.wikipedia.org/wiki/Elysium">the Elysian plain</a> and not <a href="https://en.wikipedia.org/wiki/Tartarus">Tartarus</a>.</p> <h1 id="basic-syntax-and-structure">Basic syntax and structure</h1> <p>I want to start with a very simple macro that uses minimum syntax and build up from there. For this, I'm mostly referencing <a href="https://doc.rust-lang.org/reference/macros-by-example.html">The Rust Reference - Macros By Example</a>. I also ended up on <a href="https://doc.rust-lang.org/rust-by-example/macros.html">Rust By Example - macro_rules!</a> by accident at least twice thinking &quot;I swear this page looked different 5 minutes ago&quot;, both are good to keep handy!</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#96b5b4;">macro_rules! </span><span>my_simplest_macro_yet { </span><span> () =&gt; {} </span><span>} </span><span> </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> my_simplest_macro_yet!(); </span><span>} </span></code></pre> <p>Alright, first goal achieved! It's <em>technically</em> a macro, and I'd say it certainly achieves the goal of doing nothing quite well.</p> <p>While looking at the syntax tree for <code>macro_rules!</code> I noticed that <a href="https://users.rust-lang.org/t/braces-square-brackets-and-parentheses-in-macro-call/9984/7">you can also use parenthesis or square brackets</a> instead of curly brackets as long as you also add a semicolon at the end:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#96b5b4;">macro_rules! </span><span>my_simplest_macro_yet [ </span><span> () =&gt; {} </span><span>]; </span></code></pre> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#96b5b4;">macro_rules! </span><span>my_simplest_macro_yet ( </span><span> () =&gt; {} </span><span>); </span></code></pre> <p>As far as I can tell there is no real difference between these three. I don't think I've actually seen these last two formats in the wild... but hey, if you want to be the oddball here's one way to do it!</p> <p>As described in <a href="https://doc.rust-lang.org/book/ch19-06-macros.html">The Book</a>, you'll write declarative macros in Rust using a pattern matching syntax very similar to what is employed for <code>match</code> expressions. If the macro inputs match the thingies (matcher) to the left of the <code>=&gt;</code>, apply the block of codestuffs (transcriber) from the right side.</p> <p>The inputs you provide to macros like <code>println!</code> are not <em>normal</em> inputs. In the world of Rust macros, these inputs are lexical tokens of the language, so your actual code is the input to it. This is also the case on the output side as well. <code>println!</code> doesn't <em>technically</em> perform writing to standard out, but it generates code that will.</p> <p>It'll be handy to see the actual results of the macros, so let's explore how to see their expanded outputs before dabbling with their inputs.</p> <h1 id="show-me-the-macro">Show me the macro!</h1> <p>Since the macros aren't <em>normal</em> code, debugging them isn't quite normal either. I want some way to experiment and inspect the results, and <a href="https://github.com/dtolnay/cargo-expand">cargo-expand</a> currently looks like the simplest way to do that.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cargo install cargo-expand </span></code></pre> <p>To take a look at what's going on, I'll run <code>cargo expand</code>:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ cargo expand </span><span> Checking rusty-macros v0.1.0 </span><span>error: the option `Z` is only accepted on the nightly compiler </span><span>error: could not compile `rusty-macros` </span></code></pre> <p>Or not, I suppose! I'll run <code>rustup default nightly</code> to appease the deities of macro expansion and then try again.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ cargo expand </span><span> Checking rusty-macros v0.1.0 </span><span> Finished dev [unoptimized + debuginfo] target(s) in 0.08s </span></code></pre> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span>#![</span><span style="color:#bf616a;">feature</span><span>(prelude_import)] </span><span>#[</span><span style="color:#bf616a;">prelude_import</span><span>] </span><span style="color:#b48ead;">use </span><span>std::prelude::rust_2021::*; </span><span>#[</span><span style="color:#bf616a;">macro_use</span><span>] </span><span style="color:#b48ead;">extern crate</span><span> std; </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">main</span><span>() {} </span></code></pre> <p>Hmm... well I guess I'm not sure what I actually expected here, but this does seem like a pretty reasonable implementation of my <em>does nothing at all</em> macro! Maybe it's a little more useful to consider what the default Rust &quot;Hello World&quot; expands to.</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> println!(&quot;</span><span style="color:#a3be8c;">Hello, world!</span><span>&quot;); </span><span>} </span></code></pre> <p><code>cargo expand</code></p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span>#![</span><span style="color:#bf616a;">feature</span><span>(prelude_import)] </span><span>#[</span><span style="color:#bf616a;">prelude_import</span><span>] </span><span style="color:#b48ead;">use </span><span>std::prelude::rust_2021::*; </span><span>#[</span><span style="color:#bf616a;">macro_use</span><span>] </span><span style="color:#b48ead;">extern crate</span><span> std; </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> { </span><span> ::std::io::_print(::core::fmt::Arguments::new_v1(&amp;[&quot;</span><span style="color:#a3be8c;">Hello, world!</span><span style="color:#96b5b4;">\n</span><span>&quot;], &amp;[])); </span><span> }; </span><span>} </span></code></pre> <p>This is a bit more educational, as I can see the <code>println!</code> macro has indeed been replaced with code that will do the needful. I also get the same expansion if I modify by basic macro to wrap around the same functionality like this:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#96b5b4;">macro_rules! </span><span>say_hello_world { </span><span> () =&gt; { </span><span> println!(&quot;</span><span style="color:#a3be8c;">Hello, world!</span><span>&quot;) </span><span> } </span><span>} </span><span> </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> say_hello_world!() </span><span>} </span></code></pre> <p>What if I add a touch of conditional logic to the matching block for this macro? I'm interested to see what happens (this is my kind of computer science right here: less math and more experimentation!)</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#96b5b4;">macro_rules! </span><span>say_hello_world { </span><span> () =&gt; { </span><span> </span><span style="color:#b48ead;">if </span><span style="color:#d08770;">1 </span><span>&gt; </span><span style="color:#d08770;">0 </span><span>{ </span><span> println!(&quot;</span><span style="color:#a3be8c;">Hello, world!</span><span>&quot;) </span><span> } </span><span> } </span><span>} </span><span> </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> say_hello_world!() </span><span>} </span></code></pre> <p>What say you, <code>cargo expand</code>?</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> </span><span style="color:#b48ead;">if </span><span style="color:#d08770;">1 </span><span>&gt; </span><span style="color:#d08770;">0 </span><span>{ </span><span> { </span><span> ::std::io::_print(::core::fmt::Arguments::new_v1(&amp;[&quot;</span><span style="color:#a3be8c;">Hello, world!</span><span style="color:#96b5b4;">\n</span><span>&quot;], &amp;[])); </span><span> } </span><span> } </span><span>} </span></code></pre> <p>There are two things about this result I find interesting.</p> <p>The first observation is that my always-true <code>if</code> condition survived expansion. I added this useless condition to test if/where it would be optimized away. Indeed, I don't see this condition <em>actually</em> implemented in the compiled program, which indicates to me that the optimization pass(es) occur sometime later in the compilation process. This makes total sense though, as optimization during macro expansion would likely be a bit premature. It feels like something useful to be aware of.</p> <p>The second observation is not quite as useful: <code>println!</code> seems to be implemented in a way that that nests the output in its own block. This results in the expansion coming out as <code>if 1 &gt; 0 {{ print }}</code>... <code>(maybe (equal ((it) (&quot;just that Lisp influence again&quot;))))</code>?</p> <p>I'm also a little tickled that <code>print!</code> doesn't have these extra curlies:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#96b5b4;">macro_rules! </span><span>say_hello_world { </span><span> () =&gt; { </span><span> </span><span style="color:#b48ead;">if </span><span style="color:#d08770;">1 </span><span>&gt; </span><span style="color:#d08770;">0 </span><span>{ </span><span> print!(&quot;</span><span style="color:#a3be8c;">Hello, world!</span><span style="color:#96b5b4;">\n</span><span>&quot;) </span><span> } </span><span> } </span><span>} </span><span> </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> say_hello_world!() </span><span>} </span></code></pre> <p><code>cargo expand</code></p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> </span><span style="color:#b48ead;">if </span><span style="color:#d08770;">1 </span><span>&gt; </span><span style="color:#d08770;">0 </span><span>{ </span><span> ::std::io::_print(::core::fmt::Arguments::new_v1(&amp;[&quot;</span><span style="color:#a3be8c;">Hello, world!</span><span style="color:#96b5b4;">\n</span><span>&quot;], &amp;[])) </span><span> } </span><span>} </span></code></pre> <p>Seeing the expanded macros should be pretty helpful for debugging, and with that it's time for some tinkering with macro inputs!</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="charon-s-obol">Charon's obol</h1> <p><a href="https://en.wikipedia.org/wiki/Charon%27s_obol">Charon's obol?!</a> Pardon me, I'm probably enjoying myself a little too much with the Parenthesis Hell and Hades references. I hope you don't mind though because it's time to visit the ferryman (he just wants your tokens, dammit!)</p> <p><img src="/images/rusty-macros/facepalm.jpg" alt=":facepalm:" /></p> <p>To start off as simply as possible, I'll first look at matching some simple literals in my macro.</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#96b5b4;">macro_rules! </span><span>say_what { </span><span> (true) =&gt; { </span><span> print!(&quot;</span><span style="color:#a3be8c;">Hello, world!</span><span style="color:#96b5b4;">\n</span><span>&quot;) </span><span> }; </span><span> (false) =&gt; { </span><span> print!(&quot;</span><span style="color:#a3be8c;">Goodbye world?</span><span style="color:#96b5b4;">\n</span><span>&quot;) </span><span> }; </span><span> (2) =&gt; { </span><span> print!(&quot;</span><span style="color:#a3be8c;">Hello, worlds!</span><span style="color:#96b5b4;">\n</span><span>&quot;) </span><span> } </span><span>} </span><span> </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> say_what!(</span><span style="color:#d08770;">2</span><span>) </span><span>} </span></code></pre> <p>It does exactly what I'm expecting it to!</p> <p><code>Hello, worlds!</code></p> <p>Looking at how this expands, I can verify that this <code>say_what!</code> macro indeed acts a lot like a <code>match</code> expression (so far).</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> ::std::io::_print(::core::fmt::Arguments::new_v1(&amp;[&quot;</span><span style="color:#a3be8c;">Hello, worlds!</span><span style="color:#96b5b4;">\n</span><span>&quot;], &amp;[])) </span><span>} </span></code></pre> <p>Post-expansion there are no indications left over of the original macro, the inputs to it, nor the un-matched branches. This macro isn't quite useful though, so it'll be much more interesting to implement something that's closer to a more realistic goal.</p> <h1 id="if-at-first-you-don-t-succeed-that-makes-me-feel-better-because-neither-did-i">If at first you don't succeed... that makes me feel better because neither did I</h1> <p>The example I have in mind is a <code>retry!</code> macro. The scenario I'm giving myself is that I have some kind of network function that happens to use <a href="https://en.wikipedia.org/wiki/IP_over_Avian_Carriers">IPoAC</a>. Since this protocol has pretty questionable reliability I'd like to retry the request a couple times before giving up.</p> <p><img src="/images/rusty-macros/pigeon.gif" alt="DJ pidgey" /></p> <p>Here's where I'm starting from:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">use </span><span>rand::prelude::*; </span><span style="color:#b48ead;">use </span><span>anyhow::{Result,anyhow}; </span><span> </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">pigeon_post</span><span>() -&gt; Result&lt;()&gt; { </span><span> </span><span style="color:#b48ead;">if </span><span style="color:#96b5b4;">random</span><span>() { </span><span> Ok(()) </span><span> } </span><span style="color:#b48ead;">else </span><span>{ </span><span> Err(anyhow!(&quot;</span><span style="color:#a3be8c;">The pigeon got distracted in a grocery store parking lot</span><span>&quot;)) </span><span> } </span><span>} </span><span> </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> </span><span style="color:#b48ead;">match </span><span style="color:#96b5b4;">pigeon_post</span><span>() { </span><span> Ok(_) =&gt; println!(&quot;</span><span style="color:#a3be8c;">Success: The pigeon arrived!</span><span>&quot;), </span><span> Err(error) =&gt; println!(&quot;</span><span style="color:#a3be8c;">Error: </span><span style="color:#d08770;">{}</span><span>&quot;, error) </span><span> } </span><span>} </span></code></pre> <p>I want my <code>retry!</code> macro to take 2 arguments: the identifier of the unreliable function to call and a numeric literal for the maximum amount of attempts to make.</p> <p>The first couple ways I tried to implement this macro weren't going very well, even when I could see the problems pretty clearly using <code>cargo expand</code>. What ended up working for me was to go backwards: writing the result I want from the macro, moving that into the new macro, setting up the inputs then refactoring those inputs into the transcribed code.</p> <p>So starting off in <code>main()</code>, I wrote what I wanted out of the macro:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> </span><span style="color:#b48ead;">for </span><span>_ in </span><span style="color:#d08770;">0</span><span>..</span><span style="color:#d08770;">3 </span><span>{ </span><span> </span><span style="color:#b48ead;">match </span><span style="color:#96b5b4;">pigeon_post</span><span>() { </span><span> Ok(_) =&gt; { </span><span> println!(&quot;</span><span style="color:#a3be8c;">Success: The pigeon arrived!</span><span>&quot;); </span><span> </span><span style="color:#b48ead;">return </span><span> } </span><span> Err(error) =&gt; { </span><span> println!(&quot;</span><span style="color:#a3be8c;">Error: </span><span style="color:#d08770;">{}</span><span>&quot;, error) </span><span> } </span><span> } </span><span> } </span><span> println!(&quot;</span><span style="color:#a3be8c;">We should probably hire a better pigeoneer</span><span>&quot;) </span><span>} </span></code></pre> <p>To setup the inputs, I'll need to use <a href="https://doc.rust-lang.org/reference/macros-by-example.html#metavariables">metavariables</a>. These are written in the format <code>$&lt;NAME&gt;:&lt;FRAGMENT-SPECIFIER&gt;</code>. The fragment specifiers declare what kind of Rust syntax we're looking to match on and capture. Picking the right fragment specifier seems like one of those things you generally get better at with time and practice. In my case, I'm using <code>ident</code> to pass in the identifier of the function to call and <code>literal</code> for the number of attempts to make.</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#96b5b4;">macro_rules! </span><span>retry { </span><span> (</span><span style="color:#bf616a;">$iffy_call</span><span>:</span><span style="color:#b48ead;">ident</span><span>, $max_attempts:literal) =&gt; { </span><span> ... </span><span> }; </span><span>} </span></code></pre> <p>Now all I need is a bit of copy-pasta and to replace some syntax with my metavariables.</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">use </span><span>rand::prelude::*; </span><span style="color:#b48ead;">use </span><span>anyhow::{Result,anyhow}; </span><span> </span><span style="color:#96b5b4;">macro_rules! </span><span>retry { </span><span> (</span><span style="color:#bf616a;">$iffy_call</span><span>:</span><span style="color:#b48ead;">ident</span><span>, $max_attempts:literal) =&gt; { </span><span> </span><span style="color:#b48ead;">for </span><span>_ in </span><span style="color:#d08770;">0</span><span>..</span><span style="color:#bf616a;">$max_attempts </span><span>{ </span><span> </span><span style="color:#b48ead;">match </span><span style="color:#bf616a;">$iffy_call</span><span>() { </span><span> Ok(_) =&gt; { </span><span> println!(&quot;</span><span style="color:#a3be8c;">Success: The pigeon arrived!</span><span>&quot;); </span><span> </span><span style="color:#b48ead;">return </span><span> } </span><span> Err(error) =&gt; { </span><span> println!(&quot;</span><span style="color:#a3be8c;">Error: </span><span style="color:#d08770;">{}</span><span>&quot;, error) </span><span> } </span><span> } </span><span> } </span><span> println!(&quot;</span><span style="color:#a3be8c;">We should probably hire a better pigeoneer</span><span>&quot;) </span><span> }; </span><span>} </span><span> </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">pigeon_post</span><span>() -&gt; Result&lt;()&gt; { </span><span> </span><span style="color:#b48ead;">if </span><span style="color:#96b5b4;">random</span><span>() { </span><span> Ok(()) </span><span> } </span><span style="color:#b48ead;">else </span><span>{ </span><span> Err(anyhow!(&quot;</span><span style="color:#a3be8c;">The pigeon got distracted in a grocery store parking lot</span><span>&quot;)) </span><span> } </span><span>} </span><span> </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> retry!(pigeon_post, </span><span style="color:#d08770;">3</span><span>); </span><span>} </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Error: The pigeon got distracted in a grocery store parking lot </span><span>Error: The pigeon got distracted in a grocery store parking lot </span><span>Success: The pigeon arrived! </span></code></pre> <p>There it is, a working declarative retry macro! That's where I'll call it a wrap on this post. I hope you've found this entertaining and/or helpful in your own Rust adventures.</p> <p>Originally I was planning to make this a little series on Rust macros, but as I researched my topic ideas I kept finding other good resources that did a great job of covering those topics. I'm not averse to reinventing the wheel, but only when I think I have a good idea! So instead, here's some other resources I thought could be pretty helpful:</p> <ul> <li>A few macro guides that re-implement <code>vec!</code>: <ul> <li><a href="https://doc.rust-lang.org/book/ch19-06-macros.html">The Rust Book</a></li> <li><a href="https://www.youtube.com/watch?v=q6paRBbLgNw">The Crust of Rust</a></li> </ul> </li> <li>Similarly, some resources that work to implement a <code>hashmap!</code> macro: <ul> <li><a href="https://stackoverflow.com/questions/27582739/how-do-i-create-a-hashmap-literal">This stack overflow answer by Shepmaster</a></li> <li><a href="https://medium.com/@phoomparin/a-beginners-guide-to-rust-macros-5c75594498f1">A Beginner's Guide to Rust Macros</a></li> <li><a href="https://github.com/rust-lang/rfcs/issues/542">Eventually we'll get this in <code>std</code></a></li> </ul> </li> <li>Also pretty good: <a href="https://danielkeep.github.io/practical-intro-to-macros.html">A Practical Intro to Macros in Rust 1.0</a></li> </ul> 2018-12-11T00:00:00+00:00 2018-12-11T00:00:00+00:00 https://suchprogramming.com/fpga-meets-6502/ <p>So I'm working on designing my own homebrew 6502-based microcomputer. I have a lot of ideas for features I'd like, but before I make too many crazy decisions I'd like to solidify my understanding of the processor I'm building the whole system around. To this end, I am grabbing my <a href="https://store.digilentinc.com/arty-a7-artix-7-fpga-development-board-for-makers-and-hobbyists/">Arty-A7</a>, a <a href="http://www.westerndesigncenter.com/wdc/w65c02s-chip.cfm">W65C02S</a> mounted on a breadboard and a bag of jumper cables to wire them together.</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="planning-phase">Planning Phase</h1> <p>I chose to use my Arty-A7 FPGA board because it has a ton of I/O pins, it can easily meet my performance requirements and because I just plain love tinkering with FPGAs.</p> <p>This will take a smidge of organization to get started. I began by taking an inventory of the various pin functions on the W65C02S package.</p> <p><img src="/images/fpga-meets-6502/W65C02S-40-pin-dip-color.gif" alt="Pin diagram for W65C02S from Western Design Center Product Documentation" /></p> <p>Generally speaking, there are 16 pins for the address bus, 8 pins for the data bus, 13 control pins, 2 power pins and 1 no-connect. This means I’ll need to wire 37 IO pins + 2 power pins to my FPGA board. With that in mind I’ve decided to use a mix of the <a href="https://en.wikipedia.org/wiki/Pmod_Interface">PMOD</a> interfaces on my Arty board as well as the Arduino-shield pins next to them. Each PMOD has 8 IO pins so I’ll use 1 PMOD for the data bus and 2 for the address bus, with the control pins on adjacent shield header. I’ll pull power from one of the PMODs as well.</p> <p><img src="/images/fpga-meets-6502/plan.png" alt="Rough Pin Planning" /></p> <p>With my rough plan in place, I took some notes on how my 6502 pins will map to the specific pins. I planned for which PMOD pin each bit of the address and data buses will go, along with associating each control pin to a numbered shield pin. Now for the fun part!</p> <h1 id="wired-up">Wired Up</h1> <p>Now that the rough planning is complete, I carefully use the wire jumpers to connect everything as planned. I wired one pin at a time roughly starting from the top of the 6502 and working down, it’s a bit of a mess but it does the job.</p> <p><img src="/images/fpga-meets-6502/franken6502.jpg" alt="The resulting mess of wires" /></p> <p>Now that the plumbing is in place, the real fun begins. I start by opening up Vivado and begin getting these pins mapped into a design. I’ll create a new <code>top.sv</code> file to kick off the design.</p> <pre data-lang="sv" style="background-color:#2b303b;color:#c0c5ce;" class="language-sv "><code class="language-sv" data-lang="sv"><span style="color:#d08770;">`timescale 1ns </span><span>/ </span><span style="color:#d08770;">1ps </span><span> </span><span style="color:#b48ead;">module </span><span>top ( </span><span> </span><span style="color:#65737e;">// Arty stuff </span><span> input </span><span style="color:#b48ead;">logic</span><span> clk_100mhz, </span><span> input </span><span style="color:#b48ead;">logic</span><span> reset, </span><span> </span><span style="color:#65737e;">// 6502 stuff </span><span> input </span><span style="color:#b48ead;">logic</span><span> [</span><span style="color:#d08770;">15</span><span>:</span><span style="color:#d08770;">0</span><span>] address, </span><span> inout </span><span style="color:#b48ead;">logic</span><span> [</span><span style="color:#d08770;">7</span><span>:</span><span style="color:#d08770;">0</span><span>] data, </span><span> input </span><span style="color:#b48ead;">logic</span><span> vpb, </span><span> input </span><span style="color:#b48ead;">logic</span><span> phi1o, </span><span> input </span><span style="color:#b48ead;">logic</span><span> phi2o, </span><span> input </span><span style="color:#b48ead;">logic</span><span> rwb, </span><span> input </span><span style="color:#b48ead;">logic</span><span> mlb, </span><span> input </span><span style="color:#b48ead;">logic</span><span> sync, </span><span> output </span><span style="color:#b48ead;">logic</span><span> resb, </span><span> output </span><span style="color:#b48ead;">logic</span><span> sob, </span><span> output </span><span style="color:#b48ead;">logic</span><span> phi2, </span><span> output </span><span style="color:#b48ead;">logic</span><span> be, </span><span> output </span><span style="color:#b48ead;">logic</span><span> irqb, </span><span> output </span><span style="color:#b48ead;">logic</span><span> nmib, </span><span> output </span><span style="color:#b48ead;">logic</span><span> rdy </span><span>); </span><span> </span><span style="color:#b48ead;">endmodule </span></code></pre> <p>The next step is to tediously map the various pins I planned out to the pins on the pins I’ve defined in my <code>top</code> module. The PMOD pins are easily found in the <a href="https://reference.digilentinc.com/reference/programmable-logic/arty-a7/reference-manual">reference wiki for the board</a>, but I didn’t see the IO shield pins on the primary documentation page, so I referred to the <a href="https://reference.digilentinc.com/reference/programmable-logic/arty-a7/schematic">provided schematic files</a> for that information. With all that pin mapping information, I added the pins properties into a constraints file.</p> <p><img src="/images/fpga-meets-6502/6502pins.png" alt="A portion of the XDC Constraint File" /></p> <p>At this point, everything looks to be ready for tinkering. As an initial experiment to validate that it is working, I’ll use <img src="https://www.xilinx.com/products/intellectual-property/vio.html" alt="Vivado’s VIO (Virtual Input/Output)" /> IP block to manually probe and interact with these pins. Though, just before I generate that, I’ll add a few more lines to my <code>top.sv</code> to support <a href="https://suchprogramming.com/beginning-logic-design-part-8/">bi-directional</a> use of the data bus. For the 6502, when the <code>rwb</code> pin is high, the CPU is reading from the data bus, so I’ll use that as the trigger to put my <code>write_data</code> on that data bus during those times.</p> <pre data-lang="systemverilog" style="background-color:#2b303b;color:#c0c5ce;" class="language-systemverilog "><code class="language-systemverilog" data-lang="systemverilog"><span style="color:#b48ead;">logic</span><span> [</span><span style="color:#d08770;">7</span><span>:</span><span style="color:#d08770;">0</span><span>] write_data; </span><span style="color:#b48ead;">assign</span><span> data = (rwb) ? write_data : </span><span style="color:#d08770;">&#39;bZ</span><span>, </span><span> sob = </span><span style="color:#d08770;">1</span><span>, </span><span> be = </span><span style="color:#d08770;">1</span><span>, </span><span> irqb = </span><span style="color:#d08770;">1</span><span>, </span><span> nmib = </span><span style="color:#d08770;">1</span><span>, </span><span> rdy = </span><span style="color:#d08770;">1</span><span>; </span></code></pre> <p>Once I am running on real hardware, I’ll be stepping through the reset logic for the 6502. So for this project <code>irqb</code>, <code>be</code>, <code>nmib</code>, <code>rdy</code> and <code>sob</code> can all stay at <code>1</code> for now, as I won’t be using them.</p> <p>Now seems like the time to start building the VIO block. I’ll add ports for the various inputs and outputs, including the register I just added for bi-directional communication on the data bus.</p> <p><img src="/images/fpga-meets-6502/6502vio2.png" alt="" /></p> <p>I’ll let Vivado synthesize the core, and I’ll add it to the <code>top</code> module.</p> <pre data-lang="systemverilog" style="background-color:#2b303b;color:#c0c5ce;" class="language-systemverilog "><code class="language-systemverilog" data-lang="systemverilog"><span style="color:#b48ead;">vio_0 </span><span>debug_core ( </span><span> .</span><span style="color:#96b5b4;">clk</span><span>(clk_100mhz), </span><span style="color:#65737e;">// input wire clk </span><span> .</span><span style="color:#96b5b4;">probe_in0</span><span>(address), </span><span style="color:#65737e;">// input wire [15 : 0] probe_in0 </span><span> .</span><span style="color:#96b5b4;">probe_in1</span><span>(data), </span><span style="color:#65737e;">// input wire [7 : 0] probe_in1 </span><span> .</span><span style="color:#96b5b4;">probe_in2</span><span>(vpb), </span><span style="color:#65737e;">// input wire [0 : 0] probe_in2 </span><span> .</span><span style="color:#96b5b4;">probe_in3</span><span>(phi1o), </span><span style="color:#65737e;">// input wire [0 : 0] probe_in3 </span><span> .</span><span style="color:#96b5b4;">probe_in4</span><span>(phi2o), </span><span style="color:#65737e;">// input wire [0 : 0] probe_in4 </span><span> .</span><span style="color:#96b5b4;">probe_in5</span><span>(rwb), </span><span style="color:#65737e;">// input wire [0 : 0] probe_in5 </span><span> .</span><span style="color:#96b5b4;">probe_in6</span><span>(mlb), </span><span style="color:#65737e;">// input wire [0 : 0] probe_in6 </span><span> .</span><span style="color:#96b5b4;">probe_in7</span><span>(sync), </span><span style="color:#65737e;">// input wire [0 : 0] probe_in7 </span><span> .</span><span style="color:#96b5b4;">probe_out0</span><span>(write_data), </span><span style="color:#65737e;">// output wire [7 : 0] probe_out0 </span><span> .</span><span style="color:#96b5b4;">probe_out1</span><span>(resb), </span><span style="color:#65737e;">// output wire [0 : 0] probe_out1 </span><span> .</span><span style="color:#96b5b4;">probe_out2</span><span>(phi2) </span><span style="color:#65737e;">// output wire [0 : 0] probe_out2 </span><span>) </span></code></pre> <p>Now it is time for the moment of truth!</p> <h1 id="playing-with-a-running-cpu">Playing with a Running CPU</h1> <p>With all that in place, it’s time to build everything and start playing around. After running synthesis, implementation and generating a bitfile, I can flash my new design and debug core to the Arty board and pull up the VIO interface that shows me the current state of things.</p> <p><img src="/images/fpga-meets-6502/vio.png" alt="VIO showing probe states" /></p> <p>At this point a bit more knowledge on the how the 6502 operates is necessary. The <code>phi2</code> signal is the input clock for the CPU. The core of this 6502 is fully static, so this clock can be stopped at anytime and the state within the CPU preserved. This allows me to interact and step the clock manually through VIO, even though my manual input will be extremely slow for interacting with hardware. </p> <p><video controls="" src="/videos/fpga-meets-6502/vio-clocks.mp4"></video></p> <p>Since the clock looks good, I’m going to perform a reset of the 6502. Near the end of the reset cycle I’m expecting the address bus to read for the reset vector, a pointer to where the CPU should begin execution after reset. For the timing requirements on getting a reset to work properly, the <a href="http://www.westerndesigncenter.com/wdc/documentation/w65c02s.pdf">documentation from WDC</a> states specifically what to do:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>3.11 </span><span>Reset (RESB) </span><span>The Reset (RESB) input is used to initialize the microprocessor and start program execution. The </span><span>RESB signal must be held low for at least two clock cycles after VDD reaches operating voltage. </span><span> </span><span>… </span><span> </span><span>When a positive edge is detected, there will be a reset sequence lasting seven clock cycles. The </span><span>program counter is loaded with the reset vector from locations FFFC (low byte) and FFFD (high </span><span>byte). This is the start location for program control. RESB should be held high after reset for </span><span>normal operation. </span></code></pre> <p>So I’ll hold reset low for at least 2 cycles, then set it back to high. I’ll continue to manually cycle the clock until I see CPU begin to fetch the <code>RESET</code> vector. As that vector is read, I will manually give it the address <code>0xDEAD</code> to begin execution at. Then, to keep it running I’ll give it the opcode <code>0xEA</code> (NOP) so it continues to read empty instructions over the bus. It’s worth noting that the 6502 does most things in alignment to the falling edge of the <code>phi2</code> clock.</p> <p><video controls="" src="/videos/fpga-meets-6502/doing-it.mp4"></video></p> <p>The 6502 has been reset and is running the instructions it’s reading from the FPGA! In the future I’ll extend this design to better simulate an imaginary system. That’ll be my stopping place for today, hope you enjoyed!</p> 2018-05-31T00:00:00+00:00 2018-05-31T00:00:00+00:00 https://suchprogramming.com/new-linux-process-c/ <p>There are many good reasons to spawn other programs in Linux to do your bidding, but most programming languages don’t give you nearly as much control of the process as in C. In this post I will cover some of the most common ways to create new processes and manage them in C on a Linux system.</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="grab-a-fork">Grab a fork</h1> <p>The classic fork() function is the most popular way to create a new process. It works by duplicating the process that calls it. It may sound complicated but it’s a fairly simple system.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>#include &lt;unistd.h&gt; </span><span> </span><span>pid_t fork(void); </span></code></pre> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">unistd.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> pid_t fork_pid = </span><span style="color:#bf616a;">fork</span><span>(); </span><span> </span><span style="color:#b48ead;">if </span><span>(fork_pid == </span><span style="color:#d08770;">0</span><span>) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Hello from the child!</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> } </span><span style="color:#b48ead;">else </span><span>{ </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Hello from the parent!</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> } </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span>$ ./forktest </span><span>Hello from the parent! </span><span>Hello from the child! </span></code></pre> <p>When a process calls <code>fork()</code>, Linux will duplicate that current process. The value returned by <code>fork()</code> will be <code>0</code> in the child process. In the parent process <code>fork()</code> will return the PID (Process ID) of the new child process or <code>-1</code> if some error occurs.</p> <h1 id="replacing-a-running-process">Replacing a Running Process</h1> <p>fter creating a new process, it’s common to replace that child process with an entirely different program. The <code>exec()</code> family of functions can handle this for us. The <code>execl()</code> is the simplest method.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>#include &lt;unistd.h&gt; </span><span>int execl(const char *filename, const char *arg, ...); </span></code></pre> <p>All you need to provide is the location of the file to load, and the arguments you’d like to provide to it. Just like for any normal process, the first process is the process name.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span> </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">unistd.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> </span><span style="color:#bf616a;">execl</span><span>(&quot;</span><span style="color:#a3be8c;">/bin/bash</span><span>&quot;, &quot;</span><span style="color:#a3be8c;">/yes/its/bash</span><span>&quot;, &quot;</span><span style="color:#a3be8c;">-c</span><span>&quot;, &quot;</span><span style="color:#a3be8c;">echo $0 &amp;&amp; uptime</span><span>&quot;, </span><span style="color:#d08770;">NULL</span><span>); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span> } </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./execltest </span><span>/yes/its/bash </span><span>10:11:12 up 1:07, 2 users, load average: 0.07, 0.10, 0.18 </span></code></pre> <p>The <a href="http://man7.org/linux/man-pages/man3/exec.3.html">other functions</a> in the <code>exec()</code> family have various options to control arguments and environment variables for the new process that takes over.</p> <h1 id="playing-with-the-kids">Playing with the Kids</h1> <p>After you have wee ‘lil child processes, you’ll probably want to make sure they are doing as they are told. After you’ve sent the child process off to do its chores, you can use the <code>wait()</code> function to see what it returns with after it’s done.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>#include &lt;sys/wait.h&gt; </span><span> </span><span>pid_t wait(int *status); </span></code></pre> <p>When you call <code>wait()</code>, it will block the parent process until any of it’s children processes change state. The <code>status</code> pointer can be used if you’re interested in knowing what kind of state change has occurred, to determine if the program exited normally, or if it was terminated by a control signal. The return value of <code>wait()</code> is the child PID that has changed states.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">unistd.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">sys/wait.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> pid_t child = </span><span style="color:#bf616a;">fork</span><span>(); </span><span> </span><span> </span><span style="color:#b48ead;">if </span><span>(child) { </span><span> </span><span style="color:#bf616a;">wait</span><span>(</span><span style="color:#d08770;">NULL</span><span>); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">child process terminated</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> } </span><span style="color:#b48ead;">else </span><span>{ </span><span> </span><span style="color:#bf616a;">execl</span><span>(&quot;</span><span style="color:#a3be8c;">/bin/bash</span><span>&quot;, &quot;</span><span style="color:#a3be8c;">/THECHILD</span><span>&quot;, </span><span style="color:#d08770;">NULL</span><span>); </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./waitkids </span><span>$ echo $0 </span><span>/THECHILD </span><span>$ exit </span><span>exit </span><span>child process terminated </span></code></pre> <p>Check <a href="http://man7.org/linux/man-pages/man2/waitpid.2.html">the man page for wait</a> for more details on the various options available when waiting.</p> <h1 id="attack-of-the-clones">Attack of the Clones</h1> <p>The <code>fork()</code>, <code>exec()</code> and <code>wait()</code> families of functions are portable across POSIX compliant systems. If more fine grained control over the process creation is desired, we’ll need to use the Linux specific clone() function.</p> <p>I won’t cover the <code>clone()</code> in this post, as that’s probably more suited for a dedicated post. Regardless, I suggest perusing <a href="http://man7.org/linux/man-pages/man2/clone.2.html">the man page for it</a> to get an idea of what capabilities it offers.</p> <p>That covers the basics of process creation using C on Linux!</p> 2018-03-14T00:00:00+00:00 2018-03-14T00:00:00+00:00 https://suchprogramming.com/debugging-with-gdb-part-3/ <p>In my previous GDB post, I went over many common debugger commands while exploring the stack build up and initialization of stack variables for the <code>main</code> function in a simple C program. In this post I’ll use more GDB commands to further inspect the body of this small program.</p> <p>Quick references:</p> <ul> <li><a href="https://darkdust.net/files/GDB%20Cheat%20Sheet.pdf">GDB Cheat Sheet</a></li> <li>Example program source at: <a href="https://www.suchprogramming.com/debugging-with-gdb-part-1/">Debugging C Programs with GDB – Part 1</a></li> </ul> <h1 id="for-loop-assembly">For Loop Assembly</h1> <p>To dig deeper into the exact operation of my <code>main</code> function, I’ll first look at the block of instructions the compiler built when interpreting and optimizing my C code.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>0x0000000000400575 &lt;+15&gt;: mov DWORD PTR [rbp-0x4],0x0 </span><span>0x000000000040057c &lt;+22&gt;: jmp 0x400596 &lt;main+48&gt; </span><span>0x000000000040057e &lt;+24&gt;: mov eax,DWORD PTR [rbp-0x4] </span><span>0x0000000000400581 &lt;+27&gt;: mov esi,eax </span><span>0x0000000000400583 &lt;+29&gt;: mov edi,0x40064a </span><span>0x0000000000400588 &lt;+34&gt;: mov eax,0x0 </span><span>0x000000000040058d &lt;+39&gt;: call 0x400440 &lt;printf@plt&gt; </span><span>0x0000000000400592 &lt;+44&gt;: add DWORD PTR [rbp-0x4],0x1 </span><span>0x0000000000400596 &lt;+48&gt;: cmp DWORD PTR [rbp-0x4],0x9 </span><span>0x000000000040059a &lt;+52&gt;: jle 0x40057e &lt;main+24&gt; </span><span>0x000000000040059c &lt;+54&gt;: mov rax,QWORD PTR [rip+0x200a9d] </span><span>0x00000000004005a3 &lt;+61&gt;: mov rdi,rax </span><span>0x00000000004005a6 &lt;+64&gt;: call 0x400430 &lt;puts@plt&gt; </span></code></pre> <p>As seen at the end of the previous post, the instruction at <code>&lt;+15&gt;</code> (15 bytes into <code>main</code>) is setting <code>PTR [rbp-4]</code> to <code>0</code>, which for this program is the <code>i</code> iterator variable used in the for loop of <code>main</code>. This is the implementation of the initializer <code>i = 0;</code> in my for loop.</p> <p>The program then unconditionally jumps to <code>&lt;+48&gt;</code>, which compares (<code>cmp</code>) <code>i</code> to <code>9</code>. This compare instruction will change some CPU flags that are use with conditional jumps such as the <code>jle</code> (jump if less than or equal) instruction that follows. This <code>cmp</code>/<code>jle</code> combo is the implementation of the condition of the for loop: <code>i &lt; 10</code>. This shows us that the compiler decided this was better implemented as <code>i &lt;= 9</code>.</p> <p>Now I’ll watch the loop a bit more closely. I’ll start off by adding the breakpoint at the start of main with <code>break main</code>, start the program with <code>run</code>, and <code>disas main</code> to verify where the breakpoint was set.</p> <p><img src="/images/debugging-with-gdb-part-3/break-main.png" alt="" /></p> <p>Now I’ll use<code> watch i</code> to ask GDB to notify me of changes to <code>i</code>. When I continue the program, it’ll run until the value of <code>i</code> is modified, which happens during the <code>add</code> instruction at <code>&lt;main+44&gt;</code>. This instruction, <code>add DWORD PTR [rbp-0x4],0x1]</code>, is the <code>i++</code> step of my for loop.</p> <p><img src="/images/debugging-with-gdb-part-3/watch-i.png" alt="" /></p> <p>With <code>i</code> incremented, the program checks to see if another iteration of the loop should be ran by comparing <code>i</code> to <code>9</code>. When this comparison happens various <a href="https://en.wikibooks.org/wiki/X86_Assembly/X86_Architecture#EFLAGS_Register">CPU flags</a> are modified in the process. If we check the flags before and after the <code>cmp</code>, we can see what’s getting changed.</p> <p><img src="/images/debugging-with-gdb-part-3/flags.png" alt="" /></p> <p>The <code>cmp</code> operation will substitute <code>i</code> by <code>9</code> to see what the difference between them is. In this case the operation was <code>1 - 9</code>, so <code>-8</code>. The flags that <code>jle</code> will look for is the zero flag <code>ZF</code> that indicates the result is <code>0</code> (and the numbers compared are equal) or the sign flag <code>SF</code> that indicates the result of the operations resulted in a negative number. With how this program is written, <code>jle</code> will always jump the program to <code>&lt;main+24&gt;</code> until <code>i</code> is <code>10</code>, as <code>cmp</code> will have cleared both the zero and sign flags at that point so no jump will be taken.</p> <h1 id="calling-function">Calling Function</h1> <p>Inside of this loop, not too much is going on. The body of the loop is setting up some parameters to be passed to the <code>printf()</code> function, then calling it.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>0x000000000040057e &lt;+24&gt;: mov eax,DWORD PTR [rbp-0x4] </span><span>0x0000000000400581 &lt;+27&gt;: mov esi,eax </span><span>0x0000000000400583 &lt;+29&gt;: mov edi,0x40064a </span><span>0x0000000000400588 &lt;+34&gt;: mov eax,0x0 </span><span>0x000000000040058d &lt;+39&gt;: call 0x400440 &lt;printf@plt&gt; </span></code></pre> <p>So I’ll set a new breakpoint at <code>*0x40057e</code> to watch more closely what’s going on at that point. I’ll use <code>nexti</code> and <code>i r &lt;registers&gt;</code> to monitor the register changes during these operations.</p> <p><img src="/images/debugging-with-gdb-part-3/moar-mov.png" alt="" /></p> <p>So generally it looks like <code>eax</code> was used to grab the value of <code>i</code>, then set <code>esi</code> to it. I’m not sure why this was done, but the compiler usually knows what it’s doing so I trust it’s for a good reason. The last <code>mov</code> sets <code>eax</code> to <code>0</code> so the value of <code>i</code> is only set in <code>eax</code> for usage in the next instruction. There’s also a <code>mov</code> here to set <code>edi</code> to <code>0x400440</code>.</p> <p>I referenced the <a href="https://wiki.osdev.org/Calling_Conventions#Cheat_Sheets">calling convention</a> for System V x86_64 and see that <code>rdi</code> and <code>rsi</code> are the first and second registers used as parameters in a function call. Matching my C call to <code>printf</code>, <code>edi</code> is my string and <code>esi</code> is the <code>i</code> variables value. I can take a look at this string with <code>x</code> similar to how I inspected numbers before, but I’ll use <code>s</code> as the display type indicator.</p> <p><img src="/images/debugging-with-gdb-part-3/strangs.png" alt="" /></p> <p>The last instruction of the loop body is simply the <code>call</code> to the address of the <code>printf</code> function.</p> <h1 id="after-the-loop">After the Loop</h1> <p>After our for loop finally completes, there are just 6 more instructions to run before <code>main</code> is complete.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>0x000000000040059c &lt;+54&gt;: mov rax,QWORD PTR [rip+0x200a9d] # 0x601040 </span><span>0x00000000004005a3 &lt;+61&gt;: mov rdi,rax </span><span>0x00000000004005a6 &lt;+64&gt;: call 0x400430 &lt;puts@plt&gt; </span><span>0x00000000004005ab &lt;+69&gt;: mov eax,0x0 </span><span>0x00000000004005b0 &lt;+74&gt;: leave </span><span>0x00000000004005b1 &lt;+75&gt;: ret </span></code></pre> <p>I hope you’re used to what <code>mov</code> does by now, though in this case it’s doing another offset with an important register I haven’t yet touched on, <code>rip</code>. This is the Instruction Pointer, this register keeps holds the value of the next instruction the CPU should execute.</p> <p><img src="/images/debugging-with-gdb-part-3/rip.png" alt="" /></p> <p>Here I restarted my debugger and set a breakpoint at the <code>&lt;main+54&gt;</code> instruction. I can confirm that’s the next instruction to run because <code>rip</code> is set to that same address. It’s interesting that the compiler decided to use an offset from a register that’s always going to be the same value, GDB even noticed this and tells me the final address will be <code>0x602040</code>.</p> <p><img src="/images/debugging-with-gdb-part-3/puts.png" alt="" /></p> <p>Just as before, this is pulling up a string (character pointer) that’ll be used in the <code>puts</code> function call. This pointer has a much different address because of how I defined it in my program. This one was the global variable <code>done</code>, as opposed to my <code>Iteration %s\n</code> string which was a variable in the function scope of <code>main</code>.</p> <p>The last <code>mov</code> of the program is setting <code>eax</code> to <code>0</code>, this register is used as the return value of a function call and I wrote <code>main</code> to return <code>0</code>. The <code>leave</code> begins restoring the stack to as it was before <code>main</code> was called and <code>ret</code> completes the exit of my <code>main</code> function. Internally libc will hold onto the value <code>main</code> returned, do some cleanup then make a system call to <code>exit()</code> with that return value to notify the OS that the program has completed.</p> <p>With all that I hope this provides a helpful introduction to GDB and some x86 assembly. GDB has a ton of features and this only scratches the surface but I wanted to share these basics as I’ll be using them in future posts when I dive more into advanced C topics. </p> 2018-02-28T00:00:00+00:00 2018-02-28T00:00:00+00:00 https://suchprogramming.com/debugging-with-gdb-part-2/ <p>In my <a href="https://suchprogramming.com/debugging-with-gdb-part-1/">previous post</a> I covered a few basics around building a C program for debugging, looking at the code listing and assembly of the program, setting breakpoints and peeking into the registers.</p> <p>For Part 2, I’ll be using more gdb commands to explore the assembly code that’s involved in building up and initializing the stack frame at the start of a function.</p> <h1 id="stacking-up">Stacking Up</h1> <p>I want to dig down into every bit of what this program is doing, I’ll begin by pulling up the disassembly version of the program once more.</p> <p><img src="/images/debugging-with-gdb-part-2/disambly-1.png" alt="" /></p> <p>If I use <code>break main</code> to set a breakpoint at the start of my <code>main</code> function, GDB will be practical and set this breakpoint at the address <code>0x400575</code>, 15 bytes into the assembled version of main. It’ll do this because the first 6 instructions of the function are setting up the stack for the function and normally you can trust that the compiler has done a good job handling that for you.</p> <p>I want to really start at the beginning, so I’m going to instead set my breakpoint with a pointer to the first byte of the function with the command <code>break</code></p> <p><img src="/images/debugging-with-gdb-part-2/break-first.png" alt="" /></p> <p>The first assembly instruction seen here is <code>push rbp</code>. This is pushing the value stored in the <code>rbp</code> (base pointer) register onto the stack, but where is the stack? The <code>rsp</code> (stack pointer) register tell us where the top of the stack currently is.</p> <p><img src="/images/debugging-with-gdb-part-2/stack-registers.png" alt="" /></p> <p>These first 3 instructions are building a new <a href="https://en.wikipedia.org/wiki/Call_stack">call stack frame</a>. Since this program uses libc the program starts by running code to initialize a few things then calls the <code>main</code> function that was created in the source code. The compiler adds these instructions to build out the stack frame for the given function.</p> <p><img src="/images/debugging-with-gdb-part-2/stack-diagram.png" alt="The stack when printf is being called from main" /></p> <p>As mentioned before, the <code>rsp</code> register points to the points to the top of the stack. By running <code>nexti</code> (next instruction), I can execute one machine operation at a time. After letting <code>push rbp</code> run, the current value of <code>rbp</code> is added to the top of the stack. While this happens, the stack grows and <code>rsp</code> will be updated to the new top of the stack.</p> <p><img src="/images/debugging-with-gdb-part-2/top-of-stack.png" alt="" /></p> <p>The value of <code>rbp</code> hasn’t changed, but the value of <code>rsp</code> is 8 bytes smaller. In most CPU architectures the stack grows “down” like this, where the lowest call frame represents the current frame.</p> <h1 id="peeking-into-memory">Peeking Into Memory</h1> <p>We should now be able to see the <code>rbp</code> value at the top of the frame. By using the <code>x</code> command to print data in memory, we can take a look at the frame. <code>x</code> has a few useful options, while you’re learning GDB they can be tricky to remember so I recommend keeping a <a href="http://darkdust.net/files/GDB%20Cheat%20Sheet.pdf">cheat sheet</a> handy until you’ve got it down.</p> <p>To look at the value at the top of the stack, I’ll use <code>x/1xg 0x7fffffffda10</code>. This is asking to see 1 unit of data, in he<code>x</code>adecimal format, considering 64-bit “<code>g</code>iant” words. You can choose instead to dereference the <code>rsp</code> register directly in the command by using <code>$rsp</code> for the address.</p> <p><img src="/images/debugging-with-gdb-part-2/dereference-rsp.png" alt="The value of rbp is indeed at the top of the stack" /></p> <p>With the commands covered so far we can more easily inspect what each instruction is doing. The next instruction is <code>mov rbp, rsp</code>, this moves the value of <code>rsp</code> into <code>rbp</code>. This is to set the base pointer to the top of the previous stack frame which is used at the end of a function to restore the previous stack state.</p> <p><img src="/images/debugging-with-gdb-part-2/set-base.png" alt="Now rbp and rsp are set to the same value" /></p> <p>The next instruction, <code>sub rsp,0x20</code>, lowers the value of <code>rsp</code> by 32 (4 64-bit words). This is the size of the frame being built for main. Using <code>x</code> again, I’ll look at the 4 words in the stack, plus the next word after (the <code>rbp</code> value that was pushed to the stack).</p> <p><img src="/images/debugging-with-gdb-part-2/main-frame.png" alt="" /></p> <p>There’s already data in this stack! This is data left over from previous execution and is considered uninitialized, as this function doesn’t know what data was been left behind here. When gcc warns that <code>'i' is used uninitialized in this function [-Wuninitialized]</code>, it is because the program is using a variable that’s not in a known state.</p> <h1 id="pointers-irl">Pointers IRL</h1> <p>The next 3 instructions are going to be initializing the new stack frame region using pointers. If you’ve been confused about pointers in C, pointers in assembly might help demystify that concept for you. Let’s consider the next 3 instructions:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>=&gt; 0x000000000040056e &lt;+8&gt;: mov DWORD PTR [rbp-0x14],edi </span><span> 0x0000000000400571 &lt;+11&gt;: mov QWORD PTR [rbp-0x20],rsi </span><span> 0x0000000000400575 &lt;+15&gt;: mov DWORD PTR [rbp-0x4],0x0 </span></code></pre> <p>These are assignment operators that refer to addresses relative to a memory address that is stored in a register. In this case they are modifying the data in the new stack frame (addresses below <code>rbp</code>). Sometime before <code>main</code> was called, the <code>edi</code> and <code>rsi</code> registers were set to some values that the compiler wants kept around.</p> <p>The first instruction here is setting a 32-bit (Double Word) at the memory address 20 bytes below <code>rbp</code>. I’ll need to give <code>x</code> a<code> w</code> parameter so that it’ll know I’m now looking for a 32-bit word.</p> <p><img src="/images/debugging-with-gdb-part-2/20-below.png" alt="" /></p> <p>Basically the same thing going on for the next instruction, but setting the memory address 32 bytes below the base pointer to the 64-bit value currently in <code>rsi</code>.</p> <p><img src="/images/debugging-with-gdb-part-2/32-below.png" alt="" /></p> <p>These registers (<code>*di</code>, <code>*si</code>) are <a href="https://stackoverflow.com/questions/1856320/purpose-of-esi-edi-registers">index registers</a> that can be used for various string operations. They are also part of the <a href="https://wiki.osdev.org/Calling_Conventions#Cheat_Sheets">x86_64 calling convention</a> to be used when calling functions. In this case, these registers contain the parameters for <code>main</code>, my <code>argc</code> and <code>argv</code> variables.</p> <p>We can use the <code>print</code> command to look at the variables as they were named in the source code. The <code>&amp;</code> operator can be used similar to how it used in C to verify that these variables are stored at those locations in the stack.</p> <p><img src="/images/debugging-with-gdb-part-2/all-the-pointers.png" alt="" /></p> <p>The next instruction, <code>mov DWORD PTR [rbp-0x4],0x0</code>, is the <code>i</code> variable used within the <code>for</code> loop inside of main, here being initialized to the value we specified, <code>0</code>.</p> <p>At this point the stack for main is initialized and <code>main</code> is ready to do its thing. In the next post I’ll continue the GDB exploration to investigate the rest of what this function is up to!</p> 2018-02-21T00:00:00+00:00 2018-02-21T00:00:00+00:00 https://suchprogramming.com/debugging-with-gdb-part-1/ <p>When you write C code, you’re playing with power! You’re bound to let this power go to your head and shoot yourself in the foot here and there. At some point(s) your program is going to do something that just doesn’t quite make sense.</p> <p>The bad news is that your program doesn’t make any sense because you’ve written flaws into it. That’s fine, you’ve either written janky C programs, or not written any C. The good news is that GDB is here to help us learn from our mistakes!</p> <p>Through the next few posts I’ll share some tips on basic GDB usage, explore a bit of history and dig more into how the C programs on my machine are actually working.</p> <h1 id="building-for-debugging">Building for Debugging</h1> <p>To kick things off, I’m going to just slap together a quick C program and a Makefile to assist in building it and running my debugger.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#65737e;">// test.c </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">char </span><span>*done = &quot;</span><span style="color:#a3be8c;">Done!</span><span>&quot;; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> </span><span style="color:#b48ead;">int</span><span> i; </span><span> </span><span> </span><span style="color:#b48ead;">for </span><span>(i = </span><span style="color:#d08770;">0</span><span>; i &lt; </span><span style="color:#d08770;">10</span><span>; i++) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Iteration </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span>&quot;, i); </span><span> } </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#d08770;">%s</span><span style="color:#96b5b4;">\n</span><span>&quot;, done); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>This program has a simple for loop and a few print statements and I’ll use GDB to inspect what it’s doing a bit more. To provide more information to the debugger about this program I’ll use the <code>-g</code> flag when building it.</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#65737e;"># Makefile </span><span style="color:#bf616a;">CC</span><span>=</span><span style="color:#a3be8c;">gcc -g -o </span><span style="color:#bf616a;">$@</span><span style="color:#a3be8c;"> -Wall </span><span style="color:#bf616a;">$&lt; </span><span> </span><span style="color:#8fa1b3;">all</span><span>: </span><span style="color:#a3be8c;">test </span><span> </span><span style="color:#8fa1b3;">test</span><span>: </span><span style="color:#a3be8c;">test.c </span><span> </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">CC</span><span style="color:#b48ead;">) </span><span> </span><span style="color:#8fa1b3;">debug</span><span>: </span><span style="color:#a3be8c;">test </span><span> </span><span style="color:#bf616a;">gdb -q</span><span> ./test </span></code></pre> <p>For maximum laziness, I added a <code>debug</code> target to my <code>Makefile</code> here so that I can use <code>make debug</code> to jump right it. I gave <code>gdb</code> the <code>-q</code> option to quiet down since it normally has a lot to say on startup.</p> <p><img src="/images/debugging-with-gdb-part-1/make-debug.png" alt="" /></p> <p>That’s about all I need to get my program ready for debugging!</p> <h1 id="basic-commands">Basic Commands</h1> <p>Now we get to the hard part. GDB has a bajillion features so getting started can be daunting. Probably one of the best commands to learn first is the <code>run</code> command, as so far the program has been looked at a little bit, but isn’t actually running at the moment.</p> <p><img src="/images/debugging-with-gdb-part-1/gdb-run.png" alt="" /></p> <p>You can also provide arguments to the program by providing arguments to run. This program doesn’t care about arguments, but don’t let that stop you from giving it some anyway!</p> <p><img src="/images/debugging-with-gdb-part-1/gdb-runargs.png" alt="" /></p> <p>The excitement of just running a program in GDB is very short lived, I want to be able to stop the program somewhere and poke around a bit. The <code>list</code> command can spit out a listing of the program.</p> <p><img src="/images/debugging-with-gdb-part-1/gdb-list.png" alt="" /></p> <p>Initially <code>gdb</code> will show the first 10 lines of the source. You could run <code>list</code> again to see the next 10 lines but GDB has a friendly feature where hitting <code>enter</code> will automatically rerun your last command, so I used that to continue reading the full source.</p> <p>Looking at this listing, I think a good place to pause and look around would be at the <code>printf()</code> call within my <code>for</code> loop. To have GDB stop here I’ll use the <code>break</code> command and I’ll give it the argument <code>10</code> to indicate I’d like to set a breakpoint at line 10.</p> <p><img src="/images/debugging-with-gdb-part-1/break-10.png" alt="" /></p> <p>Now when I give it a run, it’ll stop the program when it hits that line.</p> <p><img src="/images/debugging-with-gdb-part-1/run-break.png" alt="" /></p> <p>To resume the program, until the next breakpoint is hit, you can use the <code>continue</code> command. Another little time-saver trick with gdb is that many commands have shortcuts, such as <code>c</code> for <code>continue</code>.</p> <p><img src="/images/debugging-with-gdb-part-1/gdb-continue.png" alt="" /></p> <h1 id="peeking-into-the-code">Peeking Into The Code</h1> <p>The ability to set breakpoints and resume execution is a good start, but even better is getting a look around at this point in time to glean more about what the program is doing. It’s time to start looking beyond the C code and see what the program is actually doing in assembly, the state of the CPU in the context of our program and what’s going on in memory.</p> <p>First let’s look at the assembly version of the <code>main</code> function. I’ll use the <code>disassemble</code> command for that, and I’ll tell it that <code>main</code> is what I’m interested in disassembling.</p> <p><img src="/images/debugging-with-gdb-part-1/disassemble.png" alt="Oh noes! Assmebly!" /></p> <p>Assembly code get’s a bad rep, but it’s not as bad as people think it is. You might not want to write a large application in assembly, and that’s reasonable, but if you want to be a strong C programmer you need to know enough assembly to figure out what your program is up to.</p> <p>x86_64 assembly has two different syntaxes to choose from, AT&amp;T syntax and Intel syntax. They both work just fine but GDB defaults to AT&amp;T syntax and I prefer the Intel syntax so I’ll use the command <code>set disassembly-flavor intel</code> to get it to my liking.</p> <p><img src="/images/debugging-with-gdb-part-1/intel-asm.png" alt="" /></p> <p>That looks better! Now let’s briefly look at a few things. Looks like my <code>main</code> function is 21 instructions long, alright… a smidge more than half of the operations are <code>mov</code> (move) instructions and I see a few branching operations, <code>jmp</code> (jump), <code>call</code> (call a subroutine), <code>jle</code> (jump if less than or equal to) and <code>ret</code> (return from subroutine).</p> <p>One thing I find interesting is the instruction at offset <code>&lt;+64&gt;</code>, <code>call 0x400430 &lt;puts@plt&gt;</code>. I did not use the <code>puts()</code> function in my code! The compiler caught on that my last <code>printf()</code> statement doesn’t need to be a format string and optimized the result a little bit.</p> <p>Let’s get back to inspecting what this program is up to, I’m currently still in the middle of my paused program, and I’m at the very start of one of my loop iterations. In this disassembly output I can see I’m at offset <code>&lt;+24&gt;</code>, as indicated by the little <code>=&gt;</code> arrow, this is the next instruction the program will run.</p> <p>The <code>mov</code> instruction moves a value from one place to another, similar to the assignment operator <code>=</code> in most programming languages. In this case the full instruction is <code>mov eax,DWORD PTR [rbp-0x4]</code> which is basically <code>eax = DWORD PTR [rbp - 0x4]</code>. Ignoring the right side of that for now, we’re assigning a value to something called <code>eax</code>. This <code>eax</code> thing is a CPU register, which is basically a variable in the hardware of the CPU. We can look at all the registers with the <code>info</code> command by saying <code>info registers</code>.</p> <p><img src="/images/debugging-with-gdb-part-1/registers.png" alt="" /></p> <p>Okay so there are a bunch of registers, and <code>eax</code> is not one of them… GREAT! This is because the x86 architecture has been through a lot, way back in the day (early 70s) Intel released their <a href="https://en.wikipedia.org/wiki/Intel_8008">8008 CPU</a> that had some 8-bit registers with names like <code>A</code> (for Accumulator).</p> <p>When Intel got to the <a href="https://en.wikipedia.org/wiki/Intel_8086">8086</a> in the late 70s they made the <code>A</code> register twice the size (16-bits) and started calling it the <code>AX</code> register. To help with software compatibility with older system the <code>AX</code> register could be used as an 8-bit register with <code>AH</code> representing the higher 8 bits and <code>AL</code> the lower 8 bits.</p> <p>Then the mid-80s showed up and Intel was like MOAR BITS and released their <a href="https://en.wikipedia.org/wiki/Intel_80386">80386</a> that had 32-bit registers, now they refer to the <code>A</code> register as <code>EAX</code> (there’s our guy!), again preserving backward compatibility by allowing the 16 and 8 bit registers to remain the same. Now-a-days our 64-bit processors are king, so we have the 64-bit register <code>RAX</code>, but can still use <code>EAX</code>, <code>AX</code>, <code>AH</code>, and <code>AL</code>.</p> <p>All that history lesson to give full context on why <code>mov eax, &lt;stuff&gt;</code> is going to modify our <code>rax</code> register!</p> <p>Now, to run just that one instruction, I’ll use the <code>nexti</code> command. I’ll then check the registers again with the shorthand version of <code>info registers</code> and just look at the <code>eax</code> register: <code>i r eax</code></p> <p><img src="/images/debugging-with-gdb-part-1/i-r-eax.png" alt="" /></p> <p>If I <code>continue</code> my program, I’ll notice that this number correlates with something in my program.</p> <p><img src="/images/debugging-with-gdb-part-1/iteration-4.png" alt="" /></p> <p>The <code>eax</code> register is getting set to the <code>i</code> value I’m setting during my for loop!</p> <p>In the next post I’ll continue digging into this program and discover more about the disassembled version of my C program and show off some more GDB commands along the way!</p> 2018-02-14T00:00:00+00:00 2018-02-14T00:00:00+00:00 https://suchprogramming.com/exploring-elf/ <p>Hello there! This post is going to be about the <a href="https://en.wikipedia.org/wiki/Executable_and_Linkable_Format">Executable and Linkable Format (ELF)</a>. This is one of the most important binary formats out there and is designed to store <a href="https://en.wikipedia.org/wiki/Object_code">object code</a> in a way that can be used on a wide variety of processor types and operating systems. It’s the most popular format used to contain compiled programs and libraries.</p> <p>Let’s take a file apart and look at what it’s made of!</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="elf-header-of-a-c-program">ELF Header of a C Program</h1> <p>To kick things off I will build a very small C program, <code>test.c</code></p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>I’ll give it a quick and lazy build with <code>make test</code> and look at the first 64 bytes of it with <code>hexdump</code>.</p> <p><img src="/images/exploring-elf/elf-hexdump.png" alt="" /></p> <p>We can see right away this is an ELF file, the first 4 bytes are <code>&quot;\x7fELF&quot;</code>. The ELF format supports 32-bit and 64-bit machines. This header will be 52 bytes long on a 32-bit machine and 64 bytes long on a 64-bit machine to support the longer memory addresses. For this post I’ll only be looking at 64-bit files.</p> <p>Here’s the structure of the 64-bit header from the <a href="https://www.uclibc.org/docs/elf-64-gen.pdf">ELF-64 documentation</a>:</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">typedef struct </span><span>{ </span><span> </span><span style="color:#b48ead;">unsigned char</span><span> e_ident[</span><span style="color:#d08770;">16</span><span>]; </span><span style="color:#65737e;">/* ELF identification */ </span><span> Elf64_Half e_type; </span><span style="color:#65737e;">/* Object file type */ </span><span> Elf64_Half e_machine; </span><span style="color:#65737e;">/* Machine type */ </span><span> Elf64_Word e_version; </span><span style="color:#65737e;">/* Object file version */ </span><span> Elf64_Addr e_entry; </span><span style="color:#65737e;">/* Entry point address */ </span><span> Elf64_Off e_phoff; </span><span style="color:#65737e;">/* Program header offset */ </span><span> Elf64_Off e_shoff; </span><span style="color:#65737e;">/* Section header offset */ </span><span> Elf64_Word e_flags; </span><span style="color:#65737e;">/* Processor-specific flags */ </span><span> Elf64_Half e_ehsize; </span><span style="color:#65737e;">/* ELF header size */ </span><span> Elf64_Half e_phentsize; </span><span style="color:#65737e;">/* Size of program header entry */ </span><span> Elf64_Half e_phnum; </span><span style="color:#65737e;">/* Number of program header entries */ </span><span> Elf64_Half e_shentsize; </span><span style="color:#65737e;">/* Size of section header entry */ </span><span> Elf64_Half e_shnum; </span><span style="color:#65737e;">/* Number of section header entries */ </span><span> Elf64_Half e_shstrndx; </span><span style="color:#65737e;">/* Section name string table index */ </span><span>} Elf64_Ehdr </span></code></pre> <p>A visual representation of this structure:</p> <p><img src="/images/exploring-elf/elf-header.png" alt="" /></p> <p>The <code>e_ident</code> field contains all the information needed to understand what format and version of an ELF file this is. As I mentioned before, the first 4 bytes of this should be <code>\x7f</code>, <code>E</code>, <code>L</code>, <code>F</code>, the <a href="https://en.wikipedia.org/wiki/Magic_number_(programming)">magic number</a> for all ELF files. The remaining bytes of the <code>e_ident</code> field will indicate if the file is 32 or 64-bit, if the file is in little <a href="https://en.wikipedia.org/wiki/Endianness">endian</a> or big endian format and a few other details about what the file was built for.</p> <p>The <code>e_type</code> field indicates what type of data is represented in the file, if it’s a object (<code>1</code>), executable (<code>2</code>), shared object (<code>3</code>) or core file (<code>4</code>). The <code>e_machine</code> field represents what <a href="https://en.wikipedia.org/wiki/Instruction_set_architecture">instruction set</a> the file is in. <code>e_version</code> is always <code>1</code> for now.</p> <p>Up this this point the header is the same for 32-bit and 64-bit machines. The next three fields <code>e_entry</code> (program entry point), <code>e_phoff</code> (program header offset) and <code>e_shoff</code> (section header offset) are used to refer to memory addresses so are twice as long on a 64-bit machine. The <code>e_entry</code> field is a pointer to the start of program in virtual memory. If you’re unfamiliar with what virtual memory is I suggest <a href="https://www.youtube.com/watch?v=2quKyPnUShQ">this video</a> from <a href="https://www.youtube.com/channel/UCgyqtNWZmIxTx3b6OxTSALw">Android Authority’s YouTube channel</a>. The <code>e_phoff</code> and <code>e_shoff</code> indicate the offsets within the ELF file to the program and section headers tables.</p> <p>The rest of the header contains some flags, counts and sizes relating to the program and section headers in the file, I suggest checking out the elf header <a href="https://en.wikipedia.org/wiki/Executable_and_Linkable_Format#File_header">documentation on Wikipedia</a> for more detail on the header.</p> <p>We can easily see all of this header detail in a human readable way with the <code>readelf</code> program.</p> <p><img src="/images/exploring-elf/readelf.png" alt="" /></p> <h1 id="a-simpler-elf-file">A Simpler ELF File</h1> <p>I want to dive a bit deeper into the file, but even an empty C program like this contains a ton of data compared to a small <code>Hello, world!</code> program written in assembly. Here’s my hello world assembly source for a x86_64 Linux machine:</p> <pre data-lang="asm" style="background-color:#2b303b;color:#c0c5ce;" class="language-asm "><code class="language-asm" data-lang="asm"><span style="color:#96b5b4;">section </span><span style="color:#8fa1b3;">.text </span><span> </span><span> </span><span style="color:#96b5b4;">global </span><span style="color:#8fa1b3;">_start </span><span> </span><span style="color:#8fa1b3;">_start: </span><span> </span><span style="color:#65737e;"> ; write(1, msg, len) </span><span style="color:#8fa1b3;"> </span><span style="color:#b48ead;">mov </span><span style="color:#bf616a;">rax</span><span>, </span><span style="color:#d08770;">1 </span><span style="color:#8fa1b3;"> </span><span style="color:#b48ead;">mov </span><span style="color:#bf616a;">rdi</span><span>, </span><span style="color:#d08770;">1 </span><span style="color:#8fa1b3;"> </span><span style="color:#b48ead;">mov </span><span style="color:#bf616a;">rsi</span><span>, </span><span style="color:#8fa1b3;">msg </span><span style="color:#8fa1b3;"> </span><span style="color:#b48ead;">mov </span><span style="color:#bf616a;">rdx</span><span>, </span><span style="color:#8fa1b3;">len </span><span style="color:#8fa1b3;"> </span><span style="color:#b48ead;">syscall </span><span> </span><span style="color:#65737e;"> ; exit(0) </span><span style="color:#8fa1b3;"> </span><span style="color:#b48ead;">mov </span><span style="color:#bf616a;">rax</span><span>, </span><span style="color:#d08770;">60 </span><span style="color:#8fa1b3;"> </span><span style="color:#b48ead;">mov </span><span style="color:#bf616a;">rdi</span><span>, </span><span style="color:#d08770;">0 </span><span style="color:#8fa1b3;"> </span><span style="color:#b48ead;">syscall </span><span> </span><span style="color:#96b5b4;">section </span><span style="color:#8fa1b3;">.data </span><span> </span><span style="color:#8fa1b3;">msg </span><span style="color:#96b5b4;">db </span><span style="color:#a3be8c;">&quot;Hello, world!&quot;</span><span>,</span><span style="color:#d08770;">0xa </span><span style="color:#8fa1b3;">len </span><span style="color:#96b5b4;">equ </span><span style="color:#8fa1b3;">$ </span><span>- </span><span style="color:#8fa1b3;">msg </span></code></pre> <p>I won’t dive into the syntax of this assembly, but it makes one call to Linux to write <code>Hello, world!\n</code> to stdout (<code>1</code>) and a second call to exit the program with <code>0</code>. I’ll assemble it into an elf64 object file with <code>nasm -f elf64 hello.asm</code> and link it into an executable with <code>ld -s -o hello hello.o</code>.</p> <p><img src="/images/exploring-elf/asm-build.png" alt="" /></p> <p>This file is a lot smaller than the C program, 512 bytes vs 8552. There is also a lot less data in the ELF program and sections headers as indicated by the header.</p> <p><img src="/images/exploring-elf/asm-readelf.png" alt="" /></p> <p>There are 2 program headers instead of 9 and 4 sections headers instead of 31, so much less to look at!</p> <h1 id="elf-sections">ELF Sections</h1> <p>Let’s look at more of what <code>readelf</code> can tell us about this ELF file, starting with closer inspection of the sections headers.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ readelf -a hello </span><span>ELF Header: </span><span> Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 </span><span> Class: ELF64 </span><span> Data: 2&#39;s complement, little endian </span><span> Version: 1 (current) </span><span> OS/ABI: UNIX - System V </span><span> ABI Version: 0 </span><span> Type: EXEC (Executable file) </span><span> Machine: Advanced Micro Devices X86-64 </span><span> Version: 0x1 </span><span> Entry point address: 0x4000b0 </span><span> Start of program headers: 64 (bytes into file) </span><span> Start of section headers: 256 (bytes into file) </span><span> Flags: 0x0 </span><span> Size of this header: 64 (bytes) </span><span> Size of program headers: 56 (bytes) </span><span> Number of program headers: 2 </span><span> Size of section headers: 64 (bytes) </span><span> Number of section headers: 4 </span><span> Section header string table index: 3 </span><span> </span><span>Section Headers: </span><span> [Nr] Name Type Address Offset </span><span> Size EntSize Flags Link Info Align </span><span> [ 0] NULL 0000000000000000 00000000 </span><span> 0000000000000000 0000000000000000 0 0 0 </span><span> [ 1] .text PROGBITS 00000000004000b0 000000b0 </span><span> 0000000000000027 0000000000000000 AX 0 0 16 </span><span> [ 2] .data PROGBITS 00000000006000d8 000000d8 </span><span> 000000000000000e 0000000000000000 WA 0 0 4 </span><span> [ 3] .shstrtab STRTAB 0000000000000000 000000e6 </span><span> 0000000000000017 0000000000000000 0 0 1 </span><span>Key to Flags: </span><span> W (write), A (alloc), X (execute), M (merge), S (strings), l (large) </span><span> I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown) </span><span> O (extra OS processing required) o (OS specific), p (processor specific) </span><span> </span><span>There are no section groups in this file. </span><span> </span><span>Program Headers: </span><span> Type Offset VirtAddr PhysAddr </span><span> FileSiz MemSiz Flags Align </span><span> LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000 </span><span> 0x00000000000000d7 0x00000000000000d7 R E 200000 </span><span> LOAD 0x00000000000000d8 0x00000000006000d8 0x00000000006000d8 </span><span> 0x000000000000000e 0x000000000000000e RW 200000 </span><span> </span><span> Section to Segment mapping: </span><span> Segment Sections... </span><span> 00 .text </span><span> 01 .data </span><span> </span><span>There is no dynamic section in this file. </span><span> </span><span>There are no relocations in this file. </span><span> </span><span>The decoding of unwind sections for machine type Advanced Micro Devices X86-64 is not currently supported. </span><span> </span><span>No version information found in this file. </span></code></pre> <p>Looking first at the section table, there are 4 section entries. The sections headers have the following structure:</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">typedef struct </span><span>{ </span><span> uint32_t sh_name; </span><span> uint32_t sh_type; </span><span> uint32_t sh_flags; </span><span> Elf32_Addr sh_addr; </span><span> Elf32_Off sh_offset; </span><span> uint32_t sh_size; </span><span> uint32_t sh_link; </span><span> uint32_t sh_info; </span><span> uint32_t sh_addralign; </span><span> uint32_t sh_entsize; </span><span>} Elf32_Shdr; </span></code></pre> <p>Precise detail on this structure, and what the fields are for can be found pretty easily in the <a href="https://linux.die.net/man/5/elf">Linux man page for elf</a>. Going on the output of <code>readelf</code>, the first section is is a <code>NULL</code> section. I don’t quite understand <a href="https://stackoverflow.com/questions/26812142/what-is-the-use-of-the-sht-null-section-in-elf">why this first section is empty</a>, but it is part of the ELF standard.</p> <p>The next two sections are <code>.text</code> and <code>.data</code>, which were defined in the assembly code. These are both of type <code>PROGBITS</code> which means all the data in the section is defined and formatted for the programs own usage, it is a chunk of arbitrary bytes as far as ELF is concerned.</p> <p>The <code>address</code> listed is where the section should be loaded in the program’s virtual memory space when the program is loaded by the operating system, if that section should be loaded into memory space of the process. The <code>offset</code> indicates how many bytes into the ELF file is the start of this section and <code>size</code> is the length of that section. So we can see here that <code>.text</code> will be loaded at <code>0x4000b0</code> for the process, which is <code>0xb0</code> bytes into the ELF binary file on the filesystem and that chunk of data is <code>0x27</code> bytes long.</p> <p>The <code>flags</code> tells us a few other bits about the section, both <code>.text</code> and <code>.data</code> have the allocate flag, so it should reside in the programs memory space when the program is executed. The <code>.text</code> section is the only section marked as executable, so the system should only allow code execution to occur within that section. The <code>.data</code> section is the only section with the writable flag, so the memory in that range can be modified by the running program.</p> <p>The program headers are pretty similar to the section headers, they describe information the system needs to run the program. They describe program segments and generally point to the same regions of memory and the binary file as the sections do, but with some detail that’s specific to executable and shared objects. I’ll defer again to <a href="https://linux.die.net/man/5/elf">the manual</a> for all the raw details.</p> <p>That covers my introduction to the ELF format. Hopefully this provides good enough context to explore a bit more on your own. I suggest trying to see if you can build an ELF file from scratch or modify one that is already built for some binary hacking shenanigans!</p> 2018-02-07T00:00:00+00:00 2018-02-07T00:00:00+00:00 https://suchprogramming.com/barebones-linux-iso/ <p>My post on <a href="https://suchprogramming.com/barebones-linux-system/">Building a Barebones Linux System</a> has been the most popular post on my blog so far. Back when my blog had comments, a reader left a comment asking how the system described in that post could be loaded onto an ISO file along with GRUB to make it into a bootable image. I think that’s a pretty good follow-up so today I will share the process on how to do just that!</p> <h1 id="continuing-the-barebones-journey">Continuing the Barebones Journey</h1> <p>It’s been a bit since I posted the original Barebones Linux post and in <a href="https://suchprogramming.com/getting-busybox-with-it/">Getting Busybox With It</a> I added BusyBox to the system and structured my build into a <code>Makefile</code> to preserve that effort (<a href="https://github.com/KennethWilke/my-barebones-linux">available on my Github</a>). That is what I’ll use as the starting point for this post.</p> <p>Back then I used the Linux Kernel version <code>4.13.3</code> and BusyBox version <code>1.27.2</code>, while I’m pulling this up again I’ll change a few of my <code>Makefile</code> variables to use the latest versions for today, Linux <code>4.15</code> and BusyBox <code>1.28.0</code>. After updating my versions I let the <code>Makefile</code> run and realized I needed to update my BusyBox config. I opted for the changes that the BusyBox build system suggested and gave it another test in <a href="https://www.qemu.org/">QEMU</a> to make sure it was all good to go.</p> <p>Now to build an ISO for it! I looked to see what resources I could find on ow to get that done. I saw what <a href="https://wiki.osdev.org/GRUB">osdev.org</a> had to offer and found a lot of good info on <a href="http://lukeluo.blogspot.com/2013/06/grub-how-to-2-make-boot-able-iso-with.html">how grub-mkresue works here</a>. I took a peek at the <a href="https://www.gnu.org/software/grub/manual/grub/grub.html#Making-a-GRUB-bootable-CD_002dROM">grub2 manual</a> and it looks easier than I expected it would be.</p> <h1 id="creating-a-grub-iso">Creating a GRUB ISO</h1> <p>As an initial test I wanted to verify that a normal grub2 rescue image would book in QEMU. I used <code>grub-mkrescue -o test.iso</code> to build this image.</p> <p><img src="/images/barebones-linux-iso/mkrescue.png" alt="" /></p> <p>Then to test it I ran <code>qemu-system-x86_64 -m 2048 -cdrom test.iso -boot d</code></p> <p><img src="/images/barebones-linux-iso/qemu-grub.png" alt="" /></p> <p>Easy enough. There is no <code>grub.cfg</code> so it gives you this basic rescue shell, but I’m glad it works. The grub documentation and examples I found show that you can give <code>grub-mkrescue</code> a directory that it will include in the build ISO and I decided to give that a shot.</p> <p>I returned to my barebones repo directory and created a directory <code>iso</code> to store my files that I’d like to go in the ISO, and the <code>boot/grub</code> directory within that to store my grub config.</p> <p><img src="/images/barebones-linux-iso/mkdir-grub.png" alt="" /></p> <p>I copied my built <code>vmlinuz</code> and <code>initramfs</code> files to <code>iso/boot/</code> and wrote the smallest <code>grub.cfg</code> to test it.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>menuentry &#39;barebones&#39; { </span><span> linux /boot/vmlinuz </span><span> initrd /boot/initramfs </span><span>} </span></code></pre> <p>With all the files where they should be I run <code>grub-mkrescue</code> again to make the build.</p> <p><img src="/images/barebones-linux-iso/mkiso-barebones.png" alt="" /></p> <p>Then I’ll give my <code>barebones.iso</code> a test in QEMU and there’s my menu entry:</p> <p><img src="/images/barebones-linux-iso/grub-barebones.png" alt="" /></p> <p>Selected my barebones OS and let it do it’s thing:</p> <p><img src="/images/barebones-linux-iso/hello-iso.png" alt="" /></p> <p>Huzzah! IT’S ALIVE!</p> <p>I thought there was going to be a bit more to this but I was pleasantly surprised!</p> 2018-01-31T00:00:00+00:00 2018-01-31T00:00:00+00:00 https://suchprogramming.com/c-strings-and-standard-input/ <p>Many C tutorials out there will show you some bad ways to do things. In this case I mean objectively bad, as security issues could arrise from naive approaches. Though a hacker wouldn't exactly mind this so maybe that is subjective. I’ll pick on <a href="https://www.tutorialspoint.com/cprogramming/c_input_output.htm">this input and output tutorial</a> as an example.</p> <p>It has what may appear as a pretty reasonable way to read input with the deprecated <code>gets()</code> function.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>( ) { </span><span> </span><span> </span><span style="color:#b48ead;">char</span><span> str[</span><span style="color:#d08770;">100</span><span>]; </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>( &quot;</span><span style="color:#a3be8c;">Enter a value :</span><span>&quot;); </span><span> </span><span style="color:#96b5b4;">gets</span><span>( str ); </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>( &quot;</span><span style="color:#96b5b4;">\n</span><span style="color:#a3be8c;">You entered: </span><span>&quot;); </span><span> </span><span style="color:#96b5b4;">puts</span><span>( str ); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>Or via <code>scanf()</code> like this</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>( ) { </span><span> </span><span> </span><span style="color:#b48ead;">char</span><span> str[</span><span style="color:#d08770;">100</span><span>]; </span><span> </span><span style="color:#b48ead;">int</span><span> i; </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>( &quot;</span><span style="color:#a3be8c;">Enter a value :</span><span>&quot;); </span><span> </span><span style="color:#96b5b4;">scanf</span><span>(&quot;</span><span style="color:#d08770;">%s %d</span><span>&quot;, str, &amp;i); </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>( &quot;</span><span style="color:#96b5b4;">\n</span><span style="color:#a3be8c;">You entered: </span><span style="color:#d08770;">%s %d </span><span>&quot;, str, i); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>In either case, the program wants you to enter a string. The string can only fit 100 ascii characters, though should really only have 99 so that your string can end with a <code>0</code> byte to be properly <code>NULL</code>-terminated. When I give it 120 <code>a</code>‘s, my system is reasonably displeased as I clobber over other parts of my stack.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./badhabits </span><span>Enter a value :aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 1 </span><span> </span><span>*** stack smashing detected ***: ./badhabits terminated </span><span>You entered: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 1 Aborted (core dumped) </span></code></pre> <p>One option would be to use a better format string. Referring to something like the <a href="https://www.gnu.org/software/libc/manual/html_node/String-Input-Conversions.html#String-Input-Conversions">GNU C Library Manual</a>, we can see that the <code>scanf()</code> function has a few other tricks up its sleeve that can help us.</p> <p>If we really wanted a 99 character limit on this string, we could change the format string to <code>&quot;%99s %d&quot;</code>.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./badhabits </span><span>Enter a value :aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa </span><span> </span><span>You entered: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 0 </span></code></pre> <p>In this case <code>scanf()</code> will truncate the string so that it fits into the size of the buffer. If the library is <a href="https://en.wikipedia.org/wiki/POSIX">POSIX</a> compliant, the m modifier can also be used to ask <code>scanf()</code> to dynamically allocate your string with <code>malloc()</code> and give you a pointer to that newly allocated memory space that now holds the input string nicely.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> </span><span style="color:#b48ead;">char </span><span>*name; </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Enter your name: </span><span>&quot;); </span><span> </span><span style="color:#96b5b4;">scanf</span><span>(&quot;</span><span style="color:#d08770;">%ms</span><span>&quot;, &amp;name); </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Hello </span><span style="color:#d08770;">%s</span><span style="color:#a3be8c;">!</span><span style="color:#96b5b4;">\n</span><span>&quot;, name); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./betterscanf </span><span>Enter your name: test </span><span>Hello test! </span></code></pre> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="beyond-scanf">Beyond Scanf</h1> <p>Personally, I’m not a big fan of <code>scanf()</code> in general. When hunting for other options, I first will peruse the <a href="https://www.gnu.org/software/libc/manual/html_node/index.html">GNU C Library’s manual</a>. In <a href="https://www.gnu.org/software/libc/manual/html_node/Line-Input.html">section 12.9</a> I find an approach that fits a common need I have, reading one line at a time with <a href="https://linux.die.net/man/3/getline">getline()</a>.</p> <p>The <code>getline()</code> function offers a few things that I like. It expects to work with dynamically allocated buffers and will allocate or reallocate them to size for you.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span>ssize_t </span><span style="color:#8fa1b3;">getline</span><span>(</span><span style="color:#b48ead;">char </span><span>**</span><span style="color:#bf616a;">lineptr</span><span>, size_t *</span><span style="color:#bf616a;">n</span><span>, FILE *</span><span style="color:#bf616a;">stream</span><span>); </span></code></pre> <p>Using it is pretty simple, it takes a pointer to a <code>char</code> pointer (<code>lineptr</code>) along with a pointer to a <code>size_t</code> type number (<code>n</code>). It’ll read a line from the <code>stream</code> file descriptor and return a <code>ssize_t</code> (signed size) value of the number of bytes read or <code>-1</code> on failure.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#define </span><span>_GNU_SOURCE </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> </span><span style="color:#b48ead;">char </span><span>*string = </span><span style="color:#d08770;">NULL</span><span>; </span><span> size_t buffer_size = </span><span style="color:#d08770;">0</span><span>; </span><span> ssize_t read_size; </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Enter some stuff!</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> read_size = </span><span style="color:#bf616a;">getline</span><span>(&amp;string, &amp;buffer_size, stdin); </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Read </span><span style="color:#d08770;">%zd</span><span style="color:#a3be8c;"> bytes, buffer is </span><span style="color:#d08770;">%zd</span><span style="color:#a3be8c;"> bytes</span><span style="color:#96b5b4;">\n</span><span>&quot;, read_size, buffer_size); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Line read:</span><span style="color:#96b5b4;">\n</span><span style="color:#d08770;">%s</span><span>&quot;, string); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>You need to make sure to set the string to <code>NULL</code> if it’s not already dynamically allocated or you’ll be passing whatever just happened to be laying around in the stack.</p> <p>Also worth noting, for the environment I’m building this within I needed to place the processor directive <code>#define _GNU_SOURCE</code> prior to including <code>stdio.h</code> to properly pull in the <code>getline()</code> functionality without angering the compiler.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./getline </span><span>Enter some stuff! </span><span>weeeeeeeeeeee </span><span>Read 14 bytes, buffer is 120 bytes </span><span>Line read: </span><span>weeeeeeeeeeee </span></code></pre> <p>In my run here, the <code>buffer_size</code> is getting set to a larger size than the string, whose length I got back from the <code>getline()</code> call. There is some consideration here on the part of the library that it is more efficient to give a longer buffer since it is likely to be added to later on and resizing buffers can be slow.</p> <h1 id="fancier-getline">Fancier Getline</h1> <p>I like the idea of making a function that abstracts this a bit so it’s a bit friendlier to use. I can reuse portions of the <code>FancyString</code> type I built in a <a href="https://suchprogramming.com/c-structs-and-org/">previous post</a> to build some functions that will let me dynamically read a line in a single step.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#define </span><span>_GNU_SOURCE </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdlib.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">typedef struct </span><span>{ </span><span> ssize_t length; </span><span> </span><span style="color:#b48ead;">char </span><span>*string; </span><span> size_t buffer_size; </span><span>} FancyString; </span><span> </span><span> </span><span style="color:#b48ead;">void </span><span style="color:#8fa1b3;">FancyString_free</span><span>(FancyString *</span><span style="color:#bf616a;">target</span><span>) { </span><span> </span><span style="color:#b48ead;">if </span><span>(target-&gt;string) { </span><span> </span><span style="color:#96b5b4;">free</span><span>(target-&gt;string); </span><span> } </span><span> </span><span style="color:#96b5b4;">free</span><span>(target); </span><span>} </span><span> </span><span> </span><span>FancyString* </span><span style="color:#8fa1b3;">fancy_getline</span><span>(FILE *</span><span style="color:#bf616a;">stream</span><span>) { </span><span> FancyString *new = </span><span style="color:#96b5b4;">malloc</span><span>(sizeof(*new)); </span><span> new-&gt;string = </span><span style="color:#d08770;">NULL</span><span>; </span><span> new-&gt;buffer_size = </span><span style="color:#d08770;">0</span><span>; </span><span> </span><span> new-&gt;length = </span><span style="color:#bf616a;">getline</span><span>(&amp;(new-&gt;string), &amp;(new-&gt;buffer_size), stream); </span><span> </span><span style="color:#b48ead;">if </span><span>(new-&gt;length == -</span><span style="color:#d08770;">1</span><span>) { </span><span> </span><span style="color:#96b5b4;">free</span><span>(new); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">NULL</span><span>; </span><span> } </span><span style="color:#b48ead;">else </span><span>{ </span><span> </span><span style="color:#b48ead;">return</span><span> new; </span><span> } </span><span>} </span><span> </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> FancyString *line = </span><span style="color:#bf616a;">fancy_getline</span><span>(stdin); </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Read </span><span style="color:#d08770;">%zd</span><span style="color:#a3be8c;"> bytes, buffer is </span><span style="color:#d08770;">%zd</span><span style="color:#a3be8c;"> bytes</span><span style="color:#96b5b4;">\n</span><span>&quot;, </span><span> line-&gt;length, </span><span> line-&gt;buffer_size); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Line read:</span><span style="color:#96b5b4;">\n</span><span style="color:#d08770;">%s</span><span>&quot;, line-&gt;string); </span><span> </span><span> </span><span style="color:#bf616a;">FancyString_free</span><span>(line); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./fancy_getline </span><span>this is a test of the fanciness </span><span>Read 32 bytes, buffer is 120 bytes </span><span>Line read: </span><span>this is a test of the fanciness </span></code></pre> <p>I find this a bit more convenient to manage the lines of input, this could even be extended to include a function that would operate like <a href="https://docs.python.org/3/library/io.html#io.IOBase.readlines">readlines() in Python</a>. I’ll modify my <code>FancyString</code> to support usage as a linked list, I’ll share more about linked lists and other data structure patterns in a future post.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#define </span><span>_GNU_SOURCE </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdlib.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">typedef struct</span><span> _fancystring { </span><span> ssize_t length; </span><span> </span><span style="color:#b48ead;">char </span><span>*string; </span><span> size_t buffer_size; </span><span> </span><span style="color:#b48ead;">struct</span><span> _fancystring *next; </span><span>} FancyString; </span><span> </span><span> </span><span style="color:#b48ead;">void </span><span style="color:#8fa1b3;">FancyString_free</span><span>(FancyString *</span><span style="color:#bf616a;">target</span><span>) { </span><span> </span><span style="color:#b48ead;">if </span><span>(target-&gt;string) { </span><span> </span><span style="color:#96b5b4;">free</span><span>(target-&gt;string); </span><span> } </span><span> </span><span style="color:#96b5b4;">free</span><span>(target); </span><span>} </span><span> </span><span> </span><span>FancyString* </span><span style="color:#8fa1b3;">fancy_getline</span><span>(FILE *</span><span style="color:#bf616a;">stream</span><span>) { </span><span> FancyString *new = </span><span style="color:#96b5b4;">malloc</span><span>(sizeof(*new)); </span><span> new-&gt;string = </span><span style="color:#d08770;">NULL</span><span>; </span><span> new-&gt;buffer_size = </span><span style="color:#d08770;">0</span><span>; </span><span> new-&gt;next = </span><span style="color:#d08770;">NULL</span><span>; </span><span> </span><span> new-&gt;length = </span><span style="color:#bf616a;">getline</span><span>(&amp;(new-&gt;string), &amp;(new-&gt;buffer_size), stream); </span><span> </span><span style="color:#b48ead;">if </span><span>(new-&gt;length == -</span><span style="color:#d08770;">1</span><span>) { </span><span> </span><span style="color:#96b5b4;">free</span><span>(new); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">NULL</span><span>; </span><span> } </span><span style="color:#b48ead;">else </span><span>{ </span><span> </span><span style="color:#b48ead;">return</span><span> new; </span><span> } </span><span>} </span><span> </span><span>FancyString* </span><span style="color:#8fa1b3;">fancy_readlines</span><span>(FILE *</span><span style="color:#bf616a;">stream</span><span>) { </span><span> FancyString *first = </span><span style="color:#d08770;">NULL</span><span>; </span><span> FancyString *last = </span><span style="color:#d08770;">NULL</span><span>; </span><span> FancyString *i = </span><span style="color:#d08770;">NULL</span><span>; </span><span> </span><span> </span><span style="color:#b48ead;">while </span><span>((i = </span><span style="color:#bf616a;">fancy_getline</span><span>(stream)) != </span><span style="color:#d08770;">NULL</span><span>) { </span><span> </span><span style="color:#b48ead;">if </span><span>(first == </span><span style="color:#d08770;">NULL</span><span>) { </span><span> first = i; </span><span> last = i; </span><span> } </span><span style="color:#b48ead;">else </span><span>{ </span><span> last-&gt;next = i; </span><span> last = i; </span><span> } </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">return</span><span> first; </span><span>} </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Enter many lines, end with CTRL+D</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span> FancyString *line = </span><span style="color:#bf616a;">fancy_readlines</span><span>(stdin); </span><span> FancyString *previous_line; </span><span> </span><span> </span><span style="color:#b48ead;">int</span><span> i = </span><span style="color:#d08770;">1</span><span>; </span><span> </span><span style="color:#b48ead;">while </span><span>(line != </span><span style="color:#d08770;">NULL</span><span>) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Line </span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;">: </span><span style="color:#d08770;">%s</span><span>&quot;, i, line-&gt;string); </span><span> i++; </span><span> previous_line = line; </span><span> line = line-&gt;next; </span><span> </span><span style="color:#bf616a;">FancyString_free</span><span>(previous_line); </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./readlines </span><span>Enter many lines, end with CTRL+D </span><span>this is a line </span><span>and this! </span><span>and moar </span><span>and moooooaoOOAOOAOARRRRR </span><span>Line 1: this is a line </span><span>Line 2: and this! </span><span>Line 3: and moar </span><span>Line 4: and moooooaoOOAOOAOARRRRR </span></code></pre> <p>That'll be it for this post, I hope you learned something or find some of this useful!</p> 2018-01-24T00:00:00+00:00 2018-01-24T00:00:00+00:00 https://suchprogramming.com/command-line-c/ <p>Today I’m going to share some tips and tricks to using command line arguments with C programs. First I’ll explore the plain ‘ol <code>argc</code>/<code>argv</code> style followed by a <a href="https://www.gnu.org/software/libc/manual/html_node/Getopt.html">getopt</a> approach.</p> <p>Let’s jump right to it!</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="classic-approach">Classic Approach</h1> <p>Most C programmers will be quite familiar with this approach, so I’ll keep it brief. The most common function signature for <code>main</code> is <code>int main(int argc, char *argv[])</code>. In this setup the <code>argc</code> will tell you the number of arguments passed to the program at launch, with <code>argc</code> getting a list of string pointers to those arguments.</p> <p>With a small test program, we can inspect these variables pretty easily. I’ll even look at where the <code>argv</code> pointer is and where the string pointers within it are pointing to, because why not!</p> <p>My <code>classic.c</code> source:</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main </span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> </span><span style="color:#b48ead;">int</span><span> i; </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Argument Count: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span>&quot;, argc); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">argv is at </span><span style="color:#d08770;">%p</span><span style="color:#96b5b4;">\n</span><span>&quot;, argv); </span><span> </span><span> </span><span style="color:#b48ead;">for </span><span>(i = </span><span style="color:#d08770;">0</span><span>; i &lt;= argc; i++) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;"> at </span><span style="color:#d08770;">%p</span><span style="color:#a3be8c;">: </span><span style="color:#d08770;">%s</span><span style="color:#96b5b4;">\n</span><span>&quot;, i, argv[i], argv[i]); </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>Testing it:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./classic well hello there </span><span>Argument Count: 4 </span><span>argv is at 0x7ffc923396b8 </span><span>0 at 0x7ffc92339f4d: ./classic </span><span>1 at 0x7ffc92339f57: well </span><span>2 at 0x7ffc92339f5c: hello </span><span>3 at 0x7ffc92339f62: there </span><span>4 at (nil): (null) </span></code></pre> <p>The first argument is the name of the program as it was executed, the remaining <code>argc - 1</code> are the command line arguments to that program. They are most often accessed using an index operator <code>[i]</code> as was done here, though usually <code>i &lt; argc;</code> is used in the <code>for</code> loop condition to skip the <code>null</code> element that terminates the array.</p> <p>Since this is a <code>null</code> terminated array, you could also use <a href="https://suchprogramming.com/fun-with-pointers/">pointer arithmetic</a> to process through the list and not use <code>argc</code> at all.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main </span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> </span><span> </span><span style="color:#b48ead;">char </span><span>**arg = argv; </span><span> </span><span style="color:#b48ead;">while </span><span>(*arg) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#d08770;">%s</span><span style="color:#96b5b4;">\n</span><span>&quot;, *arg); </span><span> arg++; </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>$ ./classic do it live ./classic do it live</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> </span><span>How you interpret the command line arguments from there is entirely up to you, which is why you&#39;ll see a lot of variance in command line applications. As with other programming languages, some patterns are very common and there are libraries to assist in implementing them. </span><span> </span><span># Getopt Basics </span><span> </span><span>The [getopt()](http://man7.org/linux/man-pages/man3/getopt.3.html) function is part of any Standard C Library implementation that follows a [POSIX](https://en.wikipedia.org/wiki/POSIX) standard. Many libraries will also follow the GNU standard for [getopt_long()](https://www.gnu.org/software/libc/manual/html_node/Getopt-Long-Options.html#Getopt-Long-Options). </span><span> </span><span>The general idea is that options use a format starting with `-` followed by a single letter to indicate something about what the user wants the program to do. As an example, many programs on Linux will have a `-v` option you can use to see more verbose console output, or a `-h` option to get help on using the program. </span><span> </span><span>The `getopt` implementation has a few variables that represent the current internal state of its parsing system. </span><span> </span><span>```c </span><span>extern int optind, opterr, optopt; </span></code></pre> <p>The <code>optind</code> variable is the index value of the next argument that should be handled by the <code>getopt()</code> function. <code>opterr</code> will let you control if the <code>getopt()</code> function should print errors to the console. If the <code>getopt()</code> call returns<code> ?</code> because it did not recognize the option being given, <code>optopt</code> will be set to the character it did not recognize.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">unistd.h</span><span>&gt; </span><span> </span><span> </span><span style="color:#b48ead;">void </span><span style="color:#8fa1b3;">print_getopt_state</span><span>(</span><span style="color:#b48ead;">void</span><span>) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">optind: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\t</span><span>&quot; &quot;</span><span style="color:#a3be8c;">opterr: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\t</span><span>&quot; &quot;</span><span style="color:#a3be8c;">optopt: </span><span style="color:#d08770;">%c</span><span style="color:#a3be8c;"> (</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;">)</span><span style="color:#96b5b4;">\n</span><span>&quot; , </span><span> optind, opterr, optopt, optopt </span><span> ); </span><span>} </span><span> </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main </span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> </span><span style="color:#bf616a;">print_getopt_state</span><span>(); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./getopt </span><span>optind: 1 opterr: 1 optopt: ? (63) </span></code></pre> <p>To use the <code>getopt()</code> function you provide the <code>argc</code> and <code>argv</code> variables along with an <code>optstring</code> variable that contains the list of options it should look for.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">unistd.h</span><span>&gt; </span><span> </span><span> </span><span style="color:#b48ead;">void </span><span style="color:#8fa1b3;">print_getopt_state</span><span>(</span><span style="color:#b48ead;">void</span><span>) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">optind: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\t</span><span>&quot; &quot;</span><span style="color:#a3be8c;">opterr: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\t</span><span>&quot; &quot;</span><span style="color:#a3be8c;">optopt: </span><span style="color:#d08770;">%c</span><span style="color:#a3be8c;"> (</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;">)</span><span style="color:#96b5b4;">\n</span><span>&quot; , </span><span> optind, opterr, optopt, optopt </span><span> ); </span><span>} </span><span> </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main </span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> </span><span style="color:#b48ead;">int</span><span> character; </span><span> </span><span style="color:#b48ead;">char </span><span>*options = &quot;</span><span style="color:#a3be8c;">v</span><span>&quot;; </span><span> </span><span> </span><span style="color:#bf616a;">print_getopt_state</span><span>(); </span><span> </span><span> character = </span><span style="color:#bf616a;">getopt</span><span>(argc, argv, options); </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">getopt returned: &#39;</span><span style="color:#d08770;">%c</span><span style="color:#a3be8c;">&#39; (</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;">)</span><span style="color:#96b5b4;">\n</span><span>&quot;, character, character); </span><span> </span><span style="color:#bf616a;">print_getopt_state</span><span>(); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./getopt </span><span>optind: 1 opterr: 1 optopt: ? (63) </span><span>getopt returned: &#39;�&#39; (-1) </span><span>optind: 1 opterr: 1 optopt: (0) </span><span> </span><span>$ ./getopt -v </span><span>optind: 1 opterr: 1 optopt: ? (63) </span><span>getopt returned: &#39;v&#39; (118) </span><span>optind: 2 opterr: 1 optopt: (0) </span><span> </span><span>$ ./getopt -h </span><span>optind: 1 opterr: 1 optopt: ? (63) </span><span>./getopt: invalid option -- &#39;h&#39; </span><span>getopt returned: &#39;?&#39; (63) </span><span>optind: 2 opterr: 1 optopt: h (104) </span></code></pre> <p>On each run of <code>getopt()</code>, until it reaches the end of the argument list and returns <code>-1</code>, it will check the next argument and return the option found or <code>?</code> if an unrecognized option was given.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">unistd.h</span><span>&gt; </span><span> </span><span> </span><span style="color:#b48ead;">void </span><span style="color:#8fa1b3;">print_getopt_state</span><span>(</span><span style="color:#b48ead;">void</span><span>) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">optind: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\t</span><span>&quot; &quot;</span><span style="color:#a3be8c;">opterr: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\t</span><span>&quot; &quot;</span><span style="color:#a3be8c;">optopt: </span><span style="color:#d08770;">%c</span><span style="color:#a3be8c;"> (</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;">)</span><span style="color:#96b5b4;">\n</span><span>&quot; , </span><span> optind, opterr, optopt, optopt </span><span> ); </span><span>} </span><span> </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main </span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> </span><span style="color:#b48ead;">int</span><span> character; </span><span> </span><span style="color:#b48ead;">char </span><span>*options = &quot;</span><span style="color:#a3be8c;">abcd</span><span>&quot;; </span><span> </span><span> </span><span style="color:#bf616a;">print_getopt_state</span><span>(); </span><span> </span><span> character = </span><span style="color:#bf616a;">getopt</span><span>(argc, argv, options); </span><span> </span><span> </span><span style="color:#b48ead;">while</span><span>(character != -</span><span style="color:#d08770;">1</span><span>) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">getopt returned: &#39;</span><span style="color:#d08770;">%c</span><span style="color:#a3be8c;">&#39; (</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;">)</span><span style="color:#96b5b4;">\n</span><span>&quot;, character, character); </span><span> </span><span style="color:#bf616a;">print_getopt_state</span><span>(); </span><span> </span><span> character = </span><span style="color:#bf616a;">getopt</span><span>(argc, argv, options); </span><span> } </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">getopt returned: &#39;</span><span style="color:#d08770;">%c</span><span style="color:#a3be8c;">&#39; (</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;">)</span><span style="color:#96b5b4;">\n</span><span>&quot;, character, character); </span><span> </span><span style="color:#bf616a;">print_getopt_state</span><span>(); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./getopt -d -a -b </span><span>optind: 1 opterr: 1 optopt: ? (63) </span><span>getopt returned: &#39;d&#39; (100) </span><span>optind: 2 opterr: 1 optopt: (0) </span><span>getopt returned: &#39;a&#39; (97) </span><span>optind: 3 opterr: 1 optopt: (0) </span><span>getopt returned: &#39;b&#39; (98) </span><span>optind: 4 opterr: 1 optopt: (0) </span><span>getopt returned: &#39;�&#39; (-1) </span><span>optind: 4 opterr: 1 optopt: (0) </span></code></pre> <p>You can also include multiple options in a single argument by just not separating them. Multiple instances of the same option will be iterated on multiple times.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./getopt -baddd </span><span>optind: 1 opterr: 1 optopt: ? (63) </span><span>getopt returned: &#39;b&#39; (98) </span><span>optind: 1 opterr: 1 optopt: (0) </span><span>getopt returned: &#39;a&#39; (97) </span><span>optind: 1 opterr: 1 optopt: (0) </span><span>getopt returned: &#39;d&#39; (100) </span><span>optind: 1 opterr: 1 optopt: (0) </span><span>getopt returned: &#39;d&#39; (100) </span><span>optind: 1 opterr: 1 optopt: (0) </span><span>getopt returned: &#39;d&#39; (100) </span><span>optind: 2 opterr: 1 optopt: (0) </span><span>getopt returned: &#39;�&#39; (-1) </span><span>optind: 2 opterr: 1 optopt: (0) </span></code></pre> <h1 id="optional-and-positional-arguments">Optional and Positional Arguments</h1> <p>A colon after an option in the <code>optstring</code> can be used to indicate that option requires an argument, while two colons can indicate that it supports an argument but is not required. In either case, if an argument is given to an option that supports it, <code>getopt()</code> will set the <code>optarg</code> pointer it provides to the argument.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">unistd.h</span><span>&gt; </span><span> </span><span> </span><span style="color:#b48ead;">void </span><span style="color:#8fa1b3;">print_getopt_state</span><span>(</span><span style="color:#b48ead;">void</span><span>) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">optind: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\t</span><span>&quot; &quot;</span><span style="color:#a3be8c;">opterr: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\t</span><span>&quot; &quot;</span><span style="color:#a3be8c;">optopt: </span><span style="color:#d08770;">%c</span><span style="color:#a3be8c;"> (</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;">)</span><span style="color:#96b5b4;">\t</span><span>&quot; &quot;</span><span style="color:#a3be8c;">optarg: </span><span style="color:#d08770;">%s</span><span style="color:#96b5b4;">\n</span><span>&quot; , </span><span> optind, opterr, optopt, optopt, optarg </span><span> ); </span><span>} </span><span> </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main </span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> </span><span style="color:#b48ead;">int</span><span> character; </span><span> </span><span style="color:#b48ead;">char </span><span>*options = &quot;</span><span style="color:#a3be8c;">abc:d::</span><span>&quot;; </span><span> </span><span> </span><span style="color:#bf616a;">print_getopt_state</span><span>(); </span><span> </span><span> character = </span><span style="color:#bf616a;">getopt</span><span>(argc, argv, options); </span><span> </span><span> </span><span style="color:#b48ead;">while</span><span>(character != -</span><span style="color:#d08770;">1</span><span>) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">getopt returned: &#39;</span><span style="color:#d08770;">%c</span><span style="color:#a3be8c;">&#39; (</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;">)</span><span style="color:#96b5b4;">\n</span><span>&quot;, character, character); </span><span> </span><span style="color:#bf616a;">print_getopt_state</span><span>(); </span><span> </span><span> character = </span><span style="color:#bf616a;">getopt</span><span>(argc, argv, options); </span><span> } </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">getopt returned: &#39;</span><span style="color:#d08770;">%c</span><span style="color:#a3be8c;">&#39; (</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;">)</span><span style="color:#96b5b4;">\n</span><span>&quot;, character, character); </span><span> </span><span style="color:#bf616a;">print_getopt_state</span><span>(); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./getopt -dwith -dwithout -c </span><span>optind: 1 opterr: 1 optopt: ? (63) optarg: (null) </span><span>getopt returned: &#39;d&#39; (100) </span><span>optind: 2 opterr: 1 optopt: (0) optarg: with </span><span>getopt returned: &#39;d&#39; (100) </span><span>optind: 3 opterr: 1 optopt: (0) optarg: without </span><span>./getopt: option requires an argument -- &#39;c&#39; </span><span>getopt returned: &#39;?&#39; (63) </span><span>optind: 4 opterr: 1 optopt: c (99) optarg: (null) </span><span>getopt returned: &#39;�&#39; (-1) </span><span>optind: 4 opterr: 1 optopt: c (99) optarg: (null) </span></code></pre> <p>If <code>optstring</code> begins with <code>-</code>, non-option positional arguments can also be handled. In these cases <code>getopt()</code> will return the value <code>1</code> to indicate it has found a positional argument and set the <code>optarg</code> pointer to it.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">unistd.h</span><span>&gt; </span><span> </span><span> </span><span style="color:#b48ead;">void </span><span style="color:#8fa1b3;">print_getopt_state</span><span>(</span><span style="color:#b48ead;">void</span><span>) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">optind: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\t</span><span>&quot; &quot;</span><span style="color:#a3be8c;">opterr: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\t</span><span>&quot; &quot;</span><span style="color:#a3be8c;">optopt: </span><span style="color:#d08770;">%c</span><span style="color:#a3be8c;"> (</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;">)</span><span style="color:#96b5b4;">\t</span><span>&quot; &quot;</span><span style="color:#a3be8c;">optarg: </span><span style="color:#d08770;">%s</span><span style="color:#96b5b4;">\n</span><span>&quot; , </span><span> optind, opterr, optopt, optopt, optarg </span><span> ); </span><span>} </span><span> </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main </span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> </span><span style="color:#b48ead;">int</span><span> character; </span><span> </span><span style="color:#b48ead;">char </span><span>*options = &quot;</span><span style="color:#a3be8c;">-abc:d::</span><span>&quot;; </span><span> </span><span> </span><span style="color:#bf616a;">print_getopt_state</span><span>(); </span><span> </span><span> character = </span><span style="color:#bf616a;">getopt</span><span>(argc, argv, options); </span><span> </span><span> </span><span style="color:#b48ead;">while</span><span>(character != -</span><span style="color:#d08770;">1</span><span>) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">getopt returned: &#39;</span><span style="color:#d08770;">%c</span><span style="color:#a3be8c;">&#39; (</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;">)</span><span style="color:#96b5b4;">\n</span><span>&quot;, character, character); </span><span> </span><span style="color:#bf616a;">print_getopt_state</span><span>(); </span><span> </span><span> character = </span><span style="color:#bf616a;">getopt</span><span>(argc, argv, options); </span><span> } </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">getopt returned: &#39;</span><span style="color:#d08770;">%c</span><span style="color:#a3be8c;">&#39; (</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;">)</span><span style="color:#96b5b4;">\n</span><span>&quot;, character, character); </span><span> </span><span style="color:#bf616a;">print_getopt_state</span><span>(); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./getopt well -a -b now </span><span>optind: 1 opterr: 1 optopt: ? (63) optarg: (null) </span><span>getopt returned: &#39;
&#39; (1) </span><span>optind: 2 opterr: 1 optopt: (0) optarg: well </span><span>getopt returned: &#39;a&#39; (97) </span><span>optind: 3 opterr: 1 optopt: (0) optarg: (null) </span><span>getopt returned: &#39;b&#39; (98) </span><span>optind: 4 opterr: 1 optopt: (0) optarg: (null) </span><span>getopt returned: &#39;
&#39; (1) </span><span>optind: 5 opterr: 1 optopt: (0) optarg: now </span><span>getopt returned: &#39;�&#39; (-1) </span><span>optind: 5 opterr: 1 optopt: (0) optarg: (null) </span></code></pre> <p>If <code>optstring</code> begins with a <code>+</code>, it will stop option parsing and return the value<code> -1</code> at the first non-option argument. The <code>-</code> and <code>+</code> prefixes are only provided by C libraries that follow the GNU extension’s</p> <h1 id="long-options">Long Options</h1> <p>The <code>GNU</code> extensions provide a fancier version of <code>getopt</code> that supports longer, more verbose options that begin with <code>--</code>. It has the same first 3 parameters as <code>getopt()</code>. The 4th parameter is an array, <code>longopts</code>, of <code>struct option</code> structures that describe the longer options. The last parameter is an integer pointer, <code>longindex</code>, that on match will be the index of the matched option from the <code>longopts</code> array. <code>longindex</code> may be set to <code>NULL</code> if you don’t plan to use it.</p> <p>The <code>struct option</code> has this format:</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">struct </span><span>option { </span><span> </span><span style="color:#b48ead;">const char </span><span>*name; </span><span> </span><span style="color:#b48ead;">int</span><span> has_arg; </span><span> </span><span style="color:#b48ead;">int </span><span>*flag; </span><span> </span><span style="color:#b48ead;">int</span><span> val; </span><span>}; </span></code></pre> <p>The <code>name</code> is the option name that is supported. The <code>has_arg</code> field can be set to <code>no_argument</code>, <code>required_argument</code> or <code>optional_argument</code>, which correlate to the values <code>0</code>, <code>1</code>, and <code>2</code>.</p> <p>The <code>flag</code> pointer, will be set to <code>val</code> on a match and <code>getopt_long()</code> will return <code>0</code>. If <code>flag</code> is <code>NULL</code>, <code>val</code> will be what <code>getopt_long()</code>returns when the long option is matched. Often a program will have a long option that returns the short option. This is common for an option like <code>--help</code>, to set <code>val</code> to <code>h</code> so that the code that handles <code>-h</code> can handle either.</p> <p>Here’s an example that uses a few different ways of handling the long options.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">unistd.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">getopt.h</span><span>&gt; </span><span> </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main </span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> </span><span style="color:#b48ead;">int</span><span> character; </span><span> </span><span style="color:#b48ead;">char </span><span>*options = &quot;</span><span style="color:#a3be8c;">h</span><span>&quot;; </span><span> </span><span style="color:#b48ead;">int</span><span> longindex; </span><span> </span><span style="color:#b48ead;">int</span><span> moartest_flag = </span><span style="color:#d08770;">0</span><span>; </span><span> </span><span> </span><span style="color:#b48ead;">struct</span><span> option longopts[] = { </span><span> {&quot;</span><span style="color:#a3be8c;">help</span><span>&quot;, no_argument, </span><span style="color:#d08770;">NULL</span><span>, &#39;</span><span style="color:#a3be8c;">h</span><span>&#39;}, </span><span> {&quot;</span><span style="color:#a3be8c;">echo</span><span>&quot;, required_argument, </span><span style="color:#d08770;">NULL</span><span>, </span><span style="color:#d08770;">0</span><span>}, </span><span> {&quot;</span><span style="color:#a3be8c;">longtest</span><span>&quot;, optional_argument, &amp;moartest_flag, </span><span style="color:#d08770;">12</span><span>}, </span><span> {</span><span style="color:#d08770;">NULL</span><span>, </span><span style="color:#d08770;">0</span><span>, </span><span style="color:#d08770;">NULL</span><span>, </span><span style="color:#d08770;">0</span><span>} </span><span> }; </span><span> </span><span> </span><span style="color:#b48ead;">while</span><span>((character = </span><span style="color:#bf616a;">getopt_long</span><span>(argc, argv, options, longopts, &amp;longindex)) != -</span><span style="color:#d08770;">1</span><span>) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">getopt_long returned: &#39;</span><span style="color:#d08770;">%c</span><span style="color:#a3be8c;">&#39; (</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;">)</span><span style="color:#96b5b4;">\n</span><span>&quot;, character, character); </span><span> </span><span style="color:#b48ead;">switch </span><span>(character) { </span><span> </span><span style="color:#b48ead;">case </span><span>&#39;</span><span style="color:#a3be8c;">h</span><span>&#39;: </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">help!</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span style="color:#b48ead;">break</span><span>; </span><span> </span><span style="color:#b48ead;">case </span><span style="color:#d08770;">0</span><span>: </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">longindex: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span>&quot;, longindex); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">longopts[longindex].name = </span><span style="color:#d08770;">%s</span><span style="color:#96b5b4;">\n</span><span>&quot;, longopts[longindex].</span><span style="color:#bf616a;">name</span><span>); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">optarg: </span><span style="color:#d08770;">%s</span><span style="color:#96b5b4;">\n</span><span>&quot;, optarg); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">moartest_flag: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span>&quot;, moartest_flag); </span><span> } </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./getopt --help --longtest --echo wee </span><span>getopt_long returned: &#39;h&#39; (104) </span><span>help! </span><span>getopt_long returned: &#39;&#39; (0) </span><span>longindex: 2 </span><span>longopts[longindex].name = longtest </span><span>optarg: (null) </span><span>moartest_flag: 12 </span><span>getopt_long returned: &#39;&#39; (0) </span><span>longindex: 1 </span><span>longopts[longindex].name = echo </span><span>optarg: wee </span><span>moartest_flag: 12 </span></code></pre> <p>In this code, <code>-h</code> and <code>--help</code> will match the <code>h</code> case and print my help message. Both <code>--longtest</code> and -<code>-echo</code> are handled by the <code>0</code> case, I can tell them apart from the value that <code>longindex</code> gets set to. When <code>--longtest</code> is given, the <code>moartest_flag</code> integer will get set to 12 since I provided the pointer to that integer in my <code>flags</code> field for that option.</p> <p>That covers what I wanted to share on parsing command line arguments! There are other strategies out there for handling it, but these are the most common and I hope this guide proves helpful. </p> 2018-01-17T00:00:00+00:00 2018-01-17T00:00:00+00:00 https://suchprogramming.com/c-structs-and-org/ <p>Many C tutorials very briefly gloss over structures and most don’t offer any guidance on how to organize your code and build it in a more modular fashion.</p> <p>In this post I will share the general system that I follow and prefer when it comes to organizing my C code. There are many other perfectly valid ways to organize your code; this is what works well for me and hopefully for you too!</p> <h1 id="code-organization">Code Organization</h1> <p>For this post, I’m going to define a new type, <code>FancyString</code>, along with a few functions that can provide some useful operations with it. I’ll have a <code>main.c</code> file to hold the application code I’m using to play with it, and <code>main.h</code> to include my headers for it. All of the declarations and data needed to use <code>FancyString</code> will be in <code>fancystring.h</code>, with the implementations of the functions declared in <code>fancystring.c</code>.</p> <p>As I normally do, I’ll start by building a <a href="/making-with-make/">Makefile</a>.</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#bf616a;">PROGRAM </span><span>= </span><span style="color:#a3be8c;">test </span><span style="color:#bf616a;">COMPILE_PROGRAM </span><span>= </span><span style="color:#a3be8c;">gcc -Wall -o </span><span style="color:#bf616a;">$@ $&lt; </span><span style="color:#bf616a;">COMPILE_OBJECT </span><span>= </span><span style="color:#a3be8c;">gcc -Wall -c -o </span><span style="color:#bf616a;">$@ $&lt; </span><span> </span><span> </span><span style="color:#8fa1b3;">all</span><span>: </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">PROGRAM</span><span style="color:#b48ead;">) </span><span> </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">PROGRAM</span><span style="color:#b48ead;">)</span><span>: </span><span style="color:#a3be8c;">main.c main.h fancystring.o </span><span> </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">COMPILE_PROGRAM</span><span style="color:#b48ead;">) </span><span style="color:#bf616a;">fancystring.o </span><span> </span><span style="color:#8fa1b3;">fancystring.o</span><span>: </span><span style="color:#a3be8c;">fancystring.c fancystring.h </span><span> </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">COMPILE_OBJECT</span><span style="color:#b48ead;">) </span><span> </span><span style="color:#8fa1b3;">clean</span><span>: </span><span> </span><span style="color:#d08770;">@</span><span style="color:#bf616a;">rm -vf </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">PROGRAM</span><span style="color:#b48ead;">)</span><span> fancystring.o </span></code></pre> <p>This time around, I’m using the <code>-c</code> flag for <code>gcc</code> to have it compile some the code to an object file <code>fancystring.o</code>, and not link it into a program or library. When I compile the actual program I’ll include this object file as a source, and it’ll end up in my <code>test</code> program. This will let me skip recompiling my <code>fancystring</code> code if I don’t make any changes to it.</p> <p>For <code>main.c</code>, I’ll start with a basic shell of a program.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&quot;</span><span style="color:#a3be8c;">main.h</span><span>&quot; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>For the accompanying header, <code>main.h</code>, it’ll look pretty bare. I’ll just include my <code>fancystring.h</code> header.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&quot;</span><span style="color:#a3be8c;">fancystring.h</span><span>&quot; </span></code></pre> <p>Now for <code>fancystring.c</code>, I don’t have anything quite yet. I’ll get to that file when I start implementing some code that uses my new structure. For <code>fancystring.h</code>, I’ll have a few standard library includes but eventually my structure will be defined here along with declarations of functions that will work with it.</p> <p>In larger projects you might have multiple files that reference this header, and if it’s included twice the compiler will lose it’s cool because you defined some things multiple times. To prevent this, I’ll wrap the whole header in a preprocessor <code>ifndef</code> (if not defined) / <code>endif</code> combination. When the first preprocessor line sees that my macro <code>__FANCYSTRING_H__</code> is not defined the next line of the will define that macro so later it can be checked against to make sure the contents of the header will only be handled one time, no matter how many bits of code are including it.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#ifndef</span><span> __FANCYSTRING_H__ </span><span style="color:#b48ead;">#define </span><span>__FANCYSTRING_H__ </span><span> </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdlib.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">string.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">#endif </span></code></pre> <p>That’ll give the general framework to build the rest of the code within. Anything I don’t want duplicated during compile will go between my <code>ifndef</code> and its <code>endif</code>.</p> <h1 id="building-the-fancystring-type">Building the FancyString Type</h1> <p>I’ll start off by declaring my structure for the <code>FancyString</code> type.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">struct </span><span>FancyString { </span><span> </span><span style="color:#b48ead;">unsigned int</span><span> length; </span><span> </span><span style="color:#b48ead;">char </span><span>*string; </span><span>}; </span></code></pre> <p>This structure has 2 fields: <code>length</code> contains the length of the string that the <code>string</code> pointer points to.</p> <p>When you define a structure like this, all your variables of this structure type would have to be defined as <code>struct FancyString variableName;</code>. To make it a bit shorter we can declare this structure as a data type instead with <code>typedef</code>.</p> <p>The general syntax of <code>typedef</code> is</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>typedef &lt;declaration or existing type&gt; &lt;new type name&gt;; </span></code></pre> <p>You could use this to create a type named <code>byte</code> that is a shortcut to an <code>unsigned char</code> with <code>typedef unsigned char byte;</code>. In this case I’ll declare my structure and type all in one go within my <code>fancystring.h</code> header.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#ifndef</span><span> __FANCYSTRING_H__ </span><span style="color:#b48ead;">#define </span><span>__FANCYSTRING_H__ </span><span> </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdlib.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">string.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">typedef struct </span><span>{ </span><span> </span><span style="color:#b48ead;">unsigned int</span><span> length; </span><span> </span><span style="color:#b48ead;">char </span><span>*string; </span><span>} FancyString; </span><span> </span><span style="color:#b48ead;">#endif </span></code></pre> <p>With my structure in place, I’ll whip up a few functions that will help me dynamically allocate an empty <code>FancyString</code> object and free it from memory. I’ll add the declarations of these functions to my <code>fancystring.h</code> header so that other files can be aware of it’s existence.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#ifndef</span><span> __FANCYSTRING_H__ </span><span style="color:#b48ead;">#define </span><span>__FANCYSTRING_H__ </span><span> </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdlib.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">string.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">typedef struct </span><span>{ </span><span> </span><span style="color:#b48ead;">unsigned int</span><span> length; </span><span> </span><span style="color:#b48ead;">char </span><span>*string; </span><span>} FancyString; </span><span> </span><span>FancyString* </span><span style="color:#8fa1b3;">FancyString_new</span><span>(); </span><span style="color:#b48ead;">void </span><span style="color:#8fa1b3;">FancyString_free</span><span>(FancyString *</span><span style="color:#bf616a;">target</span><span>); </span><span> </span><span style="color:#b48ead;">#endif </span></code></pre> <p>The actual implementation of these will be in <code>fancystring.c</code>.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&quot;</span><span style="color:#a3be8c;">fancystring.h</span><span>&quot; </span><span> </span><span>FancyString* </span><span style="color:#8fa1b3;">FancyString_new</span><span>() { </span><span> FancyString *new = </span><span style="color:#96b5b4;">malloc</span><span>(sizeof(*new)); </span><span> new-&gt;length = </span><span style="color:#d08770;">0</span><span>; </span><span> new-&gt;string = </span><span style="color:#d08770;">NULL</span><span>; </span><span> </span><span> </span><span style="color:#b48ead;">return</span><span> new; </span><span>} </span><span> </span><span> </span><span style="color:#b48ead;">void </span><span style="color:#8fa1b3;">FancyString_free</span><span>(FancyString *</span><span style="color:#bf616a;">target</span><span>) { </span><span> </span><span style="color:#b48ead;">if </span><span>(target-&gt;string) { </span><span> </span><span style="color:#96b5b4;">free</span><span>(target-&gt;string); </span><span> } </span><span> </span><span style="color:#96b5b4;">free</span><span>(target); </span><span>} </span></code></pre> <p>The first function here dynamically allocates memory for the structure with <a href="https://linux.die.net/man/3/malloc">malloc()</a>, and sets the other variables to useful defaults.</p> <p>Structure fields can be accessed in two ways. If you’re dealing with a direct instance of a structure, rather than a pointer to the structure, you use the <code>.</code> operator to access the field. For this structure I’m always going to use it as a pointer, so I use the <code>-&gt;</code> operator instead.</p> <p>I could still use <code>.</code> operator with a pointer if i really wanted to by first dereferencing the pointer with something like <code>(*new).length = 0;</code> but that is pretty wonky so I won’t be doing that.</p> <h1 id="building-around-the-structure">Building Around the Structure</h1> <p>This string is not yet fancy at all, but there’s a few things I’ve decided for it, firstly that it’s only used for dynamically allocated strings. Though it’ll still be pretty nice to define it with normal static strings in C. I’ll add another function that’ll let me create a new <code>FancyString</code> from an ordinary C string.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span>FancyString* </span><span style="color:#8fa1b3;">FancyString_new_from_string</span><span>(</span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">string</span><span>) { </span><span> FancyString *new = </span><span style="color:#bf616a;">FancyString_new</span><span>(); </span><span> </span><span> new-&gt;length = </span><span style="color:#96b5b4;">strlen</span><span>(string); </span><span> new-&gt;string = </span><span style="color:#96b5b4;">malloc</span><span>(new-&gt;length + </span><span style="color:#d08770;">1</span><span>); </span><span> </span><span style="color:#96b5b4;">strcpy</span><span>(new-&gt;string, string); </span><span> </span><span> </span><span style="color:#b48ead;">return</span><span> new; </span><span>} </span></code></pre> <p>Now I’ll give it a test in my <code>main</code> function.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&quot;</span><span style="color:#a3be8c;">main.h</span><span>&quot; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> FancyString *test = </span><span style="color:#bf616a;">FancyString_new_from_string</span><span>(&quot;</span><span style="color:#a3be8c;">Well hello!</span><span>&quot;); </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#d08770;">%s</span><span style="color:#96b5b4;">\n</span><span>&quot;, test-&gt;string); </span><span> </span><span> </span><span style="color:#bf616a;">FancyString_free</span><span>(test); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>It works like a charm! Though it’s still not particularly useful. As a final exercise, and to give it some purpose in life, I’ll create another function that let’s me concatenate other C strings to the end of it. Unlike normal <a href="https://linux.die.net/man/3/strcat">strcat()</a>, the FancyString concatenation will automatically resize my string so that it won’t risk a <a href="https://en.wikipedia.org/wiki/Buffer_overflow">buffer overflow</a>.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">void </span><span style="color:#8fa1b3;">FancyString_concat</span><span>(FancyString *</span><span style="color:#bf616a;">target</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">string</span><span>) { </span><span> </span><span style="color:#b48ead;">if</span><span>(!target-&gt;string) { </span><span> target-&gt;length = </span><span style="color:#96b5b4;">strlen</span><span>(string); </span><span> target-&gt;string = </span><span style="color:#96b5b4;">malloc</span><span>(target-&gt;length + </span><span style="color:#d08770;">1</span><span>); </span><span> </span><span style="color:#96b5b4;">strcpy</span><span>(target-&gt;string, string); </span><span> } </span><span style="color:#b48ead;">else </span><span>{ </span><span> target-&gt;length += </span><span style="color:#96b5b4;">strlen</span><span>(string); </span><span> target-&gt;string = </span><span style="color:#96b5b4;">realloc</span><span>(target-&gt;string, target-&gt;length + </span><span style="color:#d08770;">1</span><span>); </span><span> </span><span style="color:#96b5b4;">strcat</span><span>(target-&gt;string, string); </span><span> } </span><span>} </span></code></pre> <p>This new function will do the initial allocation if the <code>FancyString</code> was initially empty, otherwise it’ll use <a href="https://linux.die.net/man/3/realloc">realloc()</a> to expand the dynamically allocated buffer and then concatenate it all together.</p> <p>I’ll give this all one more test in <code>main</code>:</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&quot;</span><span style="color:#a3be8c;">main.h</span><span>&quot; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> FancyString *test = </span><span style="color:#bf616a;">FancyString_new</span><span>(); </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">new: </span><span style="color:#d08770;">%s</span><span style="color:#96b5b4;">\n</span><span>&quot;, test-&gt;string); </span><span> </span><span> </span><span style="color:#bf616a;">FancyString_concat</span><span>(test, &quot;</span><span style="color:#a3be8c;">hello</span><span>&quot;); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">first concat: </span><span style="color:#d08770;">%s</span><span style="color:#96b5b4;">\n</span><span>&quot;, test-&gt;string); </span><span> </span><span> </span><span style="color:#bf616a;">FancyString_concat</span><span>(test, &quot;</span><span style="color:#a3be8c;">, world</span><span>&quot;); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">second concat: </span><span style="color:#d08770;">%s</span><span style="color:#96b5b4;">\n</span><span>&quot;, test-&gt;string); </span><span> </span><span> </span><span style="color:#bf616a;">FancyString_free</span><span>(test); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ make </span><span>gcc -Wall -o test main.c fancystring.o </span><span>$ ./test </span><span>new: (null) </span><span>first concat: hello </span><span>second concat: hello, world </span></code></pre> <p>Works as expected! That’ll wrap things up for this post, I hope this might help with your journey with C!</p> 2018-01-10T00:00:00+00:00 2018-01-10T00:00:00+00:00 https://suchprogramming.com/fun-with-pointers/ <p>Pointers are one of the most misunderstood concepts in C but also one the most powerful tools it provides. In general they are just numbers in memory like everything else, but their value is interpreted as an address to other data. In this document I’ll attempt to demystify the arcane pointer and show some practical examples of their power.</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="everything-lives-in-memory">Everything Lives In Memory</h1> <p>Before I get into pointers, I want to clarify some aspects of how a program runs. This will all be based on my local Linux environment so your mileage may vary, but conceptually it’s all the same.</p> <p>The C compiler and linker you use takes your code, parses it, and converts it to a binary file that works for the targeted processor and operating system. The compiler will validate your C code and generate machine code that your processor can work with, the linker will do some organizational tasks to allow your program to use libraries and be loaded by the operating system.</p> <p>Most, if not all, UNIX systems use the Executable and Linkable Format (<a href="http://en.wikipedia.org/wiki/Executable_and_Linkable_Format">ELF</a>) for compiled programs and libraries. It has a header that describes what type of binary the file is, and has a lot of other data that tells the operating system where in <a href="http://web.archive.org/web/20160716011430/http://en.wikipedia.org/wiki/Virtual_memory">virtual memory</a> the program’s assets should be loaded.</p> <p>Here’s a simple program that we’ll inspect a little bit. It spits out the addresses of a few things, the locations in the running virtual memory space of a program. The <code>&amp;</code> operator before a variable will give you the address of that variable.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">int</span><span> my_global = </span><span style="color:#d08770;">123</span><span>; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">void</span><span>) </span><span>{ </span><span> </span><span style="color:#b48ead;">int</span><span> my_local = </span><span style="color:#d08770;">456</span><span>; </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">main has an address, it is </span><span style="color:#d08770;">%p</span><span style="color:#96b5b4;">\n</span><span>&quot;, main); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">my_global has an address, it is </span><span style="color:#d08770;">%p</span><span style="color:#96b5b4;">\n</span><span>&quot;, &amp;my_global); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">my_local has an address, it is </span><span style="color:#d08770;">%p</span><span style="color:#96b5b4;">\n</span><span>&quot;, &amp;my_local); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">printf has an address, it is </span><span style="color:#d08770;">%p</span><span style="color:#96b5b4;">\n</span><span>&quot;, printf); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>Tons of crap ends up in the binary for a C program, like all the stuff that provides you the goodness of the standard C library and the stuff the C library itself needs. One of the more interesting things we can look at is the symbol table. Symbols help give us names to find important things in the program. I’ll use <code>readelf</code> to dump the symbol table from the binary, and filter out globals.</p> <p><img src="/images/fun-with-pointers/symbol-table.png" alt="" /></p> <p>From this list there are a few things that were defined in my source file. Entries 4 and 53 both refer to the <code>printf</code> call I’m using from glibc which for this program is located at the memory address <code>0x400470</code>, entry 59 is the <code>my_global</code> variable from my program at the address <code>0x601040</code> and entry 64 is my <code>main</code> function at <code>0x400596</code>. The only thing missing is the <code>my_local</code> variable that’s inside of my <code>main</code> function.</p> <p>If I run the program a few times I’ll see where the program says these things are.</p> <p><img src="/images/fun-with-pointers/where-the-pointers-at-1.png" alt="" /></p> <p>As seen here, <code>main</code>, <code>my_global</code> and <code>printf</code> all have consistent locations in memory and it matches what was listed in the symbol table, while <code>my_local</code> changes between calls of the program. This is because the local variable is part of the function which is part of the <a href="https://en.wikipedia.org/wiki/Call_stack">call stack</a> and Linux uses <a href="https://en.wikipedia.org/wiki/Address_space_layout_randomization">a security feature that randomizes where the stack begins</a>. As functions are called in your C program the stack is used for function arguments, function local variables and a pointer to code that should be ran after the function is complete. Each function call gets a new section of the stack, referred to as a Stack Frame.</p> <p><img src="/images/fun-with-pointers/memory-address.jpg" alt="" /></p> <p>Just like a string and everything else on the computer, the executable code resides somewhere in memory. I can use <code>objdump</code> to get the machine code version of main, and it’ll show us each instruction that main is composed of and the addresses of each processor operation.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>0000000000400596 &lt;main&gt;: </span><span> 400596: 55 push rbp </span><span> 400597: 48 89 e5 mov rbp,rsp </span><span> 40059a: 48 83 ec 10 sub rsp,0x10 </span><span> 40059e: 64 48 8b 04 25 28 00 mov rax,QWORD PTR fs:0x28 </span><span> 4005a5: 00 00 </span><span> 4005a7: 48 89 45 f8 mov QWORD PTR [rbp-0x8],rax </span><span> 4005ab: 31 c0 xor eax,eax </span><span> 4005ad: c7 45 f4 c8 01 00 00 mov DWORD PTR [rbp-0xc],0x1c8 </span><span> 4005b4: be 96 05 40 00 mov esi,0x400596 </span><span> 4005b9: bf b8 06 40 00 mov edi,0x4006b8 </span><span> 4005be: b8 00 00 00 00 mov eax,0x0 </span><span> 4005c3: e8 a8 fe ff ff call 400470 &lt;printf@plt&gt; </span><span> 4005c8: be 40 10 60 00 mov esi,0x601040 </span><span> 4005cd: bf d8 06 40 00 mov edi,0x4006d8 </span><span> 4005d2: b8 00 00 00 00 mov eax,0x0 </span><span> 4005d7: e8 94 fe ff ff call 400470 &lt;printf@plt&gt; </span><span> 4005dc: 48 8d 45 f4 lea rax,[rbp-0xc] </span><span> 4005e0: 48 89 c6 mov rsi,rax </span><span> 4005e3: bf 00 07 40 00 mov edi,0x400700 </span><span> 4005e8: b8 00 00 00 00 mov eax,0x0 </span><span> 4005ed: e8 7e fe ff ff call 400470 &lt;printf@plt&gt; </span><span> 4005f2: be 70 04 40 00 mov esi,0x400470 </span><span> 4005f7: bf 28 07 40 00 mov edi,0x400728 </span><span> 4005fc: b8 00 00 00 00 mov eax,0x0 </span><span> 400601: e8 6a fe ff ff call 400470 &lt;printf@plt&gt; </span><span> 400606: b8 00 00 00 00 mov eax,0x0 </span><span> 40060b: 48 8b 55 f8 mov rdx,QWORD PTR [rbp-0x8] </span><span> 40060f: 64 48 33 14 25 28 00 xor rdx,QWORD PTR fs:0x28 </span><span> 400616: 00 00 </span><span> 400618: 74 05 je 40061f &lt;main+0x89&gt; </span><span> 40061a: e8 41 fe ff ff call 400460 &lt;__stack_chk_fail@plt&gt; </span><span> 40061f: c9 leave </span><span> 400620: c3 ret </span></code></pre> <p>Don’t be too daunted with the assembly here, we’re not here to talk about that. Just grasp that when this program is running <code>main</code> will be residing here in the memory space of the program.</p> <p>To the memory, there is no difference between program code and bytes that may be strings, integers or pictures of cats. To prove this, I’ll tell the compiler that I want to use <code>main</code> as a string, and I’ll look at the first character of it.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">void</span><span>) </span><span>{ </span><span> </span><span style="color:#b48ead;">char </span><span>*string = (</span><span style="color:#b48ead;">char</span><span>*) main; </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">character: </span><span style="color:#d08770;">%c</span><span style="color:#96b5b4;">\n</span><span>&quot;, string[</span><span style="color:#d08770;">0</span><span>]); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">hex: </span><span style="color:#d08770;">%x</span><span style="color:#96b5b4;">\n</span><span>&quot;, string[</span><span style="color:#d08770;">0</span><span>]); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span> </span><span>} </span></code></pre> <p>When I build and run this program, this is what I get:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./test </span><span>character: U </span><span>hex: 55 </span></code></pre> <p>Just like my previous program, <code>main</code> starts with the assembly operation <code>push rbp</code>, which if you look at as an ascii character would be a <code>U</code>, or <code>0x55</code> as a hexadecimal number.</p> <h3 id="why-you-confuse-me-so">Why You Confuse Me So?!</h3> <p>The reason I’m pushing this so hard is to really drive home the fact that everything lives in memory and that the only difference between data types is how you treat the data in memory. You can interpret any given chunk of memory as any type, in the case of pointers we treat a chunk of memory as an address that points to another chunk of memory.</p> <p>Let’s look at some less esoteric examples of how this can be useful.</p> <p>The Great Unknown</p> <p>For someone writing a C application, it would be fantastic if you knew how much memory a program would use before running it. Sadly this is almost never the case, so we need ways to get more memory when we need it. The call stack can suffice to create space for the variables we use inside of a function, but when the function ends we lose scope of those variables. To dynamically allocate space for keeps we use <a href="https://www.geeksforgeeks.org/memory-layout-of-c-program/">the heap</a>, most commonly via <a href="https://www.tutorialspoint.com/c_standard_library/c_function_malloc.htm">malloc()</a> (memory allocate).</p> <p>The malloc() function is provided by the C standard library, it asks the operating system to give your program a chunk of memory to work on. This is useful for making space as needed to store data in your program.</p> <p>Here’s an example, this program will take a number of bytes to allocate. It’ll request a chunk of memory of that size, print out it’s address, and then releases that chunk of memory back to the system (to avoid <a href="https://en.wikipedia.org/wiki/Memory_leak">memory leaks</a>!.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdlib.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) </span><span>{ </span><span> </span><span style="color:#b48ead;">void </span><span>*dynamic_space; </span><span> </span><span style="color:#b48ead;">int</span><span> size; </span><span> </span><span> </span><span style="color:#b48ead;">if</span><span>(argc != </span><span style="color:#d08770;">2</span><span>) </span><span> { </span><span> </span><span style="color:#96b5b4;">fprintf</span><span>(stderr, &quot;</span><span style="color:#a3be8c;">Usage: </span><span style="color:#d08770;">%s</span><span style="color:#a3be8c;"> &lt;number of bytes to allocate&gt;</span><span style="color:#96b5b4;">\n</span><span>&quot;, argv[</span><span style="color:#d08770;">0</span><span>]); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">1</span><span>; </span><span> } </span><span> </span><span> size = </span><span style="color:#96b5b4;">atoi</span><span>(argv[</span><span style="color:#d08770;">1</span><span>]); </span><span> </span><span style="color:#b48ead;">if</span><span>(size &lt; </span><span style="color:#d08770;">1</span><span>) </span><span> { </span><span> </span><span style="color:#96b5b4;">fprintf</span><span>(stderr, &quot;</span><span style="color:#a3be8c;">Number of bytes must be larger than 0</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">1</span><span>; </span><span> } </span><span> </span><span> dynamic_space = </span><span style="color:#96b5b4;">malloc</span><span>(size); </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">We allocated </span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;"> bytes, they are located at </span><span style="color:#d08770;">%p</span><span style="color:#96b5b4;">\n</span><span>&quot;, size, dynamic_space); </span><span> </span><span> </span><span style="color:#96b5b4;">free</span><span>(dynamic_space); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>I’ll give this program a few runs asking for different amounts of memory</p> <p><img src="/images/fun-with-pointers/much-allocation.png" alt="" /></p> <p>As you can see, the memory gets allocated in a variety of places even if you ask for the same amount of memory on another run of the program. On my machine, when I asked for a huge allocation it provided me a much higher memory address to use. I’m not sure why this happened but if the system thinks that’s where it should be who am I to argue.</p> <p>It’s easy enough to ask for space dynamically, the new problem introduced by this is how to be organize it all.</p> <h1 id="pointer-usage">Pointer Usage</h1> <p>There is no one perfect way to organize your pointers, but there are a ton of tried and true methods that are useful for a variety of cases.</p> <h3 id="null-terminated-arrays">Null terminated arrays</h3> <p>One of most common structures is a null terminated array, in the case of the <code>char</code> type we normally just call this a string. One of the handy things about these types of structures is they allow for <a href="https://www.tutorialspoint.com/cprogramming/c_pointer_arithmetic.htm">pointer arithmetic</a>. A null terminated array is a contiguous block of memory that contains no information about it’s size, but you can find the end of it by looking for the <code>NULL</code> or <code>0</code> (same thing in C) value at the end of it.</p> <p>In C the <code>++</code> and <code>--</code> operators work to increment and decrement values, but in the case of pointers they increment by the size of the data type the pointer works with, so it’s identical to incrementing a number with character pointers since characters use a single byte, but for a 4 byte integer it’s more like<code> +=4</code>. Using this functionality you can loop through a null terminated list until the value the pointer points to is zero pretty easily.</p> <p>Here’s an example of that:</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">char</span><span> my_string[] = &quot;</span><span style="color:#a3be8c;">well hello there</span><span>&quot;; </span><span style="color:#b48ead;">int</span><span> many_numbers[] = {</span><span style="color:#d08770;">1</span><span>, </span><span style="color:#d08770;">2</span><span>, </span><span style="color:#d08770;">3</span><span>, </span><span style="color:#d08770;">0</span><span>}; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> </span><span style="color:#b48ead;">char </span><span>*a_character; </span><span> </span><span style="color:#b48ead;">int </span><span>*an_integer; </span><span> </span><span> </span><span style="color:#65737e;">// Start where my_string points, printing each character until a_character </span><span> </span><span style="color:#65737e;">// points to a zero value </span><span> </span><span style="color:#b48ead;">for</span><span>(a_character = my_string; *a_character; a_character++) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">at </span><span style="color:#d08770;">%p</span><span style="color:#a3be8c;"> we have &#39;</span><span style="color:#d08770;">%c</span><span style="color:#a3be8c;">&#39;</span><span style="color:#96b5b4;">\n</span><span>&quot;, a_character, *a_character); </span><span> } </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span> </span><span style="color:#b48ead;">for </span><span>(an_integer = many_numbers; *an_integer; an_integer++) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">at </span><span style="color:#d08770;">%p</span><span style="color:#a3be8c;"> we have </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span>&quot;, an_integer, *an_integer); </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>And here’s that program in action</p> <p><img src="/images/fun-with-pointers/nullterms.png" alt="" /></p> <p>As you can see the address is incremented by 1 for character pointers and 4 for integer pointers. We can keep reading as long as the dereferenced value (the value in the address the pointer points to) is not zero. When it is zero we’re at the end of the array.</p> <p>To drive this home further, here’s a program that’ll determine the length of a string given as the first argument.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> </span><span style="color:#b48ead;">int</span><span> size = </span><span style="color:#d08770;">0</span><span>; </span><span> </span><span style="color:#b48ead;">char </span><span>*ptr; </span><span> </span><span> </span><span style="color:#b48ead;">if </span><span>(argc != </span><span style="color:#d08770;">2</span><span>) { </span><span> </span><span style="color:#96b5b4;">fprintf</span><span>(stderr, &quot;</span><span style="color:#a3be8c;">Usage: </span><span style="color:#d08770;">%s</span><span style="color:#a3be8c;"> &lt;string&gt;</span><span style="color:#96b5b4;">\n</span><span>&quot;, argv[</span><span style="color:#d08770;">0</span><span>]); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">1</span><span>; </span><span> } </span><span> </span><span> ptr = argv[</span><span style="color:#d08770;">1</span><span>]; </span><span> </span><span style="color:#b48ead;">while </span><span>(*ptr) { </span><span> size++; </span><span> ptr++; </span><span> } </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">first argument is </span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;"> bytes long</span><span style="color:#96b5b4;">\n</span><span>&quot;, size); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>And let’s see how that fares</p> <p><img src="/images/fun-with-pointers/strlen-the-program.png" alt="" /></p> <h3 id="passing-by-reference">Passing By Reference</h3> <p>In many languages, we hear about how things are passed by reference, as opposed to things that are passed by value. In many interpreted languages you have little to no control over how that works, but C lets (makes?) you control that completely.</p> <p>Let’s say I want to make a function that randomizes two numbers for me. In C you can’t return more than one value from a function, so setting two variables with one function call isn’t possible without pointers. I could return an array of two integers that have random values, then set my variables to those values, but that means my function doesn’t do everything I want it to on it’s own.</p> <p>The easiest way to do this is to pass the addresses (pointers) to the variables I want randomized, then my function can dereference them as if they were ordinary variables with <code>*</code>.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdlib.h</span><span>&gt; </span><span> </span><span> </span><span style="color:#b48ead;">void </span><span style="color:#8fa1b3;">randomize_them</span><span>(</span><span style="color:#b48ead;">int </span><span>*</span><span style="color:#bf616a;">a</span><span>, </span><span style="color:#b48ead;">int </span><span>*</span><span style="color:#bf616a;">b</span><span>) { </span><span> *a = </span><span style="color:#bf616a;">random</span><span>(); </span><span> *b = </span><span style="color:#bf616a;">random</span><span>(); </span><span>} </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">void</span><span>) { </span><span> </span><span style="color:#b48ead;">int</span><span> my_a = </span><span style="color:#d08770;">1</span><span>; </span><span> </span><span style="color:#b48ead;">int</span><span> my_b = </span><span style="color:#d08770;">2</span><span>; </span><span> </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">before</span><span style="color:#96b5b4;">\n</span><span style="color:#a3be8c;">a = </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span style="color:#a3be8c;">b = </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span>&quot;, my_a, my_b); </span><span> </span><span style="color:#bf616a;">randomize_them</span><span>(&amp;my_a, &amp;my_b); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">after</span><span style="color:#96b5b4;">\n</span><span style="color:#a3be8c;">a = </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span style="color:#a3be8c;">b = </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span>&quot;, my_a, my_b); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p><img src="/images/fun-with-pointers/ran-damn.png" alt="" /></p> <p>Now that wasn’t too bad, was it?</p> <p>That’ll wrap things up for this post, this covers most of the basics about pointers. I will be covering a bit more pointer-fu in the next post about defining and using structures in C, but I think this provides a good start for now.</p> 2018-01-05T00:00:00+00:00 2018-01-05T00:00:00+00:00 https://suchprogramming.com/epoll-in-3-easy-steps/ <p>It wasn't very long ago that it was a feat of greatness to get a single webserver setup to <a href="http://www.kegel.com/c10k.html">support 10,000 concurrent connections</a>. There were many factors that made it possible to develop webservers, such as <a href="https://www.nginx.com/">nginx</a>, that could handle more connections with greater efficiency than their predecessors. One of the biggest factors was the advent of constant-time polling ( <a href="https://rob-bell.net/2009/06/a-beginners-guide-to-big-o-notation/">O(1)</a> ) mechanisms for monitoring file descriptors introduced into most operating systems.</p> <p>In the <a href="https://www.nostarch.com/">No Starch Press</a> book, <a href="https://nostarch.com/tlpi">The Linux Programming Interface</a>, section 63.4.5 provides a table of observations that describes the time it takes to check different quantities of file descriptors via some of the most common polling methods.</p> <p><img src="/images/epoll-in-3-easy-steps/poll-times.png" alt="Chart of polling times" /></p> <p>As this shows, the performance benefits of epoll are decent enough to have an impact on even as few as 10 descriptors. As the number of descriptors increases, using regular <a href="http://man7.org/linux/man-pages/man2/poll.2.html">poll()</a> or <a href="http://man7.org/linux/man-pages/man2/select.2.html">select()</a> becomes a very unattractive option compared to epoll().</p> <p>This tutorial will run through some of the basics of using epoll() on Linux 2.6.27+.</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h2 id="prerequisite-knowledge">Prerequisite knowledge</h2> <p>This tutorial assumes you’re familiar and comfortable with Linux, the syntax of C and the usage of file descriptors in UNIX-like systems.</p> <h1 id="getting-started">Getting started</h1> <p>Make a new directory to work out of for this tutorial, here’s the Makefile we’re using.</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#8fa1b3;">all</span><span>: </span><span style="color:#a3be8c;">epoll_example </span><span> </span><span style="color:#8fa1b3;">epoll_example</span><span>: </span><span style="color:#a3be8c;">epoll_example.c </span><span> </span><span style="color:#bf616a;">gcc -Wall -Werror -o $@</span><span> epoll_example.c </span><span> </span><span style="color:#8fa1b3;">clean</span><span>: </span><span> </span><span style="color:#d08770;">@</span><span style="color:#bf616a;">rm -v</span><span> epoll_example </span></code></pre> <p>Throughout this post I’ll be using functionality described by the following headers:</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#65737e;">// for fprintf() </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">unistd.h</span><span>&gt; </span><span style="color:#65737e;">// for close(), read() </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">sys/epoll.h</span><span>&gt; </span><span style="color:#65737e;">// for epoll_create1(), epoll_ctl(), struct epoll_event </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">string.h</span><span>&gt; </span><span style="color:#65737e;">// for strncmp </span></code></pre> <h1 id="step-1-create-epoll-file-descriptor">Step 1: Create epoll file descriptor</h1> <p>First I’ll go through the process of just creating and closing an epoll instance.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#65737e;">// for fprintf() </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">unistd.h</span><span>&gt; </span><span style="color:#65737e;">// for close() </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">sys/epoll.h</span><span>&gt; </span><span style="color:#65737e;">// for epoll_create1() </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>() </span><span>{ </span><span> </span><span style="color:#b48ead;">int</span><span> epoll_fd = </span><span style="color:#bf616a;">epoll_create1</span><span>(</span><span style="color:#d08770;">0</span><span>); </span><span> </span><span> </span><span style="color:#b48ead;">if </span><span>(epoll_fd == -</span><span style="color:#d08770;">1</span><span>) { </span><span> </span><span style="color:#96b5b4;">fprintf</span><span>(stderr, &quot;</span><span style="color:#a3be8c;">Failed to create epoll file descriptor</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">1</span><span>; </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">if </span><span>(</span><span style="color:#bf616a;">close</span><span>(epoll_fd)) { </span><span> </span><span style="color:#96b5b4;">fprintf</span><span>(stderr, &quot;</span><span style="color:#a3be8c;">Failed to close epoll file descriptor</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">1</span><span>; </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>Running this should work and display no output, if you do get errors you’re either probably running a very old Linux kernel or your system needs real help.</p> <p>This first example uses <a href="https://linux.die.net/man/2/epoll_create1">epoll_create1()</a> to create a file descriptor to a new epoll instance given to us by the mighty kernel. While it doesn’t do anything with it quite yet we should still make sure to clean it up before the program terminates. Since it’s like any other Linux file descriptor we can just use <a href="https://linux.die.net/man/2/close">close()</a> for this.</p> <h5 id="level-triggered-and-edge-triggered-event-notifications">Level triggered and edge triggered event notifications</h5> <p><a href="https://www.quora.com/What-are-the-key-differences-between-edge-triggered-and-level-triggered-interrupts">Level-triggered and edge-triggered</a> are terms borrowed from electrical engineering. When we’re using epoll the difference is important. In edge triggered mode we will only receive events when the state of the watched file descriptors change; whereas in level triggered mode we will continue to receive events until the underlying file descriptor is no longer in a ready state. Generally speaking level triggered is the default and is easier to use and is what I’ll use for this tutorial, though it’s good to know edge triggered mode is available.</p> <h1 id="step-2-add-file-descriptors-for-epoll-to-watch">Step 2: Add file descriptors for epoll to watch</h1> <p>The next thing to do is tell epoll what file descriptors to watch and what kinds of events to watch for. In this example I’ll use one of my favorite file descriptors in Linux, good ol' file descriptor <code>0</code> (also known as Standard Input).</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#65737e;">// for fprintf() </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">unistd.h</span><span>&gt; </span><span style="color:#65737e;">// for close() </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">sys/epoll.h</span><span>&gt; </span><span style="color:#65737e;">// for epoll_create1(), epoll_ctl(), struct epoll_event </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>() </span><span>{ </span><span> </span><span style="color:#b48ead;">struct</span><span> epoll_event event; </span><span> </span><span style="color:#b48ead;">int</span><span> epoll_fd = </span><span style="color:#bf616a;">epoll_create1</span><span>(</span><span style="color:#d08770;">0</span><span>); </span><span> </span><span> </span><span style="color:#b48ead;">if </span><span>(epoll_fd == -</span><span style="color:#d08770;">1</span><span>) { </span><span> </span><span style="color:#96b5b4;">fprintf</span><span>(stderr, &quot;</span><span style="color:#a3be8c;">Failed to create epoll file descriptor</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">1</span><span>; </span><span> } </span><span> </span><span> event.</span><span style="color:#bf616a;">events </span><span>= EPOLLIN; </span><span> event.</span><span style="color:#bf616a;">data</span><span>.</span><span style="color:#bf616a;">fd </span><span>= </span><span style="color:#d08770;">0</span><span>; </span><span> </span><span> </span><span style="color:#b48ead;">if </span><span>(</span><span style="color:#bf616a;">epoll_ctl</span><span>(epoll_fd, EPOLL_CTL_ADD, </span><span style="color:#d08770;">0</span><span>, &amp;event)) { </span><span> </span><span style="color:#96b5b4;">fprintf</span><span>(stderr, &quot;</span><span style="color:#a3be8c;">Failed to add file descriptor to epoll</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span style="color:#bf616a;">close</span><span>(epoll_fd); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">1</span><span>; </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">if </span><span>(</span><span style="color:#bf616a;">close</span><span>(epoll_fd)) { </span><span> </span><span style="color:#96b5b4;">fprintf</span><span>(stderr, &quot;</span><span style="color:#a3be8c;">Failed to close epoll file descriptor</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">1</span><span>; </span><span> } </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span> </span><span>} </span></code></pre> <p>Here I’ve added an instance of an epoll_event structure and used <a href="https://linux.die.net/man/2/epoll_ctl">epoll_ctl()</a> to add the file descriptor <code>0</code> to our epoll instance <code>epoll_fd</code>. The event structure we pass in for the last argument lets epoll know we’re looking to watch only input events, <code>EPOLLIN</code>, and lets us provide some user-defined data that will be returned for events.</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="step-3-profit">Step 3: Profit</h1> <p>That’s right! We’re almost there. Now let epoll do it’s magic.</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#define </span><span>MAX_EVENTS </span><span style="color:#d08770;">5 </span><span style="color:#b48ead;">#define </span><span>READ_SIZE </span><span style="color:#d08770;">10 </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#65737e;">// for fprintf() </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">unistd.h</span><span>&gt; </span><span style="color:#65737e;">// for close(), read() </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">sys/epoll.h</span><span>&gt; </span><span style="color:#65737e;">// for epoll_create1(), epoll_ctl(), struct epoll_event </span><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">string.h</span><span>&gt; </span><span style="color:#65737e;">// for strncmp </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>() </span><span>{ </span><span> </span><span style="color:#b48ead;">int</span><span> running = </span><span style="color:#d08770;">1</span><span>, event_count, i; </span><span> size_t bytes_read; </span><span> </span><span style="color:#b48ead;">char</span><span> read_buffer[READ_SIZE + </span><span style="color:#d08770;">1</span><span>]; </span><span> </span><span style="color:#b48ead;">struct</span><span> epoll_event event, events[MAX_EVENTS]; </span><span> </span><span style="color:#b48ead;">int</span><span> epoll_fd = </span><span style="color:#bf616a;">epoll_create1</span><span>(</span><span style="color:#d08770;">0</span><span>); </span><span> </span><span> </span><span style="color:#b48ead;">if </span><span>(epoll_fd == -</span><span style="color:#d08770;">1</span><span>) { </span><span> </span><span style="color:#96b5b4;">fprintf</span><span>(stderr, &quot;</span><span style="color:#a3be8c;">Failed to create epoll file descriptor</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">1</span><span>; </span><span> } </span><span> </span><span> event.</span><span style="color:#bf616a;">events </span><span>= EPOLLIN; </span><span> event.</span><span style="color:#bf616a;">data</span><span>.</span><span style="color:#bf616a;">fd </span><span>= </span><span style="color:#d08770;">0</span><span>; </span><span> </span><span> </span><span style="color:#b48ead;">if</span><span>(</span><span style="color:#bf616a;">epoll_ctl</span><span>(epoll_fd, EPOLL_CTL_ADD, </span><span style="color:#d08770;">0</span><span>, &amp;event)) </span><span> { </span><span> </span><span style="color:#96b5b4;">fprintf</span><span>(stderr, &quot;</span><span style="color:#a3be8c;">Failed to add file descriptor to epoll</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span style="color:#bf616a;">close</span><span>(epoll_fd); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">1</span><span>; </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">while </span><span>(running) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#96b5b4;">\n</span><span style="color:#a3be8c;">Polling for input...</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> event_count = </span><span style="color:#bf616a;">epoll_wait</span><span>(epoll_fd, events, MAX_EVENTS, </span><span style="color:#d08770;">30000</span><span>); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;"> ready events</span><span style="color:#96b5b4;">\n</span><span>&quot;, event_count); </span><span> </span><span style="color:#b48ead;">for </span><span>(i = </span><span style="color:#d08770;">0</span><span>; i &lt; event_count; i++) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Reading file descriptor &#39;</span><span style="color:#d08770;">%d</span><span style="color:#a3be8c;">&#39; -- </span><span>&quot;, events[i].</span><span style="color:#bf616a;">data</span><span>.</span><span style="color:#bf616a;">fd</span><span>); </span><span> bytes_read = </span><span style="color:#bf616a;">read</span><span>(events[i].</span><span style="color:#bf616a;">data</span><span>.</span><span style="color:#bf616a;">fd</span><span>, read_buffer, READ_SIZE); </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#d08770;">%zd</span><span style="color:#a3be8c;"> bytes read.</span><span style="color:#96b5b4;">\n</span><span>&quot;, bytes_read); </span><span> read_buffer[bytes_read] = &#39;</span><span style="color:#96b5b4;">\0</span><span>&#39;; </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Read &#39;</span><span style="color:#d08770;">%s</span><span style="color:#a3be8c;">&#39;</span><span style="color:#96b5b4;">\n</span><span>&quot;, read_buffer); </span><span> </span><span> </span><span style="color:#b48ead;">if</span><span>(!</span><span style="color:#96b5b4;">strncmp</span><span>(read_buffer, &quot;</span><span style="color:#a3be8c;">stop</span><span style="color:#96b5b4;">\n</span><span>&quot;, </span><span style="color:#d08770;">5</span><span>)) </span><span> running = </span><span style="color:#d08770;">0</span><span>; </span><span> } </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">if </span><span>(</span><span style="color:#bf616a;">close</span><span>(epoll_fd)) { </span><span> </span><span style="color:#96b5b4;">fprintf</span><span>(stderr, &quot;</span><span style="color:#a3be8c;">Failed to close epoll file descriptor</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">1</span><span>; </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>Finally we’re getting down to business!</p> <p>I added a few new variables here to support and expose what I’m doing. I also added a while loop that’ll keep reading from the file descriptors being watched until one of them says ‘stop’. I used <a href="https://linux.die.net/man/2/epoll_wait">epoll_wait()</a> to wait for events to occur from the epoll instance, the results will be stored in the events array up to <code>MAX_EVENTS</code> with a timeout of 30 second. The return value of epoll_wait() indicates how many members of the events array were filled with event data. Beyond that it’s just printing out what it got and doing some basic logic to close things out!</p> <p>Here’s the example in action:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ ./epoll_example </span><span> </span><span>Polling for input.. </span></code></pre> <p><em>hello!</em></p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>1 ready events </span><span>Reading file descriptor &#39;0&#39; -- 7 bytes read. </span><span>Read &#39;hello! </span><span>&#39; </span><span> </span><span>Polling for input... </span></code></pre> <p><em>this is too long for the buffer we made</em></p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>1 ready events </span><span>Reading file descriptor &#39;0&#39; -- 10 bytes read. </span><span>Read &#39;this is to&#39; </span><span> </span><span>Polling for input... </span><span>1 ready events </span><span>Reading file descriptor &#39;0&#39; -- 10 bytes read. </span><span>Read &#39;o long for&#39; </span><span> </span><span>Polling for input... </span><span>1 ready events </span><span>Reading file descriptor &#39;0&#39; -- 10 bytes read. </span><span>Read &#39; the buffe&#39; </span><span> </span><span>Polling for input... </span><span>1 ready events </span><span>Reading file descriptor &#39;0&#39; -- 10 bytes read. </span><span>Read &#39;r we made </span><span>&#39; </span><span> </span><span>Polling for input... </span></code></pre> <p><em>stop</em></p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>1 ready events </span><span>Reading file descriptor &#39;0&#39; -- 5 bytes read. </span><span>Read &#39;stop </span><span>&#39; </span></code></pre> <p>First I gave it a small string that fits in the buffer and it works fine and continues iterating over the loop. The second input was too long for the read buffer, and is where level triggering helped us out; events continued to populate until it read all of what was left in the buffer, in edge triggering mode we would have only received 1 notification and the application as-is would not progress until more was written to the file descriptor being watching.</p> <p>I hope this helped you get some bearings on how to use epoll()!</p> 2017-12-20T00:00:00+00:00 2017-12-20T00:00:00+00:00 https://suchprogramming.com/beginning-logic-design-part-14/ 2017-12-15T00:00:00+00:00 2017-12-15T00:00:00+00:00 https://suchprogramming.com/cross-compiling-c-code-for-arm/ <p>Hello! In this brief post I will share how you can cross compile some simple C code for running on an ARM based device. I have a desire to write a few simple things in C for tinkering with <a href="https://github.com/openbmc/openbmc">OpenBMC</a> on <a href="https://blog.rackspace.com/zaius-barreleye-g2-server-development-update-2">Barreleye G2</a> and figured I would share my process.</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="the-build-box">The Build Box</h1> <p>To build this program I’m going to use a freshly built Ubuntu 16.04.3 VM, that way I know for sure what dependencies are needed. My host system is also running Ubuntu 16.04.3 and I’m using <a href="https://virt-manager.org/">Virt Manager</a> as an interface to <a href="https://libvirt.org/">libvirt</a> that is serving my VMs via <a href="https://www.qemu.org/">QEMU</a> and <a href="https://www.linux-kvm.org/page/Main_Page">KVM</a>.</p> <p><img src="/images/cross-compiling-c-code-for-arm/ubuntu-kickstart.gif" alt="Starting the install" /></p> <p>After a few minutes I am ready to go!</p> <h1 id="the-codes">The Codes</h1> <p>The code for this is pretty simple, I’ll just build the ol’ hello world!</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Hello, ARM!</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>I’ll SSH into my VM so that my commands are easier to follow. I start by creating the source file with vim.</p> <p><img src="/images/cross-compiling-c-code-for-arm/vimit.gif" alt="Writing hello.c in VIM" /></p> <h1 id="building-and-testing">Building and Testing</h1> <p>Ubuntu offers many pre-built cross compilers. All you need to do is run a <code>sudo apt update</code> to ensure your package listing is up to date and a <code>sudo apt install -y build-essential gcc-arm-linux-gnueabi</code> will get you what you need.</p> <p>To build, I’ll use the <code>arm-linux-gnueabi-gcc</code> compiler instead of just <code>gcc</code>. I’ll verify it was indeed for an ARM machine with <code>readelf</code>.</p> <p><img src="/images/cross-compiling-c-code-for-arm/buildit.gif" alt="Building and copying the binary" /></p> <p>Next, to test it, I’ll scp it over to my OpenBMC system, ssh into it and give it a go!</p> <p><img src="/images/cross-compiling-c-code-for-arm/testit.gif" alt="Building and copying the binary" /></p> <p>There it is! I’ve successfully cross compiled a simple program to run on an ARM machine.</p> 2017-12-13T00:00:00+00:00 2017-12-13T00:00:00+00:00 https://suchprogramming.com/beginning-logic-design-part-13/ 2017-12-06T00:00:00+00:00 2017-12-06T00:00:00+00:00 https://suchprogramming.com/beginning-logic-design-part-12/ 2017-12-05T00:00:00+00:00 2017-12-05T00:00:00+00:00 https://suchprogramming.com/making-with-make/ <p>I enjoy writing in a variety of low level and compiled languages. One of the tools I use almost every time regardless of the language, or sometimes mix of languages, is <code>make</code>. In this article, I want to share some of the ways that <code>make</code> can be used and some of the tips and tricks I employ the most when using <code>make</code>.</p> <p>I’ll be writing and running these examples on my Ubuntu 16.04.3 laptop with GNU Make 4.1.</p> <h1 id="make-without-makefiles">Make Without Makefiles</h1> <p>I almost always use <code>make</code> with a <code>Makefile</code>, as it’s significantly more customizable that way, but I think it’s important to know that <code>make</code> does have some usage you can employ even without a <code>Makefile</code>.</p> <p>Without a <code>Makefile</code>, <code>make</code> can still follow some of it’s <a href="https://ftp.gnu.org/old-gnu/Manuals/make-3.79.1/html_chapter/make_10.html#SEC93">implicit rules</a> for building some files. Let’s say we have the following <code>test.cpp</code> file ready to build:</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Hello from test.cpp!</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>You could build the <code>test</code> program with a simple call to <code>make test</code>.</p> <p><img src="/images/making-with-make/make-binary.png" alt="" /></p> <p>Or if you’d like to build a <code>test.o</code> object file, you can do <code>make test.o</code> and if that’s present when you do <code>make test</code> it’ll build the binary using the already built object file.</p> <p><img src="/images/making-with-make/make-object.png" alt="" /></p> <p>There is a pretty good smattering of things you can build this way, check <a href="https://ftp.gnu.org/old-gnu/Manuals/make-3.79.1/html_chapter/make_10.html#SEC93">the documentation</a> if you’re interested in more detail on the rules and supported targets.</p> <h1 id="looking-at-makefiles">Looking at Makefiles</h1> <p>Most projects will have a depth beyond what <code>make</code> is able to determine with all it’s smarty-pantsness, and in those cases the beloved <code>Makefile</code> is there to make life easy.</p> <p>If you’ve pulled down and built a common open source project and looked at the <code>Makefile</code>, or generated one with a tool like <a href="https://www.gnu.org/software/automake/manual/html_node/Autotools-Introduction.html">autotools</a> or <a href="https://cmake.org/">cmake</a>, you may have looked at it with much confusion.</p> <p>As an example I’ll look at <a href="https://github.com/libuv/libuv">libuv</a>, my favorite cross platform asynchronous I/O library. After cloning the repo down, running <code>./autogen.sh</code> to generate the build configuration script, then running the <code>./configure</code> script I get a nearly 5000 line <code>Makefile</code>. To me it looks like mostly gibberish and some tests. In all fairness there is a lot of good things happening in there but it’s not good for learning how to write a <code>Makefile</code>.</p> <p>A <code>Makefile</code> doesn’t always need to be that cray-cray. For my own projects I try to keep it pretty simple, though over time it generally becomes more complex. An example of mine, from my post on <a href="/barebones-linux-system/">building a barebones Linux system</a>, is on my github <a href="https://github.com/KennethWilke/my-barebones-linux/blob/master/Makefile">here</a>. I’m not the only person crazy enough to stick with a handwritten Makefile; the <a href="https://redis.io/">Redis</a> database also uses a <a href="https://github.com/redis/redis/blob/79866a6361829ed0602dedff9cb378c66977227a/src/Makefile">handwritten Makefile</a>, and Redis is production quality and awesome AF.</p> <p>Let’s start looking at the basics of making your own Makefile!</p> <h1 id="makefile-basics">Makefile Basics</h1> <p>Most of the time when you use <code>make</code>, it will be looking for a file named <code>Makefile</code> to find your targets. If you run <code>make</code> without a <code>Makefile</code>, you’ll be greeted with this lovely message:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>make: *** No targets specified and no makefile found. Stop. </span></code></pre> <p>If you had say, an empty <code>Makefile</code>, you’ll see something along the lines of:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>make: *** No targets. Stop. </span></code></pre> <p>The first thing you should be aware of regarding <code>Makefile</code> syntax is that <strong>tabs are part of the syntax!</strong> I’ve seen a few developers start building a Makefile and be like “WTF!” when nothing works because their text editor is configured to insert spaces when they hit tab.</p> <p><img src="/images/making-with-make/tabsvspaces.gif" alt="" /></p> <p>The general format of a Makefile is a a list of targets with optional dependencies and commands</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&lt;target&gt;: [dependency] [dependency] </span><span>&lt;tab&gt;[command] </span><span>&lt;tab&gt;[command] </span></code></pre> <p>As an example, I’ll define a target <code>test</code> that will not have dependencies and that target will run some echo commands.</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#8fa1b3;">test</span><span>: </span><span> </span><span style="color:#96b5b4;">echo</span><span> test! </span><span> </span><span style="color:#96b5b4;">echo</span><span> IT WERKS!!!! </span></code></pre> <p>With this in my <code>Makefile</code>, if I run <code>make</code> with no arguments it’ll run my first target <code>test</code>. The common convention is for a <code>Makefiles</code> to start with the target <code>all</code>.</p> <p><img src="/images/making-with-make/makeytest.png" alt="" /></p> <p>If I add a second target <code>moartest</code> and I want to run that one, I’ll need to specify it during my command as <code>make moartest</code>.</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#8fa1b3;">test</span><span>: </span><span> </span><span style="color:#96b5b4;">echo</span><span> test! </span><span> </span><span style="color:#96b5b4;">echo</span><span> IT WERKS!!!! </span><span> </span><span style="color:#8fa1b3;">moartest</span><span>: </span><span> </span><span style="color:#96b5b4;">echo</span><span> woah now, so fancy </span></code></pre> <p><img src="/images/making-with-make/moarmaketest.png" alt="" /></p> <h1 id="makefile-dependencies">Makefile Dependencies</h1> <p>One of my favorite things about <code>make</code> is the way it handles dependencies. If you’re using it for building a project you can organize the steps however you’d like and structure a hierarchy where one step runs before another.</p> <p>I’ll extend my previous example to add my <code>moartest</code> target as a dependency of the <code>test</code> target.</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#8fa1b3;">test</span><span>: </span><span style="color:#a3be8c;">moartest </span><span> </span><span style="color:#96b5b4;">echo</span><span> test! </span><span> </span><span style="color:#96b5b4;">echo</span><span> IT WERKS!!!! </span><span> </span><span style="color:#8fa1b3;">moartest</span><span>: </span><span> </span><span style="color:#96b5b4;">echo</span><span> woah now, so fancy </span></code></pre> <p>Now when I run <code>make</code>, <code>test</code> will be inspected since it is the first target and since <code>test</code> has <code>moartest</code> in its list of dependencies that make will first look for that target and if it’s commands execute successfully the commands for <code>test</code> will also be ran.</p> <p><img src="/images/making-with-make/evenmoarmake-1.png" alt="" /></p> <p>If for some reason the dependency commands should fail, <code>make</code> will error out at that point. To simulate this I will add an <code>exit 1</code> command to my <code>moartest</code> target.</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#8fa1b3;">test</span><span>: </span><span style="color:#a3be8c;">moartest </span><span> </span><span style="color:#96b5b4;">echo</span><span> test! </span><span> </span><span style="color:#96b5b4;">echo</span><span> IT WERKS!!!! </span><span> </span><span style="color:#8fa1b3;">moartest</span><span>: </span><span> </span><span style="color:#96b5b4;">echo</span><span> woah now, so fancy </span><span> </span><span style="color:#96b5b4;">exit</span><span> 1 </span></code></pre> <p><img src="/images/making-with-make/make-errors.png" alt="" /></p> <p>If the target name is a file, and that file already exists, the target will be skipped. Here’s an example where my randomcrap target generates a file that’s a dependency of my test target.</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#8fa1b3;">test</span><span>: </span><span style="color:#a3be8c;">randomcrap </span><span> </span><span style="color:#96b5b4;">echo</span><span> we have random! </span><span> </span><span style="color:#8fa1b3;">randomcrap</span><span>: </span><span> </span><span style="color:#bf616a;">dd</span><span> if=/dev/urandom of=randomcrap bs=1024 count=1 </span></code></pre> <p><img src="/images/making-with-make/makerandom.png" alt="" /></p> <p>A dependency doesn’t need to be a another target, in many cases it’s useful if a dependency is some source file. <code>make</code> will look to see if that source file has been updated and will re-run the target only when it seems necessary.</p> <p>Consider this example:</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#8fa1b3;">test</span><span>: </span><span style="color:#a3be8c;">copiedfile </span><span> </span><span style="color:#96b5b4;">echo</span><span> we have the latest copy! </span><span> </span><span style="color:#8fa1b3;">copiedfile</span><span>: </span><span style="color:#a3be8c;">originalfile </span><span> </span><span style="color:#bf616a;">cp</span><span> originalfile copiedfile </span></code></pre> <p>And observe how <code>make</code> responds to the absence of the source file, how it skips the file when it’s already the same as the original, and how updates to the original will be noticed during the subsequent run.</p> <p><img src="/images/making-with-make/dependency-update.gif" alt="" /></p> <h1 id="variables-in-makefiles">Variables in Makefiles</h1> <p>It is often useful to have some variables in your <code>Makefile</code>. Variables can be set with the <code>NAME=VALUE</code> syntax. In my first example here I’ll compile the following <code>hello.c</code> program:</p> <pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#include </span><span>&lt;</span><span style="color:#a3be8c;">stdio.h</span><span>&gt; </span><span> </span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">argc</span><span>, </span><span style="color:#b48ead;">char </span><span>*</span><span style="color:#bf616a;">argv</span><span>[]) { </span><span> </span><span style="color:#96b5b4;">printf</span><span>(&quot;</span><span style="color:#a3be8c;">Well hello you proverbial world you.</span><span style="color:#96b5b4;">\n</span><span>&quot;); </span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>; </span><span>} </span></code></pre> <p>To make my compilation of the program a bit more flexible I’ll make a <code>COMPILER</code> variable to setup what compiler I’d like it to use.</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#bf616a;">COMPILER </span><span>= </span><span style="color:#a3be8c;">gcc </span><span> </span><span style="color:#8fa1b3;">all</span><span>: </span><span style="color:#a3be8c;">hello-world </span><span> </span><span style="color:#8fa1b3;">hello-world</span><span>: </span><span style="color:#a3be8c;">hello.c </span><span> </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">COMPILER</span><span style="color:#b48ead;">) </span><span style="color:#bf616a;">-o</span><span> hello-world hello.c </span></code></pre> <p><img src="/images/making-with-make/gcc-make.png" alt="" /></p> <p>Now if I wanted to switch my various build targets to use <code>clang</code> instead, I can just modify my <code>COMPILER</code> variable.</p> <p><img src="/images/making-with-make/clang-make.png" alt="" /></p> <p>I could even move my program name and source file to their own variables, and reference that variable as my target and its dependencies.</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#bf616a;">COMPILER </span><span>= </span><span style="color:#a3be8c;">clang </span><span style="color:#bf616a;">PROGRAM </span><span>= </span><span style="color:#a3be8c;">hello-world </span><span style="color:#bf616a;">SOURCE </span><span>= </span><span style="color:#a3be8c;">hello.c </span><span> </span><span style="color:#8fa1b3;">all</span><span>: </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">PROGRAM</span><span style="color:#b48ead;">) </span><span> </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">PROGRAM</span><span style="color:#b48ead;">)</span><span>: </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">SOURCE</span><span style="color:#b48ead;">) </span><span> </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">COMPILER</span><span style="color:#b48ead;">) </span><span style="color:#bf616a;">-o </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">PROGRAM</span><span style="color:#b48ead;">) $(</span><span style="color:#bf616a;">SOURCE</span><span style="color:#b48ead;">) </span></code></pre> <p>Outside of the variables you define yourself, there are also some <a href="https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html">automatic variables</a> that can be pretty handy.</p> <p>The three I use the most are <code>$@</code> which if used in a command will be the name of the target, <code>$&lt;</code> which will be the first dependency for that target and <code>$^</code> which will be all of the dependencies for the target.</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#8fa1b3;">all</span><span>: </span><span style="color:#a3be8c;">automagic </span><span> </span><span style="color:#8fa1b3;">automagic</span><span>: </span><span style="color:#a3be8c;">automation magic </span><span> </span><span style="color:#96b5b4;">echo </span><span>&quot;</span><span style="color:#a3be8c;">target: </span><span style="color:#bf616a;">$@</span><span>&quot; </span><span> </span><span style="color:#96b5b4;">echo </span><span>&quot;</span><span style="color:#a3be8c;">first dependency: </span><span style="color:#bf616a;">$&lt;</span><span>&quot; </span><span> </span><span style="color:#96b5b4;">echo </span><span>&quot;</span><span style="color:#a3be8c;">all dependencies: </span><span style="color:#bf616a;">$^</span><span>&quot; </span></code></pre> <p><img src="/images/making-with-make/automagics.png" alt="" /></p> <p>These variables can be combined in interesting and useful ways. The automatic variables can even be embedded in your normal variables. Let’s say we have some C program that has a header and a code file, you’d want to rebuild the program if the header was changed but not include the header as an argument to the compiler. You could make your own compiler rule that includes most of the settings you want and define a pattern where the first dependency is included in the commands for the target.</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#bf616a;">PROGRAM </span><span>= </span><span style="color:#a3be8c;">myprogram </span><span style="color:#bf616a;">COMPILE_PROGRAM </span><span>= </span><span style="color:#a3be8c;">gcc -Wall -o </span><span style="color:#bf616a;">$@ $&lt; </span><span> </span><span style="color:#8fa1b3;">all</span><span>: </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">PROGRAM</span><span style="color:#b48ead;">) </span><span> </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">PROGRAM</span><span style="color:#b48ead;">)</span><span>: </span><span style="color:#a3be8c;">main.c main.h </span><span> </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">COMPILE_PROGRAM</span><span style="color:#b48ead;">) </span></code></pre> <p><img src="/images/making-with-make/fancy-variables.png" alt="" /></p> <h1 id="additional-command-fu">Additional Command-fu</h1> <p>There are two other things I think that are useful to know when writing the commands for the targets.</p> <p>So far we’ve seen all our output repeated, which is normally quite handy for debugging. If you feel like making your output a little prettier you can start your command with <code>@</code> to squelch the output.</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#8fa1b3;">all</span><span>: </span><span> </span><span style="color:#d08770;">@</span><span style="color:#96b5b4;">echo </span><span>&quot;</span><span style="color:#a3be8c;">one moment</span><span>&quot; </span><span> </span><span style="color:#d08770;">@</span><span style="color:#bf616a;">sleep</span><span> 5 </span><span> </span><span style="color:#d08770;">@</span><span style="color:#96b5b4;">echo </span><span>&quot;</span><span style="color:#a3be8c;">okay i&#39;m back</span><span>&quot; </span><span> </span><span style="color:#d08770;">@</span><span style="color:#bf616a;">sleep</span><span> 2 </span></code></pre> <p><img src="/images/making-with-make/sleep.gif" alt="" /></p> <p>Another good thing to be aware of is that each command is ran from your current working directory. If you want to do something like make a directory, jump into it and do more work inside of it, you’ll have to run multiple commands in a single go.</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#8fa1b3;">all</span><span>: </span><span> </span><span style="color:#d08770;">@</span><span style="color:#bf616a;">mkdir</span><span> subdirectory </span><span> </span><span style="color:#d08770;">@</span><span style="color:#96b5b4;">cd</span><span> subdirectory </span><span> </span><span style="color:#d08770;">@</span><span style="color:#96b5b4;">pwd </span><span> </span><span style="color:#d08770;">@</span><span style="color:#96b5b4;">cd</span><span> subdirectory; </span><span style="color:#96b5b4;">pwd </span></code></pre> <p><img src="/images/making-with-make/make-pwd.png" alt="" /></p> <p>If you find yourself with really long lines in your Makefile you can always add a backslash (<code>\</code>) before your new line to ask the <code>make</code> parser to ignore that as you’re just trying to make things pretty.</p> <p>And that will wrap up my post on Makefiles! I hope you find this useful!</p> <p><img src="/images/making-with-make/" alt="" /></p> <p><img src="/images/making-with-make/" alt="" /></p> <p><img src="/images/making-with-make/" alt="" /></p> 2017-11-29T00:00:00+00:00 2017-11-29T00:00:00+00:00 https://suchprogramming.com/beginning-logic-design-part-11/ 2017-11-22T00:00:00+00:00 2017-11-22T00:00:00+00:00 https://suchprogramming.com/beginning-logic-design-part-10/ 2017-11-15T00:00:00+00:00 2017-11-15T00:00:00+00:00 https://suchprogramming.com/beginning-logic-design-part-9/ 2017-11-08T00:00:00+00:00 2017-11-08T00:00:00+00:00 https://suchprogramming.com/beginning-logic-design-part-8/ 2017-11-01T00:00:00+00:00 2017-11-01T00:00:00+00:00 https://suchprogramming.com/beginning-logic-design-part-7/ 2017-10-25T00:00:00+00:00 2017-10-25T00:00:00+00:00 https://suchprogramming.com/beginning-logic-design-part-6/ 2017-10-18T00:00:00+00:00 2017-10-18T00:00:00+00:00 https://suchprogramming.com/beginning-logic-design-part-5/ 2017-10-11T00:00:00+00:00 2017-10-11T00:00:00+00:00 https://suchprogramming.com/beginning-logic-design-part-4/ 2017-10-04T00:00:00+00:00 2017-10-04T00:00:00+00:00 https://suchprogramming.com/beginning-logic-design-part-3/ 2017-09-27T00:00:00+00:00 2017-09-27T00:00:00+00:00 https://suchprogramming.com/beginning-logic-design-part-2/ 2017-09-22T00:00:00+00:00 2017-09-22T00:00:00+00:00 https://suchprogramming.com/beginning-logic-design-part-1/ 2017-09-20T00:00:00+00:00 2017-09-20T00:00:00+00:00 https://suchprogramming.com/getting-busybox-with-it/ <p>Following up on the <a href="https://suchprogramming.com/barebones-linux-system/">Building a Barebones Linux System</a> post, today I will be adding <a href="https://busybox.net/">BusyBox</a> to make the system a smidge more usable.</p> <h1 id="what-is-this-busybox">What is this BusyBox?</h1> <p>BusyBox is a refers to itself as <em>The Swiss Army Knife of Embedded Linux</em>. It is a collection of common Unix tools living within a single binary that uses the magic of <code>argv[0]</code> to facilitate a lightweight implementation of common programs like <code>sh</code>, <code>ls</code>, <code>cat</code> and many more. It is extremely common in the embedded systems community because it is a well maintained open source project and can be built very small.</p> <p>For my purposes, I will use BusyBox instead of going into a full <a href="http://www.linuxfromscratch.org/">Linux From Scratch</a> route. This will give me a shell and a few other basic userspace tools needed to allow me to make some use of this bare bones system.</p> <h1 id="automating-the-builds">Automating the builds</h1> <p>At this point I have been dabbling with this bare bones setup long enough that I want some additional tooling to help manage rebuilding as necessary. There are many good tools out there for managing custom Linux builds, but for now I’d like to continue using custom tools to learn more of what it takes to build a Linux distro from square one.</p> <p>I’ve made a <a href="https://github.com/KennethWilke/my-barebones-linux/tree/26dfc9441899541bf05aab9b07f22bf5ed9fb689">repo on Github</a> and a Makefile that will pull in the kernel, build it, and build my initramfs. It also has a <code>runvm</code> command that will help me test my builds quickly.</p> <pre data-lang="Makefile" style="background-color:#2b303b;color:#c0c5ce;" class="language-Makefile "><code class="language-Makefile" data-lang="Makefile"><span style="color:#bf616a;">KERNEL_VERSION</span><span>=</span><span style="color:#a3be8c;">4.13.3 </span><span style="color:#bf616a;">KERNEL_DIRECTORY</span><span>=</span><span style="color:#a3be8c;">linux-</span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">KERNEL_VERSION</span><span style="color:#b48ead;">) </span><span style="color:#bf616a;">KERNEL_ARCHIVE</span><span>=</span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">KERNEL_DIRECTORY</span><span style="color:#b48ead;">)</span><span style="color:#a3be8c;">.tar.xz </span><span style="color:#8fa1b3;">KERNEL_URL=https</span><span>:</span><span style="color:#a3be8c;">//cdn.kernel.org/pub/linux/kernel/v4.x/</span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">KERNEL_ARCHIVE</span><span style="color:#b48ead;">) </span><span> </span><span> </span><span style="color:#8fa1b3;">all</span><span>: </span><span style="color:#a3be8c;">vmlinuz initramfs </span><span> </span><span> </span><span style="color:#65737e;"># Kernel build targets </span><span style="color:#8fa1b3;">vmlinuz</span><span>: </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">KERNEL_DIRECTORY</span><span style="color:#b48ead;">) </span><span> </span><span style="color:#96b5b4;">cd </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">KERNEL_DIRECTORY</span><span style="color:#b48ead;">) </span><span>&amp;&amp; </span><span style="color:#bf616a;">make</span><span> defconfig &amp;&amp; </span><span style="color:#bf616a;">make -j</span><span>`</span><span style="color:#bf616a;">nproc</span><span>` </span><span> </span><span style="color:#bf616a;">cp </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">KERNEL_DIRECTORY</span><span style="color:#b48ead;">)</span><span>/arch/x86_64/boot/bzImage vmlinuz </span><span> </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">KERNEL_DIRECTORY</span><span style="color:#b48ead;">)</span><span>: </span><span> </span><span style="color:#bf616a;">wget </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">KERNEL_URL</span><span style="color:#b48ead;">) </span><span> </span><span style="color:#bf616a;">tar</span><span> xf </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">KERNEL_ARCHIVE</span><span style="color:#b48ead;">) </span><span> </span><span> </span><span style="color:#65737e;"># Initramfs build targets </span><span style="color:#8fa1b3;">initramfs</span><span>: </span><span style="color:#a3be8c;">initfs initfs/init </span><span> </span><span style="color:#96b5b4;">cd</span><span> initfs/ &amp;&amp; </span><span style="color:#bf616a;">find</span><span> . | </span><span style="color:#bf616a;">cpio -o --format</span><span>=newc &gt; ../initramfs </span><span> </span><span style="color:#8fa1b3;">initfs/init</span><span>: </span><span style="color:#a3be8c;">initfs init.c </span><span> </span><span style="color:#bf616a;">gcc -o</span><span> initfs/init</span><span style="color:#bf616a;"> -static</span><span> init.c </span><span> </span><span style="color:#8fa1b3;">initfs</span><span>: </span><span> </span><span style="color:#bf616a;">mkdir -p</span><span> initfs/bin </span><span> </span><span> </span><span style="color:#65737e;"># Utility targets </span><span style="color:#8fa1b3;">runvm</span><span>: </span><span style="color:#a3be8c;">vmlinuz initramfs </span><span> </span><span style="color:#bf616a;">qemu-system-x86_64 -m</span><span> 2048</span><span style="color:#bf616a;"> -kernel</span><span> vmlinuz</span><span style="color:#bf616a;"> -initrd</span><span> initramfs </span><span> </span><span style="color:#8fa1b3;">clean</span><span>: </span><span> </span><span style="color:#bf616a;">rm -rf</span><span> vmlinuz </span><span style="color:#b48ead;">$(</span><span style="color:#bf616a;">KERNEL_DIRECTORY</span><span style="color:#b48ead;">) $(</span><span style="color:#bf616a;">KERNEL_ARCHIVE</span><span style="color:#b48ead;">) </span></code></pre> <h1 id="configuring-and-building-busybox">Configuring and Building BusyBox</h1> <p>BusyBox uses buildroot so its configuration and build process are very similar to the kernel itself. After downloading and extracting the source I’ll use <code>make menuconfig</code> to use the ncurses based config editor.</p> <p>I’ll be leaving most settings at their default, but will configure the build to generate a static binary since I don’t yet have a <a href="https://en.wikipedia.org/wiki/C_standard_library">libc</a> implementation to link against.</p> <p><img src="/images/getting-busybox-with-it/busyboxing.gif" alt="" /></p> <p>With BusyBox built, I’ll add it to my <code>initfs</code> directory so that my Makefile includes it in my initramfs.</p> <p><img src="/images/getting-busybox-with-it/busyinitfs.gif" alt="" /></p> <p>The <code>busybox</code> binary will act differently based on what it is called as, if we renamed it to <code>sh</code> it would act as a shell, or if named <code>ls</code> it will list our files and such. To allow it to serve all of these functions I’ll check what commands my build implements and create <a href="https://en.wikipedia.org/wiki/Symbolic_link">symlinks</a> for each program it provides.</p> <p><img src="/images/getting-busybox-with-it/busylinking.gif" alt="" /></p> <h1 id="switching-the-build-to-busybox">Switching the Build to BusyBox</h1> <p>With my process for building BusyBox roughly figured out, I’ll save a copy of my BusyBox config, so that future builds will also be static linked. I’ll also include a small shell script that builds my symlinks.</p> <pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#96b5b4;">cd</span><span> initfs/bin </span><span style="color:#b48ead;">for</span><span> i </span><span style="color:#b48ead;">in </span><span>`</span><span style="color:#bf616a;">./busybox --list</span><span>` </span><span style="color:#b48ead;">do </span><span> </span><span style="color:#bf616a;">ln -s</span><span> busybox $</span><span style="color:#bf616a;">i </span><span style="color:#b48ead;">done </span></code></pre> <p>Now that I have BusyBox in my initramfs, I’ll switch my init process from my custom binary to a shell script that will forever spawn a shell to use.</p> <pre data-lang="sh" style="background-color:#2b303b;color:#c0c5ce;" class="language-sh "><code class="language-sh" data-lang="sh"><span style="color:#65737e;">#!/bin/sh </span><span> </span><span style="color:#b48ead;">while </span><span style="color:#bf616a;">true </span><span style="color:#b48ead;">do </span><span> </span><span style="color:#bf616a;">sh </span><span> </span><span style="color:#bf616a;">sleep</span><span> 1 </span><span style="color:#b48ead;">done </span></code></pre> <p>With this script in place I’ll update my Makefile and repo to include the changes I’ve made here.</p> <p>And now we test!</p> <p><img src="/images/getting-busybox-with-it/busytesting.gif" alt="" /></p> <p>There are a few deficiencies in here still, e.g. <code>ps</code> won’t work because I have no <a href="https://en.wikipedia.org/wiki/Procfs">procfs</a> mounted and <code>sh</code> will complain about being unable to access tty.</p> <p>To get a few more things to run sanely, I’ll add <code>/dev</code>, <code>/proc</code> and <code>/sys</code> to my initramfs and modify my init script to mount the procfs and <a href="https://en.wikipedia.org/wiki/Sysfs">sysfs</a>. I’ll also run <code>mdev -s</code> which is provided by BusyBox to populate <code>/dev</code> and <code>/sys</code>. Lastly, I’ll launch my shell with <code>setsid cttyhack sh</code> so that <a href="http://tldp.org/LDP/abs/html/x9644.html">job control</a> is enabled.</p> <p>With all that setup, the system is substantially more usable! I’ll <a href="https://github.com/KennethWilke/my-barebones-linux/commit/424fc5491a12cea2ec35c4dfa1ca02c7a24eaafc">push these changes to my repo</a> and call it a wrap.</p> 2017-09-11T00:00:00+00:00 2017-09-11T00:00:00+00:00 https://suchprogramming.com/barebones-linux-barreleye/ 2017-09-08T00:00:00+00:00 2017-09-08T00:00:00+00:00 https://suchprogramming.com/barebones-linux-system/ <p>In this post, I will describe the process I’ve used to create a very minimal <a href="https://en.wikipedia.org/wiki/Linux">Linux</a> build. It is minimal for the sake of learning, to get a clearer picture of how barebones a operating system could be put together. The build will consist of an <a href="https://suchprogramming.com/barebones-linux-system/%22https://en.wikipedia.org/wiki/Kernel_(operating_system)%22">operating system kernel</a> and an <a href="https://en.wikipedia.org/wiki/Initial_ramdisk">initial ram disk</a>.</p> <p>I will be using Ubuntu 16.04 on my laptop as my build box, and use QEMU VMs as my test systems. If you’d like more information on building a linux kernel <a href="https://kernelnewbies.org/KernelBuild">there is a decent tutorial on kernelnewbies</a> as well as <a href="https://wiki.gentoo.org/wiki/Complete_Handbook/Building_the_Linux_kernel">the kernel building section of the Gentoo handbook</a>.</p> <fieldset class="ad"> <legend>ad</legend> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-5305013514298750" crossorigin="anonymous"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-5305013514298750" data-ad-slot="6574356918"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </fieldset><h1 id="the-linux-kernel">The Linux Kernel</h1> <p>It can be difficult to see the separation of what is Linux kernel and what is the Linux distribution. The Linux Kernel itself is there primarily to facilitate access to hardware and provide a safe haven for processes to live and breathe. The kernel’s user interface is its API. It provides a foundation for all other software to allow storing files, using networks, render graphics, handle keyboard input and so on. On top of the kernel your terminal, your login process, your window manager, your browser, etc. are usually provided by your <a href="https://en.wikipedia.org/wiki/Linux_distribution">distribution</a>.</p> <p>Obtaining and building the kernel is very simple, though configuring the kernel can be a little trickier depending on your needs. For the sake of this excersize the default configurations for an x86_64 system will do just fine.</p> <p>First off, I’ll create a new directory to work out of and get started. As of today, the latest linux kernel at the top of <a href="https://www.kernel.org/">kernel.org</a> is <code>4.13</code>, I will download this latest tar file, extract it and jump into the extracted directory.</p> <p><img src="/images/barebones-linux-system/download-kernel.gif" alt="Downloading the kernel" /></p> <p>Now that I have the kernel source files, I’ll use <code>make defconfig</code> to configure the defaults for a standard x86_64 machine and will do <code>make -j4</code> to compile the kernel using all 4 threads my laptop has to offer.</p> <p><img src="/images/barebones-linux-system/kernel-startbuild.gif" alt="Starting the kernel build" /></p> <p>Even with the extra threads, it takes about 8 minutes to compile on my machine. It’s not the simplest kernel as there are many default features we could disable to trim down the kernel size and compile time but I’m not worried about that for this build. Once it’s complete I copy the <code>bzImage</code> file that was generated into my barebones parent directory.</p> <p><img src="/images/barebones-linux-system/kernel-done.gif" alt="Kernel build finishing" /></p> <h1 id="init-process-and-initramfs">Init process and initramfs</h1> <p>Typically when a computer boots, the host firmware will look for a bootloader on a bootable device. The bootloader will load the kernel and pass control of the system to it. The kernel gets control, does it’s own startup shenanigans and looks for a program to run as the primary (init) userspace process (PID 1).</p> <p>In most Linux distributions, this init process can be a <a href="https://en.wikipedia.org/wiki/Init">SysV style init process</a>, <a href="https://www.freedesktop.org/wiki/Software/systemd/">SystemD</a>, <a href="http://upstart.ubuntu.com/">Upstart</a> or <a href="https://wiki.gentoo.org/wiki/OpenRC">OpenRC</a>. In this case, it’s going to be a looping hello world program. I compile it with <code>-static</code> so that libraries are <a href="https://suchprogramming.com/barebones-linux-system/%22https://en.wikipedia.org/wiki/Linker_(computing)#Static_linking%22">staticly linked</a></p> <p><img src="/images/barebones-linux-system/init-program.gif" alt="Compiling a simple hello world looping program" /></p> <p>The next step is to build the initramfs, the initial file system that will go in ram for the kernel to interact with. To do this, I will <a href="https://ryanstutorials.net/linuxtutorial/piping.php">pipe</a> a newline separated list of files to pack to <a href="https://en.wikipedia.org/wiki/Cpio">cpio</a>. I only have one file so I’ll just echo that to it. <code>cpio</code> will output the archive and I’ll store that to a file.</p> <p><img src="/images/barebones-linux-system/initramfs.gif" alt="Building an uncompressed initramfs with a single file" /></p> <h1 id="testing-the-new-build-in-qemu">Testing the new build in QEMU</h1> <p>I’ll be using qemu to test the new build. The simplest way I could boot the VM would be to use <code>qemu-system-x86_64 -kernel vmlinuz -initrd initramfs</code>, but to run this within my terminal I’ll also add <code>-append console=ttyS0 -nographic</code>.</p> <p><img src="/images/barebones-linux-system/qemu-launch.gif" alt="Testing the kernel in QEMU" /></p> <p>It boots fast and my hello world endlessly spits out its message! My barebones OS works! I may eventually use this to build a few utility images to network boot but for now this will do as a base for more tinkering.</p> 2017-02-14T00:00:00+00:00 2017-02-14T00:00:00+00:00 https://suchprogramming.com/using-capi-within-a-docker-container/ 2016-09-26T00:00:00+00:00 2016-09-26T00:00:00+00:00 https://suchprogramming.com/simulating-capi-designs-with-pslse-and-vivado/ 2016-06-28T00:00:00+00:00 2016-06-28T00:00:00+00:00 https://suchprogramming.com/peeking-into-the-alpha-data-ku3/ 2016-06-14T00:00:00+00:00 2016-06-14T00:00:00+00:00 https://suchprogramming.com/hello-afu-on-alpha-data-ku3/ 2016-03-08T00:00:00+00:00 2016-03-08T00:00:00+00:00 https://suchprogramming.com/hello-afu-part-6/ 2016-03-07T00:00:00+00:00 2016-03-07T00:00:00+00:00 https://suchprogramming.com/hello-afu-part-5/ 2016-03-02T00:00:00+00:00 2016-03-02T00:00:00+00:00 https://suchprogramming.com/hello-afu-part-4/ 2016-02-19T00:00:00+00:00 2016-02-19T00:00:00+00:00 https://suchprogramming.com/hello-afu-part-3/ 2016-02-17T00:00:00+00:00 2016-02-17T00:00:00+00:00 https://suchprogramming.com/hello-afu-part-2/ 2016-02-15T00:00:00+00:00 2016-02-15T00:00:00+00:00 https://suchprogramming.com/hello-afu-part-1/ 2016-01-18T00:00:00+00:00 2016-01-18T00:00:00+00:00 https://suchprogramming.com/tinkering-with-capi/