HFI BIOS: from idea to K3 support

On the 2nd of July I made the first commit in a repository called fu740-bios. Its README opened with a single sentence that never changed, even after the project outgrew the board in its name:

A RISC-V machine that powers on and greets you like a classic PC.

Not a boot loader with a menu. A BIOS: the rainbow banner, the memory count, the hardware summary, and the immortal Press DEL to run Set-Up — on a real monitor, through a real graphics card, driven by a real USB keyboard. I had been turning that idea over for something like twenty-five years.

On the 14th of August it does that on two entirely different machines, and today I am tagging 1.3. This is the story of the six weeks in between.

The dead end that shaped everything

The project actually starts in June, in a directory called gk208-vbios-runner, and with the obvious idea: a graphics card carries its own initialisation code in its option ROM, so let the card initialise itself. Run the NVIDIA GK208's x86 VBIOS on RISC-V under emulation and let it do the modeset for us. That is, after all, exactly what a PC BIOS does — one far call into the option ROM, and the card wakes up.

I got a long way. Far enough to be tracing the card's display supervisor register by register, watching the VBIOS arm an update at 0x610700 and waiting for hardware to process it. And far enough to find the thing that ended the approach: on a real PC the VBIOS runs against a card at cold power-on reset. On my board it ran against a card that Linux had already touched, whose display engine came up with a stale update pending bit already set. We were POSTing a dirty engine.

That is fixable. But standing there, halfway through emulating one architecture to persuade a second architecture's firmware to program a third vendor's hardware, the shape of the thing finally registered: I was building an enormous amount of machinery whose entire purpose was to avoid writing a display driver. So I wrote the display driver. Native GK208 bring-up, no x86, no VBIOS execution, no emulator. That decision is why everything after it was possible.

July: from nothing to 1.0 in seventeen days

The BIOS front-end compiles into U-Boot and takes over main_loop(). U-Boot is a superb hardware substrate and a terrible user experience; the idea was never to replace it but to put a face on it. Version 0.1 on the 4th of July was the POST screen and the card coming up natively. Then, roughly one capability per day:

• a text-mode UI toolkit, windows, and the F11 boot menu;
• Set-Up — a two-panel System Configuration Editor in the Award idiom, which is really an editor for U-Boot environment variables wearing a very good costume;
• the System Console — the full U-Boot command line, hidden behind a keystroke, because the machine should still be a development board when you need it to be;
• a working real-time clock and a Standard CMOS date page;
• an on-screen EFI console, so a boot loader's text lands on the monitor instead of only on a serial cable;
• and a Flash Utility that writes the firmware to NOR from inside Set-Up, verify-then-write, never touching the environment sector.

Version 1.0 was tagged on the 19th of July. It was covered in two news articles (one on Phoronix, one on TheRegister).

The turn: video as a contract, not a driver

The most consequential change of the whole project has no user-visible effect at all. On the 13th of July the GK208 driver stopped being source code compiled into the BIOS and became a loadable, self-relocating blob behind a published ABI — a VideoBIOS Module. The BIOS became a loader that knows nothing about any card.

This is the actual thesis of HFI, the Harmonic Firmware Initiative, and it is worth stating plainly. On a PC, every graphics card in the world can bring itself up and present a text console, because there is a contract — an interface every card implements and every BIOS can call. RISC-V has no such contract, so every board grows its own display bring-up, or does without. HFI is an attempt at that contract: a card-neutral interface, a module per controller, a firmware that loads it. Whether the module inside is a Kepler GPU driver or a SoC display controller is not the BIOS's business.

Once video was a contract rather than a driver, the rest of the year's work became possible. Two weeks later QEMU became a real second platform — not a shortcut, but the same firmware chain a board runs, its own SPL reading a FIT off an NVMe disk, settings persisting in an emulated flash chip. And in August a board arrived that would have been unthinkable in July.

August: the SpaceMiT K3, and what porting really costs

The K3 Pico ITX is a 16-core RISC-V board — eight X100 and eight A100 harts, 16 GiB of LPDDR5 — and it is outside mainline U-Boot. That single fact is most of the work. The first commit of the K3 series landed on the 3rd of August; by the 6th the board cold-started from its own NOR, through our firmware, into a Bianbu Linux desktop.

Between those dates: eighteen patches. A clock and reset controller of roughly four thousand lines, without which every driver ran on whatever the previous boot stage happened to leave enabled. UFS storage, USB, QSPI NOR, a watchdog that resets the board mid-boot if you merely stop feeding it, I²C and the identity EEPROM, two real-time clocks, eight temperature sensors, three PCIe root complexes. None of it in the BIOS front-end, which came up essentially unchanged. That was the point of the second board: it tells you which parts of your firmware were general and which were one board's habits written into shared code.

The character generator is a CPU

The K3 has no PC-style text mode and no character generator. Its display controller scans out a framebuffer in system RAM and nothing else. So the K3's VideoBIOS module does the only thing left to do: it borrows one of the machine's own harts and turns it into a character generator, painting glyphs into that framebuffer for as long as firmware owns the machine.

I still find this the most pleasing thing in the project. The card-neutral video interface was designed for graphics cards, and one of its implementations is a CPU core running a font renderer. The abstraction held.

It also creates an obligation that took a while to get right. That hart is one of the machine's processors; the operating system will bring it online and reuse its memory. A module still executing there when Linux does that is not untidy, it is fatal. So at ExitBootServices the firmware tells the module to stand down and gives the core back — unless the payload has declared, in a section of its own executable, that it speaks our handoff protocol and would like to keep the text console. mr-bml, our own loader, says so from 0.8.1 onwards.

Three bugs that were all the same bug

The K3 taught the same lesson three times, each time more expensively.

The panel was dark on a cold boot, and only on a cold boot. Every warm reboot was fine. Hot-plug detect reaches the SoC through an ordinary pad, and a pad comes out of cold reset selecting function 0 — so there was no AUX channel, no EDID, no link training, and a display controller that was blameless throughout. Every warm boot had simply inherited the pin muxing from whatever ran before.

Every PCIe endpoint stayed in reset while the PHY reported a perfect link. Nothing in the boot chain muxed the pads carrying PERST#. Same species, second occurrence.

And then, this week, the third — the one that cost the most and taught the most.

A pad with no voltage answers every write and drives nothing

The board's Ethernet did not work. The PHY answered 0xffff at every one of the thirty-two MDIO addresses and at every clock divider. That is precisely what a PHY held in reset looks like, and I read it as one for two days.

It was not. The PHY's reset line is a GPIO, and the GPIO could not drive. Its direction register accepted the write and read back correctly. The set and clear registers accepted theirs. The pin never moved.

The cause is that a pad on this SoC has no IO voltage at all until the power domain it belongs to is switched on in the AIB — the analog IO block. Until then the pad behaves as a bare input: every register in the GPIO block works perfectly, and none of it reaches the outside world. With that domain off, MDC and MDIO never left the chip. The fault had nothing to do with Ethernet, or with GPIO, or with either driver.

Three writes fix it:

writel(0xbaba, 0xd4015050);   /* APBC_ASFAR -- unlock          */
writel(0xeb10, 0xd4015054);   /* APBC_ASSAR                    */
writel(BIT(2), 0xd401e804);   /* pins 0..20 IO domain := 1.8 V */

Two properties of that register are why this was so expensive, and neither is peculiar to this SoC:

• The block is locked. It reads as zeros until the key pair is written. So comparing my registers against firmware that worked showed zeros on both sides and explained nothing — and I had already dismissed that block as empty, twice, on exactly that evidence.
• The enable is sticky. Once set it survives the MAC being stopped, held in reset, and clock-gated. So it never showed up in a before-and-after comparison of anything else either.

I achieved complete parity with the working vendor firmware — the APMU, the pads, all thirty-eight differing MAC registers, the GPIO block, every clock and power-management register either firmware touches — with the link still dead. I treated that result as a reason to compare more registers. It was nothing of the sort. It was positive evidence that the mechanism was not in the readable state at all.

What actually worked, in about forty minutes, was building the vendor's own U-Boot with a dozen lines of instrumentation that drove an unused pad and reported whether it followed. The very first line of output said the pad was already driving at the entry to the Ethernet driver's probe — which placed the cause earlier, in the driver-model probe chain, and from there in the pin controller, which reaches the AIB from a power-source property on a pin group.

When a state comparison comes out byte-identical and the behaviour still differs, that is not a mystery to be resolved by comparing more registers. Read what the working firmware does, not what it leaves behind.

Within the hour, dhcp bound an address in 57 milliseconds and ping reached my workstation.

Where 1.3 stands

On the K3, 1.3 cold-starts from NOR, lights the panel before anything else, and brings up USB, UFS, three PCIe root complexes, the fan and the network — then hands a Linux desktop a machine in the state it expects to find one, with the display still driven by a processor borrowed from that same machine. On the Unmatched it does what 1.0 did, better. On QEMU it does it with no hardware at all.

The fan deserves its own sentence, because it is the most BIOS thing in the release. The K3's fan hangs off an embedded controller that will not spin it until a host asks, and nothing in the boot chain ever asked — so firmware sitting at a Set-Up prompt drove the SoC into the low eighties Celsius. A BIOS that lets you park at its own menu owes you a cooled machine. Now it starts the fan during POST, and PC Health Status shows you the tachometer next to the temperatures.

Everything is GPL-2.0-or-later. Two manuals ship with it — a User Manual for the operator and a Technical Reference for whoever ports it next. Appendix A.7 of the latter is the pad-with-no-voltage story, written up properly, because the next person to meet it on a different SoC should not have to spend the two days I did.