FOSSi Fantasies 2022

 

2022 is behind us and as usual I'm wrapping up my open source silicon efforts of the past year in a blog post.

Hang on...a 2022 retrospective...? In... the end of April??

Yes, I am fully aware that a third of 2023 has already passed, thank you very much, and that it's way too late to write a new year's retrospective. I have just been extremely busy, but I still wanted to get it out of the system. It's only been a month since the Persian new year, so in the light of that I'm not that late. Anyway...

This year, the list is shorter than usual simply because I haven't done as much FOSSi work as previous years. The upside is that I managed to complete this article close to new year, rather than much later as in previous years.

But why have I done less work? The main reason is that I have been terribly busy with my day job which is mostly of proprietary nature. While I do a lot of interesting stuff in this capacity, it's unfortunately not much I can talk about publicly. This is another reason why I greatly prefer open source work, so that I can show, share ideas and collaborate with other people.

There is one thing however from my day job that I can and want to talk about. This summer we launched a fully remote office that we call Qamcom Anywhere. Having worked remote myself for the past four years I have been pushing to make it possible for more people in the company to do the same, and this year we got it going for real. Qamcom Anywhere has been a massive success and we have found amazing new colleagues in various parts of Sweden where we previously haven't been looking before. Given my previous experience with working remote as well as working on open source projects, which by nature tend to be highly distributed, I was tasked to run this new office. As part of the campaign we also recorded a commercial, so I can now also add movie star to my CV ;)

As for now, we have launched Qamcom Anywhere in Sweden, but hope to spread to more countries in the future. Stay tuned if you want to be colleagues!

But even outside of this I have found some time to work on my long list of open source silicon projects.

Let's start by looking at what has happened to FuseSoC over the past year. Most of the effort has been spent on getting FuseSoC in shape for a long overdue 2.0 release. A couple of major features and changes were identified as being important to complete before this release. Most notably is the support for the new flow API in Edalize, but a number of critical bug fixes and backwards-incompatible changes were put in place. Unfortunately, we never manages to get the 2.0 release out of the door, but we got close and at least released a first release candidate in late December, while the final release saw the light in the beginning of this year.

Most of my other FOSSi projects made good progress. There were a few new Edalize releases, SweRVolf got some new board support, a maintenance release to an old i2c component that I maintain (hey! it's important to put some effort into cleaning up old code, not just rewrite new code all the time) and even good old ipyxact saw a new release, which now contains ipxact2v, a very handy tool to automatically convert IP-XACT designs to Verilog top-levels. It's not fully complete, but the functionality that exists is already coming to good use in various projects.

The project that probably saw the most interesting news in 2022 was SERV. SERV itself gained support for compressed instructions, thanks to Abdul Wadood who I had the great pleasure of mentor through Linux Foundation's LFX Mentorship Program. And aside from improvements to the core itself, in 2022, fellow FOSSi superstar developer Florent Kermarrec, who might be most known for Litex, managed to run 10000 SERV cores in a Xilinx FPGA. This seems to be the current world record right now for the most RISC-V cores in a single device, but I'm very curious how the competitors will react (looking at you, Intel!).

This year was the first in three years where I didn't create any new videos (or Fully Immersive Multimedia Edutainment Experiences, as I prefer to call them). Instead I did a number of presentations using old-fashioned slides for a live audience. The first of them being the RISC-V Week in Paris in early May where I did both a presentation on FuseSoC as well as one on SERV. The SERV video was unfortunately never published, but the slides for the presentation called How much score could a CoreScore score if a CoreScore could score cores? can be found here. I did another FuseSoC talk at FPGA World in Stockholm, Sweden which was also not recorded, but the final one, from the RISC-V Summit in San Jose was. This one was title SERV: 32-bit is the new 8-bit and aims to look at how RISC-V can be competitive in the traditional 8-bit market thanks to SERV.

Both at the RISC-V week as well as the RISC-V Summit I also got the chance to meet with the people behind RVFPGA, a project I have been involved with almost since the start. For those unaware, RVFPGA is a free computer architecture course by Imagination University Programme that runs on a slightly modified version of SweRVolf that I built a couple of years ago. Right after the RISC-V Summit I also got the chance to watch a RVFPGA workshop in action, and it was super fun to see all these students working their way through the labs.

Let's see.... what else then...hmm... you know what? I'm sure other things in 2022 as well, but my memory is fading and May is just outside the door waiting to come in, so let's just cut it here before the rest of the year passes too. Here's to 2023. Happy new year!

FuseSoC 2.2

Do you know the best way to find out who is using your open source software? Introduce bugs! You will suddenly come in touch with a lot of users you didn't know existed. And let's just say I found out about a lot of new users after the release of FuseSoC 2.1. And with FuseSoC 2.1 having a lot of new features, it's perhaps not too surprising that the odd bug crept in.

But enough about that, because FuseSoC 2.2, the topic for today, has hopefully fixed what was broken. And of course we have a couple of new features as well, even though the list is somewhat shorter than usual. But let's see what the new version has to offer

JSON Schema

Generally, I'm pretty happy about the code quality of FuseSoC. It has proven to be relatively friendly to new contributors and has gone through a couple of major refactorings without too much problems over its almost 12 years of existence. But there is one part of the code base that I usually try to stay clear from.

Deep inside of the FuseSoC code base there is a yaml structure encoded inside a Python string that is parsed when the module is imported to dynamically create a tree of Python classes which are then used to recursively read and validate core description files. Pretty clever, right? This is a fantastic example of the sort of thing that seems great because it's possible and not too hard to actually do with Python. Now, the thing is, because of the cleverness of the code, it is pretty much unreadable even for me who wrote it. Every time I need to fix some bug in this area of the code I end up spending hours trying to figure out how it all works, all the time crying and asking why oh why I built it like this in the first place.

So what little time was saved on writing some more verbose code, we pay for over and over again in maintenance. Not to mention all the bizarre corner cases that arises because the code is trying to outsmart itself. Things that required us to create classes like this:

The time was ripe now to rework this whole thing into something more sensible. So what we do instead now is to have a JSON Schema definition of the CAPI2 format...encoded as a string in a Python module deep inside the FuseSoC code base. I understand this doesn't look all that much like an improvement, but it's the first, and most important, step of a journey.

Short-term this leads to a more maintainable parser and validator because we only need to care about the definition. There is battle-proven Python code already that does the actual validation and is better at pointing out where in a core description file there is an error. There are also other utilities for generating documentation to offload this from FuseSoC itself. The parsing is also a bit more consistent now and supports use flag expansion in more places.

But long-term, this paves the road for actually splitting out the CAPI2 definition to its own project that can be readily reused by other tools without having to use FuseSoC. Having the validation code in jsonschema allows for much easier reimplementations and utilities written in other languages than Python. The first case that comes to mind is JavaScript for having web-based utilities around CAPI2 or built-in validation of core description files in e.g. VS code. But it also makes it easier to implement support for CAPI2 files directly in EDA tools written in Java, C++ or why not Rust.

It should be noted that the new parser is a bit more strict than the old one, so it might complain on files that were previously deemed ok. Hopefully there shouldn't be too many of those. There's also a new command-line switch --allow-additional-properties that can be turned on to make the parser more relaxed towards elements in the core description files that it doesn't know about.

Tags

The other thing I want to mention in this release is a small code change that I think will have open up for more use cases. It's now possible to set tags for files or filesets, very much like we can set file_type or logical_name today. FuseSoC itself doesn't care about the tags, but they are passed on to Edalize through the EDAM file. In Edalize, since version 0.5.0 we have begun to look at tags in some of the flows and take decisions upon them. The only tag that is recognized today is the "simulation" tag, that can be set on HDL files to indicate they are intended for simulation and not for synthesis. This change opens up for use-cases such as gate-level simulation where we first send our code through a synthesis tool and then the created netlist is simulated together with a testbench. By marking the testbench files with simulation, we tell the synthesis tools to not try to synthesize them into the netlist but instead pass them on to the simulator untouched. Another future use-case is for TCL files. There might be many tools in a tool flow that parses TCL files and so far, there hasn't been a way to tell the backend for which tool a particular TCL file is intended. I suspect we will see a whole bunch of more use-cases in the future.

Other things

I mentioned some bugs, right? A big one was that users of the old tool API (which I believe is still most users) noticed that the FPGA image or simulation model was not rebuilt when source files were changed. The new flow API has some properties that allows us to track changes in a much better way and avoids unnecessary rebuilds in many cases. Unfortunately, when these changes were made we didn't properly test how that affected the tool API.

Another issue was reported from users who uses --no-export together with generators. The recently introduced caching mechanism forced us to rewrite much of the code around generators and unfortunately we ended up missing this case, where the generated code got removed before it was used. Whoops. Also fixed now.

All in all, I hope you enjoy the new features and the new release. Happy FuseSoCing!

Edalize 0.5.0

A new version of Edalize has just been released and I'm taking the opportunity to dust off this old blog by writing a bit about some of the highlights.

In case someone is wondering, Edalize is an open-source abstraction library for EDA tools that allow users to make a description of their RTL design once in a format called EDAM (EDA Metadata) and then Edalize can setup a project for any of the 40+ supported EDA tools. This allows quickly retargeting a design for different flows and saves users from learning all the intricacies of each of these EDA tools. Edalize does the job for you. This five minute award-winning introduction video goes into more detail for those who are interested.

Anyway, Edalize 0.5.0 was just released and it contains a whole bunch of new features.

New backends

Three new backends were added for the legacy tool API. These are called filelist for producing .f files, sandpipersaas for the SandPiper TL-Verilog compiler and openroad for the OpenROAD toolchain. And with this, we now support 40 different EDA tools with the same simple interface. Not bad considering the EDA world is mostly known for its insane level of incompatibilty, vendor lock-in and general user hostility. Still waiting for the EDA vendor to natively support the EDAM format though. That would make Edalize unnecessary and I could spend my time improving the world instead of adding new tool support.

Note: With this release, the old tool API is under feature freeze. Only backends currently under review will be considered for addition. No new features of backends will be added outside of that. All future work should be done for the new flow API instead

New flows

Two flows have been added to the new Flow API. generic is a customizable flow that can be used for all single-tool flows. It is also used internally as a parent class to the sim and lint flows which saves quite a few lines of code.

gls is a gate-level simulation flow that runs a synthesis flow to produce a netlist followed by a simulator. All simulators and synthesis tools covered by the new flow API should be automatically supported. It is also a good way to show off some of the benefits of the new flow API. Doing this with the old tool API would had been terribly complicated.

Note: Help is needed to port all the tool classes from the legacy tool API to the new flow API. There is still only a small subset of tools supported.

External tool and flow plugins

Edalize now supports PEP0420, also known as implicit namespace packages. What does means in practice is that new flows and tools can be added outside of the Edalize code base, so that Edalize users who needs support for some internal proprietary tool or wants to run an esoteric flow can add these without having to modify the Edalize code base. There is documentation that describes how this works in practice.

Avoid unnecessary rebuilds

A much-requested feature is to avoid rebuilding when the source files haven't changed. This is now possible for most cases using the new flow API. A demonstrator will also soon be released that shows an advanced example of this will soon be released. For most users (at least the ones who have already migrated to the flow API), the only visible change will be that builds will become faster in many cases.

Smaller things

The modelsim backend supports mfcu mode where all files are compiled into the same compilation unit. icarus supports multiple toplevels. yosys and vivado both support verilog output and recognizes files tagged with the simulation tag. Both these features are used by the gate-level sim flow. The internal flow API has been improved to make creation of new flows easier and have less boilerplate code.

That's pretty much it. Getting the newest version of edalize should be as easy as running pip install -U edalize (the -U forces an upgrade if you already have an older version installed). Hope you enjoy it!