Don't copy that IP

Making copies of other people's IP is a terrible thing! And it's a really big problem in the EDA world.

 

Oh, I'm not talking about the legal aspects. That's something I happily leave to the lawyers. No, I'm talking about making copies of source code instead of referencing the original code. It's a big problem because suddenly you have two different versions, and the chances that a fix that was done in one version will reach the other version are slim to none. When I was entering the world of open source silicon around 2010, this was how things were normally done. And my first reaction was to try and upstream all fixes to IP cores that I found in various projects that were using those cores. My second reaction was to try and find a more sustainable way to avoid this problem in the first place. And solving this problem was one of the initial driving forces behind FuseSoC, and it still is. So it makes me very very sad when, twelve years later, I still find random copies of IP cores, all with various subsets of fixes applied to them.

When we started working on the OpenLANE Edalize backend, one important thing was to have as many example designs as possible, both to make sure the backend was flexible enough to cover all use-cases and also to have a good set of examples for anyone interested in adding FuseSoC support for their own cores. We found a number of example designs in the OpenLANE repository which were used to test OpenLANE itself. Great! ...except... it turned out most of the examples were cores or parts of cores copied from various places. So we did what every sensible person (not really) would do. We decided to upstream all those example designs, so that the OpenLANE support and any other fixes would benefit all users of that IP core.

Bringing' it all back home

At the time we started looking at this, there were 32 different example designs. Our first job was to find out where on earth all these came from. That took a fair amount of detective work, but in the end we found what we believe is the proper upstream for all, except perhaps for one where we are a bit unsure. In that case we chose to file a PR against OpenLANE since that was the closest we could get to an upstream. A few designs were dropped from the OpenLANE examples while we were working on this, in which case we also chose to ignore them.

Once we had identified the origin, we tried to figure out how they worked and then set off to add FuseSoC support for each and every core. At the minimum we added a target for linting and a target for building with OpenLANE through the Edalize backend. In the cases where we found testbenches, we also added support for running those, with a few exceptions where that required tools we didn't have access to.

In addition to adding FuseSoC support, we also added support for running the targets as GitHub CI actions, so that every new commit to the project would automatically lint, build a GDS with OpenLANE and potentially run some testbenches.

And finally we packaged it all up and sent a humongous number of pull requests to different projects with detailed instructions how they could use this. Many of these pull requests have been accepted, but not all. There's not much more we can do about that however. If you are curious, it's possible to check the progress here

So, what does this really mean? Did we make the world a better place for you and for me and the entire human race? I like to think so. At least we didn't make it worse. There are a couple of very real benefits to this work.

• Adding FuseSoC support makes it eaiser for other users to use these cores in their own projects
• Adding testbench targets makes it easy for other people to check the cores work as expected
  
• Adding a lint target makes it easy to check code quality. Far from all of the designs we encountered pass the lint check
• On a few occasions, fixes had been made in the copies. These were fed upstream to benefit all users of those cores
• The CI actions makes it easy to check nothing breaks on future updates of the cores
• We could test the Edalize OpenLANE backend on a number of different designs to ensure it was flexible enough to handle them all
• We now have a large pool of example designs for anyone interested in doing the same for their cores
• And finally, we have seen that some of the maintainers whose cores we added support for, have started doing the same on their other cores, which is fantastic to see.

Now, our hope is of course that you too will be bitten by the FuseSoC fever and add support for your cores too so that we can keep growing the ecosystem of FuseSoC-compatible cores, which in turn will help the EDA tool developers improve their tools.

This work was sponsored by a grant from NLNet Foundation

FuseSoC 1.5

Finally! FuseSoC 1.5 is now released. I was just about to release it a few weeks ago when I discovered two quite serious bugs. Then I was just about to release it when I had written a lengthy blog post describing some of the new backends. At that point I realized that it would be better to write those bits as part of the documentation. Yes, you read it right. Documentation. This release has 45% more documentation than previous releases. A massive increase from 245 lines to 368 lines. Oh well. One day it will all be properly documented...I hope. At least the new backends are documented and I will try to give the old ones the same treatment. Promise!

Now for the good stuff...

 New backends

This version adds support for one new simulator, one new FPGA flow and three EDA vendor IP package formats.

The new FPGA flow is for the awesome project IceStorm, which provides a complete open source flow for building FPGA images for Lattice iCE40 devices. The tools for this consists of yosys to handle synthesis, arachne-pnr for place & route and icepack for creating the binary file. Both yosys and arachne-pnr can handle other vendor's devices as well, and I hope to use these tools in other flows as well in the future.

There is support for an open source VHDL simulator called GHDL, which is most likely the open source EDA tool with the best VHDL support.

The new IP package formats being handled are Altera QIP files, Xilinx Coregen files (for ISE) and Xilinx xci files (for Vivado). The last two deserves a few extra words since they use the provider mechanism as a plugin system in a clever way. Providers, in the FuseSoC world, are modules that are normally responsible for downloading a core and put it into the cache. There are providers already for getting cores from git and svn repos as well as from simple URLs. The providers are each responsible for their own options but typically look something like this

[provider]
name = github
user = olofk
repo = wb_bfm
version = v1.0

One FuseSoC user and contributor however realized that the provider doesn't really have to download anything, and can instead be used to do transformations on source code instead. He implemented this to generate HDL code from vendor IP core descriptions and added the new coregen and logicore providers. They each look like this

[provider]
name = coregen
script_file = <coregen ip description>.xco
project_file = <coregen project file>.cgp

[provider]
name = logicore
script_file = <generator script>.tcl
project_file = <vivado ip description>.xci

When a core with one of these providers is requested, the Xilinx IP generation tools for ISE or Vivado are invoked to generate the HDL code, which is then placed in the cache. For the coregen-based cores, the generated HDL files are listed in the .core file as would have been done for any core.

For the Vivado-based logicore IP however there is an extra trick. As Vivado generates an IP-XACT component description for its generated IP cores, we can simply point to that one, and automatically let FuseSoC get all the HDL files for us automatically.

I have been planning for several years to do some kind of plugin mechanism to run custom preprocessing commands on source files, and using the provider mechanism for this solves the problem in most cases. Examples of future providers could be to turn MyHDL, Migen or Chisel code into verilog, or build a verilog top-level from an IP-XACT design file. As this was not originally my idea, I was naturally extremely sceptic of the whole idea. It does have some limitations, but for most cases it gets the job done and is a clever solution for the problem.

Continious integration

Another great addition brought in by a new FuseSoC user is the continious integration testing with Travis and AppVeyor. Every time something is commited to the FuseSoC repository, Travis and Appveyor are triggered to run some basic tests. As an interesting note, it took less than a week after adding the CI support until this found a bug caused by a minor difference between python 2 and python 3. That bug would probably had gone unnoticed for a long time if not for this, so it's worth its money already. Especially so, since both services are free to use.

Windows compatibility

The same user who brought in the CI testing also done some changes to the build system to increase the windows compatibility. It still doesn't run properly on Windows, but it can now at least install, list the cores properly and run a few simulations

Improved parameter handling

There had been several improvements to the handling of command-line parameters for compile-time or run-time configurations. It's now possible to specify a default value for parameters. This can be useful to override default settings in the upstream code, for example to specify a different path to a memory initizalization file than what the upstream project uses. There is also some visual improvements to the output of fusesoc {sim,build} <core> --help to make it clearer what kind of parameter is being changed. Several bugs has been fixed in this area as well, and it should now be possible to use top-level parameters with all tools.

The old [plusarg] sections that were deprecated in favor of the [parameter *] sections are no longer parsed as I have decided that users have had enough time to migrate to the parameter sections. The warning still remains to remind everyone that it's time to move on and the article about FuseSoC 1.4 has more info on what the [parameter] sections should look like.

Improved ModelSim backend

The Modelsim backend has been rewritten, both to support loading user-specified TCL files and to make it easier to rebuild a design from the exported sources. And believe it or not, there is even documentation for the modelsim backend now!

Build system changes

Finally, I also decided to remove the old autotools scripts for installation. For some time it has been possible to install FuseSoC with either python's setuptools (e.g. pip install fusesoc) or autotools (e.g. ./configure && make && make install). With the improvements in the setuptools based system, it makes little sense to keep autotools. I don't want to keep two tools updated, so from now on, FuseSoC will only use setuptools for installations. Farewell autotools. It's not you, it's me

And...

...as always, tons of bugs were killed during the making of this release. A few notable ones are that VHDL and SystemVerilog are now supported properly in ISIM, IP-XACT 2009 and 2014 files are supported, RTL library affiliation is read from IP-XACT logicalName tags and FuseSoC doesn't crash anymore if it encounters an unknown simulator in the simulators list

I'm really excited to start working on FuseSoC 1.6 now. There are already a bunch of new things ready to be commited, that I have been holding off until FuseSoC 1.5 was released. Make sure to check the git repo for daily progress if you want to get all the new things before the next version is released.

Thanks for listening!