Saturday, December 24, 2016

FuseSoC 1.6

Ok, so everything is done. I've done enough testing to make sure the new features work as expected and none of the old stuff is broken. The sources are just waiting to be tagged and uploaded to pypi. The blog post about the new release is written, and all I need is a catchy introduction. Oh, I know. It's christmas, so I should write something funny about FuseSoC being the best gift this year. Or wait, I take a christmas song and change some words so that it's about FuseSoC instead. Or maybe not. It's not really funny, and I suspect someone has been making similar jokes for at least 2016 years. I need to do something else. You know what? I can't hold off this release another day just because I couldn't come up with a catchy introduction. No one cares anyway. Let's just get this over with. FuseSoC 1.6 is released and it's really good. Keep on reading to learn about the new features.

New dependency manager

One of the major uses of FuseSoC is as a package manager. Unfortunately it has always been quite bad at this task. Perhaps the biggest issue has been that unlike most other package managers, FuseSoC only supported exact matches of dependencies. Together with the complete lack of namespaces in Verilog, and to some extent VHDL, this could make package conflicts a nightmare in some cases

An example: A common core in the FuseSoC standard core library is vlog_tb_utils. At some point I wanted to add some more features to vlog_tb_utils, and release it as vlog_tb_utils-1.0. If a core wanted to use the updated version of vlog_tb_utils, it would not work unless all other cores also updated their dependency lists to use this new version as well. That's because the dependency chain would pull in both vlog_tb_utils and vlog_tb_utils-1.0, which contained conflicting files. What a nightmare. What is changed now is that listing a dependency on a core will now pull in the latest version of the core, so all dependencies would now get vlog_tb_utils-1.0. There are of course ways to force an older version, by instead depending on <=name_of_core or =name_of_core, which can be handy when a core has changed in an incompatible way. As most of the available cores that has already been written don't have an explicit version, FuseSoC will create a version according to the core naming rules. The version format is heavily inspired by Gentoo's portage package manager.

Writing a package manager is complicated, and since I really don't enjoy doing difficult things, I decided to look around and see what was already available. Most of the modules I found for package managment were too simple to be of any use, but the simplesat solver from Enthought turned out to meet my requirements. Mind you, it's not a perfect solution, as it is built for another use-case, so I had had to shoe-horn the code to fit into FuseSoC, but it seems to work fine, and has been running on the master branch of the git repo since this summer. The code was actually mostly finished before the FuseSoC 1.5 release, but I wanted to put out a release before adding any potentially backwards-incompatible changes. It should however be completely backwards-compatible, except for a few corner cases, where I could make changes to the core files to handle both the old and new systems.

The work on the new dependency manager also brought with it some other related changes. All cores are now also fully identified by a VLNV identifier. VLNV comes from the IP-XACT standard and means Vendor Library Name Version. All new cores are recommended to use the VLNV naming scheme, but old cores will be mapped into a VLNV name. The documentation goes through the core naming rules in more detail.

As mentioned above, the changes to FuseSoC are backwards-compatible, but older versions of FuseSoC will not be able to handle the new dependency and VLNV syntax. For this, and other reasons, I decided it was time to freeze development on the standard core library called orpsoc-cores. Instead, a new library is created here , which will supersede the old library. As I did not have time to finish the migration, and also because I'm not sure I want to bring all old cores to the new library, these two standard libraries will both be used for some time ahead. If you're doing a fresh install of FuseSoC and run fusesoc init, both libraries will be installed and added to fusesoc.conf. For existing installations, I recommend cloning the fusesoc-cores repo and add it after orpsoc-cores in your fusesoc.conf. By putting the fusesoc-cores entry after orpsoc-cores, it guarantees that all cores which exist in both libraries will use the one from fusesoc-cores. If you're unsure which one was picked up, just run fusesoc core-info on a particular core and check the Core root

Another change caused by the new dependencies is that Python 3.2 and 3.3 are no longer supported. Python 2.7, 3.4 and 3.5 will still work just fine.

Documentation has always been one of the weakest points of FuseSoC. About a month ago, I made a promise to not add any new features until I had written documentation. This of course, I did not manage to completely fulfil, but at least there is a much improved documentation available. The new documentation consists of a tutorial on how to get started with writing core files, how core names are parsed, how to deal with core libraries, a quick-start guide for running simulations, specific information for some of the backends and a migration guide for updating old core files to the current best practices.

There is still a lot to be written, but at least it's moving in the right direction. In total, there is a 148% increase in lines of documentation. Let's see if I can match that for the next release.

I also noticed that renaming the asciidoc sources to .adoc allowed github to understand that this is asciidoc, and renders it automatically, which has the great effect that the online documentation now looks good in a browser without having to convert it to html first.

New backends

This version has support for two new EDA tools

Xilinx Vivado replaced ISE for some time now, and this release finally adds support for the new tool. The code was written by fellow FOSSi Foundation director Stefan Wallentowitz, and was originally planned for FuseSoC 1.5, but because of some major refactoring, I decided to hold it off until this release.

On the simulation side, there is now also support for Aldec Riviera Pro, which is a commercial simulator, quite similar in scope to ModelSim, that seems to be gaining popularity.

Death to .system files

When I started out working with FuseSoC I had an idea of a clear separation between running simulations on cores and creating FPGA images out of systems. This led to the .core and .system files. Over the years, the role of the .system file has been less clear. It had some overlap with the .core files, was not properly documented, added some extra code paths and required an extra file. So from now on, the .system file is made optional, and it is recommended to move the settings from there into the .core files instead as stated in the migration guide


There are also many smaller but oh so important changes

FuseSoC will now pick up additional core libraries from the FUSESOC_CORES environment variable, and all core library locations can be listed with fusesoc list-paths

Both simulation and build backends have gained a --setup flag that will only export the needed files and setup an initial project file without running the EDA tools. This is very useful both for running the tools in GUI mode, or to export a build tree that can be released as a stand-alone archive for people who don't use FuseSoC.

It is now also possible to set verilog `define options on the command-line through the parameter interface. In the core file, it would look something like this

[parameter size]
datatype = int
description = Sets the size of something
paramtype = vlogdefine

and would be called with fusesoc sim some_core --size=10

I've had the opportunity to run some testing in a Windows environment which revealed a few flaws that has now been fixed. These were mostly related to the fact that even though Windows uses \ as the path separator (insane!), most EDA tools still required using / (sane!) even on Windows. Both the ModelSim and Quartus backends have been updated to reflect this, but there is a good chance the other tools need the same treatment. To make it easier to set up on windows platforms, there is also a change to use git-apply for patching cores instead of using the patch command. As FuseSoC already depends on git for setting up some things, it removes the number of external Linux-specific dependencies.

Speaking of ModelSim, there has been some improvements to make it easier to run in GUI mode and batch mode if FuseSoC is run with --setup or --build-only. FuseSoC now also supports multiple top-level modules, which is something that is often required for running with the Xilinx simulation libraries

There are probably more things that I forgot about, but they are likely not that important. So for now, please upgrade to FuseSoC 1.6 and try out the new stuff. I'm already hacking on some new things for the next version.

Have fun!