Tuesday, June 26, 2018

FuseSoC 1.8.2

FuseSoC 1.9 will have an impact so great that people will divide history into the eras before and after it's release. But before that happens, it's time for a less earth-shattering minor release. Allow me to introduce FuseSoC 1.8.2

Backend separation

The biggest feature is also the least visible for most users. A lot of time since the last release was spent on moving stuff around with the ultimate goal of splitting out the backends (the part that create project files and launches EDA tools) to a separate library. This is now complete and the former backend code now lives in a separate library called edalize, which remains part of FuseSoC for now, but will be moved out at some point. The ideas behind this will be a topic for a separate article at some point, but if you're building some Python tool that wants to use the EDA tool wrappers, look for edalize now.

Linter backends

FuseSoC.. sorry, Edalize has gained a new backend for Synopsys SpyGlass which is a powerful linter tool. For a less costly linter, it is now possible also to use the linter mode of Verilator by setting mode to lint-only in the verilator section of the tool options in a target like this

targets:
  lint:
    tools:
      verilator:
        mode: lint-only

CAPI2 changes

The latest core description format has had some minor improvements. The top-level option was originally specified as a list of modules. since there are cases when several top-modules are needed. This however is quite rare, which makes it confusing that it's just a list of one item in most cases. Therefore it can now also be specified as a string. Small improvement, but makes things less annoying. Environment variables are also expanded in file names.

Other changes

Sometimes you really want to ignore a core that's part of a library. By adding a file called FUSESOC_IGNORE to a directory, FuseSoC will ignore that directory and everything below it. There is also a series of tutorials in the works. These are very rough drafts at this point, but I encourage everyone who wants to get started with FuseSoC to take a look through them anyway. They are located here

That's pretty much it. As usual, a number of bugs were fixed but nothing worth taking up more of your precious time. Thank you for reading. See you again in the post FuseSoC 1.9 era

Thursday, March 22, 2018

FuseSoC 1.8.1

With great features come great bugs. FuseSoC 1.8 had a lot of new functionality but it also has some shortcomings. Some of it was intentional. For both the library support and CAPI2 I wanted to push out an early version with the most important functionality, test the waters and add the missing pieces later. (Un)fortunately both these features turned out to be really really good and I came to depend on them immediately. This made it more important to make sure they were useful as I don't want to use the old ways anymore. After some post-release patching I'd say things are in much better shape, so it made sense to do a new release with these things fixed. Therefore I present FuseSoC 1.8.1. It's like 1.8, just a little better.

What's changed then?

Library fixes

The library sections in fusesoc.conf has gained a sync-type flag. This allows users to explictly set what mechanism to use for synchronizing libraries against a remote source (e.g. updating). The only two current values are git, which does a git pull of the library when running fusesoc update, and local, which won't try to synchronize at all. The library sync-type options share code with the core providers, so it should be easy to add other synchronization mechanisms (e.g. rsync, webdav, mercurial) when needed. When adding a new library there is also now a --global flag. If this is present, the library will be added to the user-global fusesoc.conf in $XDG_CONFIG_HOME/fusesoc. Otherwise, a fusesoc.conf in the current workspace directory will be used.

CAPI2 fixes

CAPI2 debuted with support for just a few tools. This was a trade-off to be able to do a quick release. Of course, it soon became apparent that the other tools were needed as well. One other thing that bothered me with CAPI2 was that the tools sections were a hard-coded list in the core class. So to fix both these things at the same  time I did some refactoring and let the backends themselves declare which options they use. FuseSoC will then search for available backends at run time and pick up their available tool options. VoilĂ , all tools are now supported in CAPI2. This also opens up for more easily adding new tool backends without having to change anything else in the code base.

There are more things missing in CAPI2, but the one that I realized was needed most was the equivalent of the scripts section in CAPI1. I took the opportunity to rearrange things slightly so that I could make this more powerful than in CAPI1. Instead of a list of shell scripts to run, it is now possible to launch arbitrary commands and associate extra files to be exported to the work root and environment variables to be set.

Other fixes

Other noteworthy things that are added to FuseSoC 1.8.1 is a bash completion script that was contributed almost four years ago but never merged. This was polished up by another contributor and is now available in the extras directory of the FuseSoC repo. It is not installed by default, but has to be copied into the appropriate directory (e.g. /etc/bash_completion.d on my system).

Finally, FuseSoC no longer writes a fusesoc.log by default. Writing this logfile on every run has annoyed me for almost six years now, but I never got around to fix it. Thankfully there are other contributors who get things done.

And that's all. Hope you like it!

Monday, February 5, 2018

FuseSoC 1.8

Too tired to write something clever here. Just get it over with. Stuff has changed. Here's what's new

CAPI2

The main feature of FuseSoC 1.8 is the support for a new core description format, CAPI2, to replace the old CAPI1 format. The need for a new format to solve support some use cases has been apparent for a while. Doing some archaeology I found a commit from March 2, 2013 that mentions the next generation format, and yes, after careful evalutation of a few different formats it ended up being based on YAML. Oh well, things take time and there have been other priorities. At least it's here now, or rather, a first experimental version. The CAPI2 support in FuseSoC 1.8 still has many missing features, like documentation and support for IP-XACT and all tools to name a few. There might also be non-compatible changes ahead, but I want to push it out now together with some CAPI2 cores to reach a wider audience.

So what's so good about CAPI2? I'm hoping to find the time to write a separate article with more details some time in the future, but I have written down some of the benefits.

New tool flows

Support for other tool types than simulators and synthesis tools. This includes things like formal verification tools and linters. For linting, I have already made a quick PoC using Verilator in lint mode and there is a Spyglass backend in the works. On the formal verification side, I will start to look into what's need to add support for Yosys-SMTBMC  

Multiple targets

With the support for new flows for a design there is also a need to decide which flow to use. For this, CAPI2 supports multiple targets in a single core file. There are many uses for different targets. A design might have several testbenches that exercises different parts of the design.  These can be specified as different targets using different source files, run time parameters, tool options etc. Some designs have support for targeting multiple FPGA boards. These can also be described as separate targets, with the benefit that they can share all the common parts while freely add target-specific files and options. Targets for linting or formal verification can be separate targets as well.

Conditional expressions

CAPI2 contains a minimal expression parser inspired by the Gentoo ebuild format. An expression can look something like this


flag? (use_if_flag_is_true)
!flag? (use_if_flag_is_false)

This allows including or excluding features depending on which tool is being used or whether the core is at the root of the dependency hierarchy or being used as a dependency for another core. Examples of this can be to enable technology-specific RTL primitives for certain tools, switch between VPI or DPI extensions depending on tool support or use a C++ testbench for verilator while other simulators use a non-synthesisable verilog version. At the time of writing, these flags are defined internally in FuseSoC depending on current tool, target and some other parameters, but it will be possible in the near future for users to define their own flags and set them in the top-level core file or on the command-line.

 Core libraries

FuseSoC already has the ability to define directory paths which contain core files, either by specifying them with the cores_root option in fusesoc.conf, on the command-line with --cores-root= or through the FUSESOC_CORES environment variable. This is all good, but a bit limited. With FuseSoC 1.8 comes support for core libraries. Each library is specified in fusesoc.conf as a separate section. An equivalent of an old config file looking like this


[main]
cores_root = /home/user/.local/share/fusesoc/orpsoc-cores /home/user/cores/fusesoc-cores


will now look like this


[library.orpsoc-cores]

[library.fusesoc-cores]
location = /home/user/cores/fusesoc-cores

So far there isn't much of an improvement over the old style. One thing to notice is that if the location option is missing, it will default to $XDG_DATA_HOME/fusesoc/<library name>

Having separate sections however also allow us to do the following


[library.orpsoc-cores]
sync-uri = https://github.com/openrisc/orpsoc-cores

[library.fusesoc-cores]
location = /home/user/cores/fusesoc-cores
sync-uri = https://github.com/fusesoc/fusesoc-cores
auto-sync = false

sync-uri lets us to specify the remote source of a library. The initial supported remote location are git repositories, but other source such as mercurial, rsync, webdav etc will be support with a sync-type option in the future. The auto-sync option, which defaults to true, controls if the library shall be updated when running fusesoc update.The update command can also take a list of libraries as extra argument to selectively update libraries.

To support library management from the command-line, FuseSoC has now gained a new library subcommand. The only supported library command is currently add, to register a new library, but additional commands for listing and removing libraries will be added in time. To add a remote library, the following commands can be used


fusesoc add library enigma https://github.com/mmicko/enigmaFPGA

The resulting fusesoc.conf will look like this


[library.orpsoc-cores]
sync-uri = https://github.com/openrisc/orpsoc-cores

[library.fusesoc-cores]
location = /home/user/cores/fusesoc-cores
sync-uri = https://github.com/fusesoc/fusesoc-cores
auto-sync = false

[library.enigma]
sync-uri = https://github.com/mmicko/enigmaFPGA 


There are also optional arguments --location and --no-auto-sync to explicitly set the location and auto-sync = false

For adding a directory path as a library, the same command is used, but the sync-uri is treated as a location if FuseSoC detects that it's an existing directory


$ fusesoc library add more_cores ~/other_cores
INFO: Detecting /home/user/other_cores as a local library

will produce


[library.orpsoc-cores]
sync-uri = https://github.com/openrisc/orpsoc-cores

[library.fusesoc-cores]
location = /home/user/cores/fusesoc-cores
sync-uri = https://github.com/fusesoc/fusesoc-cores
auto-sync = false

[library.enigma]
sync-uri = https://github.com/mmicko/enigmaFPGA 

[library.more_cores]
location = /home/user/other_cores

There are more planned featues for core libraries, but this will have to do for today.

Backend features


Most of the tool backends have received fixes or new features. All tools now supports file type = user for passing arbitrary files to the build tree that might be neeeded by the tools. For Xilinx ISE, FuseSoC now supports BMM files as an allowed file type and quoting of verilog string parameters should now work. Isim will shut down properly after simulations, GHDL will receive its arguments in the correct order and XSim supports multiple top-levels. The IceStorm backend now supports verilog defines specified as command-line parameters, has gotten a yosys_synth_options parameters to set extra synthesis options and PCF constraint files are recognized as a file type

Other things

Other things worth mentioning are the new --no-export feature. Before an EDA tool is invoked, FuseSoC has always created a fresh build tree where it places all files and runs the EDA tools to avoid polluting the source directories with files and to not pick up unwanted files that might be lying around in the source trees. This also has the added benefit that the exported build tree is not dependent on FuseSoC and can be archived or sent to non-FuseSoC users if required. I have used that myself to make deliveries of projects to clients. There are however cases where this is not optimal, for example during debugging, when there is a need to make small changes to a file in the source tree without having to recreate the build tree every time. For such use-cases FuseSoC now supports a --no-export option that will reference the original source files instead of copying them to the build tree. For most cases, this should work just as fine, but beware that it might break in some situations, e.g. when multiple cores has an include file with the same name.

Another improvement related to coping files around is a new copyto attribute for files in filesets. It's used like this:


[fileset fs]
files = an/old/file.v[copyto=a/new/name.v]

or with CAPI2, like this

filesets:
  fs:
    files:
      - an/old/file.v: {copyto : a/new/name.v}

This will copy the file to a new location, relative to the work root of the EDA tool and reference this version in the EDA API file. This is quite useful in several situations, for example when a file is required to exist in the working directory. Previously this had to be done with a pre_build_script to copy the file, which also had the disadvantage of being platform-specific. It also makes it less awkward to reference files from the source directory in verilog parameters. For example the top-level of the de0_nano system in orpsoc-cores contain bootrom_file = "../src/de0_nano_0/sw/spi_uimage_loader.vh", which a) assumes source code tree is located in ../src and b) has to be updated on version bumps when the de0_nano_0 name will change. This can now be replaced with bootrom_file = spi_uimage_loader.vh and a copyto = spi_uimage_loader.vh in the .core file. Much better!
 
EDA API - the tool-agnostic file format which is used to pass all information needed by the backends such as the list of files to read and tool options - has seen some changes and will continue to do so in the future. For this reason, it has gained a version field so that external projects which want to read EDA API files will know what to expect in the file. Properly formalizing the EDA API is still a task at hand, but is getting more prioritized now as there are external projects interested in using this format so that they can reuse either the FuseSoC core files and dependency handler or the EDA tool wrappers. This means it's pretty rude to make arbitrary changes to the format without at least informing about it.

There are also some new commands and switches for the FuseSoC 1.8 release. Before CAPI2 you would either build an FPGA image or sim a simulation. Wiht the sight set on new tool types, none of these commands really describe what you do when you run an linter. So there is now a command called run. Going forward, this will be the main command to use when launching EDA tools through FuseSoC. The run command currently has switches for setting the desired target and tool, and can run any of the tree stages setup, build and run. Without any stages being explicitly set it will run them all. The tool can also be derived from the core description in many cases. build and sim will remain as special cases of run for the time being. sim will be the equivalent of run --target=sim and build will evaluate to run --target=sim --setup --build. The build command has also gained --tool and --target switches for overriding the default options. As for new switches, it is now possible to explicitly point out a FuseSoC config file to use with the --config option

That's all folks. Hope you like it. Please try it out and send me patches, bug reports or perfumed letters if you find anything you like or want to see improved.

See you later, Verilator
In a while, Synplify