Showing posts with label VHDL. Show all posts
Showing posts with label VHDL. Show all posts

Wednesday, February 3, 2016

FuseSoC 1.4

When I consider the magnitude of the subject which I am to bring before my readers-a subject, in which the interests, not of this country, nor of Europe alone, but of the whole world, and of posterity, are involved: and when I think, at the same time, on the weakness of the advocate who has undertaken this great cause-when these reflections press upon my mind, it is impossible for me not to feel both terrified and concerned at my own inadequacy to such a task. But when I reflect, however, on the encouragement which I have had, through the whole course of a long and laborious examination of this question, and how much candour I have experienced, and how conviction has increased within my own mind, in proportion as I have advanced in my labours;-when I reflect, especially, that however averse any gentleman may now be, yet we shall all be of one opinion in the end;-when I turn myself to these thoughts, I take courage-I determine to forget all my other fears, and I march forward with a firmer step in the full assurance that my cause will bear me out, and that I shall be able to justify my decision to release FuseSoC 1.4.

William Wilberforce, a man who apparently had a thing for extremely long sentences, might have said something a bit similar about the end of slavery, but that's not the topic for today. Today's topic is the new FuseSoC release. We now live in a world of git snapshots (and less slavery than 1789), and I don't expect an increase in the number after the decimal point in the configure script matter to most people. Still, I think it's nice to do a stable release once in a while. It helps packagers (where are you by the way? I'm still waiting for someone to make a FuseSoC rpm or deb), and more importantly, I can take this as an opportunity to break things. This will be the first version where a part of CAPI1 gets deprecated. There are some other noteworthy changes since FuseSoC 1.3, and although I have summarized the changes in the NEWS file, and I recently wrote a post about the new IP-Xact support there are other changes that could be worth explaining in further detail. I do realize that this would not be needed if someone had written proper documentation (where are you by the way? I'm still waiting for someone to write the documentation for me), but no one has, so this will have to do for now.

Prominent features

 

 File sets

Up until now, source code has been mainly verilog, with some support for C(++) for Verilator testbenches and VPI modules, together with a very limited support for VHDL in the Quartus backend flow. This part has been completely overhauled in this release, where filesets are introduced. Filesets is modeled on the IP-Xact filesets, and these will have a stronger connection in the future, as IP-Xact support is also introduced in this version. The verilog section will be deprecated at some point, so I suggest moving to filesets already. Here's a quick example on how to make the switch. The verilog section of the de0 nano system...

[verilog]
src_files =
 rtl/verilog/clkgen.v
 rtl/verilog/orpsoc_top.v
 backend/rtl/verilog/pll.v
 rtl/verilog/wb_intercon.v
 rtl/verilog/wb_intercon_dbg.v

tb_private_src_files =
 bench/orpsoc_tb.v
 bench/uart_decoder.v
include_files =
 rtl/verilog/include/or1200_defines.v
 rtl/verilog/include/orpsoc-defines.v
 rtl/verilog/wb_intercon.vh
 rtl/verilog/wb_intercon_dbg.vh
 sw/clear_r3_and_jump_to_0x100.vh
 sw/spi_uimage_loader.vh

tb_include_files =
 bench/spi_image.vh
 bench/test-defines.v



...becomes...

[fileset rtl_files]
files =
 rtl/verilog/clkgen.v
 rtl/verilog/orpsoc_top.v
 backend/rtl/verilog/pll.v
 rtl/verilog/wb_intercon.v
 rtl/verilog/wb_intercon_dbg.v
file_type = verilogSource
usage = sim synth

[fileset tb_files]
files =
 bench/orpsoc_tb.v
 bench/uart_decoder.v
file_type = verilogSource
usage = sim

[fileset include_files]
files =
 rtl/verilog/include/or1200_defines.v
 rtl/verilog/include/orpsoc-defines.v
 rtl/verilog/wb_intercon.vh
 rtl/verilog/wb_intercon_dbg.vh
 sw/clear_r3_and_jump_to_0x100.vh
 sw/spi_uimage_loader.vh
file_type = verilogSource
is_include_file = true
usage = sim synth

[fileset tb_include_files]
files =
 bench/spi_image.vh
 bench/test-defines.v
file_type = verilogSource
is_include_file = true
usage = sim


It's a few more lines, but the added flexibility outweighs the few extra bytes of ascii characters. It's now possible to set other file types, such as vhdlSource, or request specific language versions such as verilogSource-2001. The available file types are enumerated in the IP-Xact standard with some addtional ones allowed by FuseSoC (verilogSource-2005 and vhdlSource-2008 for now). The name of a fileset is not important, but the filesets are parsed in the order they appear in the .core file, so make sure that they are correctly ordered when dealing with languages where this is important.

The usage tag can be set to one or more items, and determines which tools that will use the fileset. These can be either a category (sim, synth) or a specific tool (verilator, quartus, modelsim...)

For languages that has a concept of libraries (VHDL, not verilog), the logical_name tag can be used to indicate library affiliation.


Another new feature is per-file attributes, which can be used to override the default value for the fileset. These attributes are placed inside square brackets at the end of the file. They are comma-separated and are either of the form attribute=value, or just attribute, to set boolean attributes. With per-file attributes, the above example can be changed to:

[fileset rtl_files]
files =
 rtl/verilog/clkgen.v
 rtl/verilog/orpsoc_top.v
 backend/rtl/verilog/pll.v
 rtl/verilog/wb_intercon.v
 rtl/verilog/wb_intercon_dbg.v
 rtl/verilog/include/or1200_defines.v[is_include_file]
 rtl/verilog/include/orpsoc-defines.v[is_include_file]
 rtl/verilog/wb_intercon.vh[is_include_file]
 rtl/verilog/wb_intercon_dbg.vh[is_include_file]
 sw/clear_r3_and_jump_to_0x100.vh[is_include_file]
 sw/spi_uimage_loader.vh[is_include_file]
file_type = verilogSource
usage = sim synth

[fileset tb_files]
files =
 bench/orpsoc_tb.v
 bench/uart_decoder.v
 bench/spi_image.vh[is_include_file,file_type=verilogSource-2005]
 bench/test-defines.v[is_include_file]
file_type = verilogSource
usage = sim



Note that the file_type attribute for spi_image.vh was only added to show how multiple attributes can be set.

If there is an IP-Xact component file for the core, FuseSoC can parse that for the filesets instead. In that case, the above example will look like this

[main]
component =
 de0_nano.xml

Yep, that's right. We can leave it all to IP-Xact. Except for when we need file types that aren't in the IP-Xact standard (again Verilog 2005 and VHDL 2008 are the primary cases here), and the usage will default to both sim and synth. Future versions of FuseSoC will allow overriding options in the IP-Xact files to handle these cases. The IP-Xact support is described in greater detail in an earlier post


Mixed-language support


With the new file set features in place, most of the simulators and synthesis tools now accept both verilog, systemVerilog and VHDL files simultaneously for mixed-language projects.

 

Compile-time parameter enhancements


Both build backends now support setting top-level verilog parameters at synthesis-time. This can be used for example to initialize a boot ROM with different contents, or setting a version tag at compile-time. Using the de0 nano example once again, the following can be added to the .core file to register a parameter

[parameter bootrom_file]
datatype    = file
description = Boot ROM contents in Verilog hex format
paramtype   = vlogparam
scope       = public


To select a different boot ROM, run fusesoc build de0_nano --bootrom_file=/path/to/bootrom.vh

Slightly related are some enhancements to the simulator backends. Both verilator and XSIM now accepts parameters which are turned into plusargs, and all simulator backends now analyse the CLI arguments before building the simulator model to save some time.

Distutils and pypi


Apparently all the cool kids put their Python code on pypi, and it was suggested that I should do the same with FuseSoC. The easy way to do that is to replace autotools with a Python-based build system called distutils...or setuptools... which at some point gets used by pip... to create an egg or a wheel... which can be installed with pip.... or easy_install. I'm really not sure. Python packaging is a complete mess with around ten competing build systems. Try to Google it if you don't believe me. It's insane. I think this tweet pretty much sums it up.

So what are the benefits of this new Python-based build system. A small reduction in line count of the configuration files and the possibility to upload to pypi. It doesn't handle dependencies unfortunately, and there is no way to uninstall a package. How I love progress! The autotools-based build system is still there, until I figure out what to do.

That's all for now. I have already plenty of things lined up for the FuseSoC 1.5 release


Saturday, December 19, 2015

FuseSoC and VUnit

I recently improved the VHDL support in FuseSoC and since I've been using vunit a bit lately I thought it could be a fun experiment to see if I could combine the strengths of these projects.

For those not aware, FuseSoC is a package manager and build system for FPGA/ASIC systems. It handles dependencies between IP cores and has functionality to build FPGA images or run testbenches a single core or a complete SoC. Each core has a .core file to describe which source files it contains, which dependencies it has on other cores and lots of other things. The aim of FuseSoC is to make it easier to reuse components and create SoCs that can target different FPGA vendors and technologies. FuseSoC is Open Souce, written in Python and can be found at https://github.com/olofk/fusesoc

VUnit is an open source unit testing framework for VHDL released under the terms of Mozilla Public License, v. 2.0. It features the functionality needed to realize continuous and automated testing of your VHDL code. VUnit doesn't replace but rather complements traditional testing methodologies by supporting a "test early and often" approach through automation.

VUnit has a lot of great functionality for writing unit tests in VHDL, but requires the users to set up the source tree with all dependecies and their libraries themselves

FuseSoC on the other hand has knowledge of each cores files and dependencies, but very little convenience functions for writing unit tests. The ones that are existing are mainly targeting verilog users.

Given these preconditions, my idea was to let FuseSoC collect all source code and give it to VUnit to run unit tests on them.

Let's get started

VUnit requires the user to write a small Python script that sets up simulation settings, collects all source files, put them into libraries and starts an external simulator. This script is then launched from the command line with options to decide which unit tests to run, which simulator to use and where the output should go among other things. Here's the example script from VUnit's user guide
 
# file: run.py
from vunit import VUnit

# Create VUnit instance by parsing command line arguments
vu = VUnit.from_argv()

# Create library 'lib'
lib = vu.add_library("lib")

# Add all files ending in .vhd in current working directory
lib.add_source_files("*.vhd")

# Run vunit function
vu.main()

FuseSoC is also launched from the command line and expects to be instructed which core or system to use, if it should do synthesis+P&R or run a simulator and other options. FuseSoC however was always meant to be used as a library as well as a command-line tool, so to make these work together, we create a VUnit run script that imports the necessary functions from FuseSoC. Thankfully both tools are written in Python, or I would have given up at this point.

The first inconvenient difference between FuseSoC and VUnit is that the vunit run script need all source files that should be compiled for any testbench to run. FuseSoC on the other hand don't know which source files to use until we tell it which core to use as it's top-level core. To work around this I decided to look at the VUnit's -o option, which is used to tell VUnit which output directory to use. I simply peek at the output directory and use that as the FuseSoC top-level core name. We now got the first lines of the new script.

from vunit import VUnit
 
vu = VUnit.from_argv()
 
top_core = os.path.basename(vu._output_path)

Now we need to do some basic FuseSoC initialization. First we create a core manager, which is a singleton instance that is handling the database of cores and their dependencies.

from fusesoc.coremanager import CoreManager, DependencyError
cm = CoreManager()

Next step is to register a core library in the core manager. Normally FuseSoC picks up locations of core libraries from the fusesoc.conf file, which can be in the current directory or ~/.config/fusesoc, or from the --cores-root=/path/to/library command-line options.

We don't have any command-line options for this, but we can get fusesoc.conf by using the Config() singleton class.

from fusesoc.config import Config
config = Config()
cm.add_cores_root(config.cores_root)

We can also add any known directories directly with

cm.add_cores_root("/path/to/corelibrary")

The core manager will scan the added directories recursively and pick up any FuseSoC .core files it finds. (Note: If a .core file is found in a directory, its subdirectories will not be searched for other .core files).

It's now time to sort out the dependency chain of the top-level core we requested earlier

try:
    cores = cm.get_depends(top_core)
except DependencyError as e:
    print("'{}' or any of its dependencies requires '{}', but this core was not found".format(top_core, e.value))
    exit(1)

If a dependency is missing, we tell the user and exit. If all was well, we now have a sorted list of cores in the 'cores' variable. Each element is a FuseSoC Core class that contains all necessary information about the core.

With all the cores found, we can now start iterating over them in order to get all the source files and other information we need and hand it over to VUnit. Some notes to the code below:

1. 'usage' is a list of tags to look for in the filesets. Only look at filesets where any of these tags are present. FuseSoC itself looks for the names 'sim' and 'synth' to indicate if the files should be used for simulation and synthesis. We can also choose to only use a fileset with a certain tool by for example setting the tag 'modelsim' or 'icarus' instead of 'sim'
2. File sets in FuseSoC can be public or private. Default is public and indicates that other cores might find the files in there useful. This is for example the files for synthesis and testbench helper functions such as BFMs or bus monitors. Private filesets are used for things like core-specifc testbenches and target-specific top-level files.
3. Even though the most common way to use libraries is to have one library for each core, there's nothing stopping us from splitting a library into several FuseSoC cores, or let one core contain multiple libraries.

usage = ['sim']
libs = OrderedDict()
for core_name in cores:
    core = cm.get_core(core_name)
    core.setup()
    basepath = core.files_root
    for fs in core.file_sets:
        if (set(fs.usage) & set(usage)) and ((core_name == top_core) or not fs.private):
            for file in fs.file:
                if file.is_include_file:
                    #TODO: incdirs not used right now
                    incdirs.add(os.path.join(basepath, os.path.dirname(file.name)))
                else:
                    try:
                        vu.library(file.logical_name)
                    except KeyError:
                        vu.add_library(file.logical_name)
                    vu.add_source_file(os.path.join(basepath, file.name), file.logical_name)


With that we are done, and can safely leave it to VUnit to do the rest

vu.main()

It all works fine, and to make it more interactive, I set up a demo project at https://github.com/olofk/fusesoc_vunit_demo. This contains a packet generator for a made-up frame format together with a testbench. The packet generator has dependencies on two utility libraries (libstorage and libaxis) that I wrote a while ago to handle some common tasks that I got fed up with rewriting everytime I started a new project. The packet generator test bench uses some VUnit functions to make the example a bit more educational.


I hope this will be useful for all people using, or thinking of using, VUnit. For all you others, you can still use FuseSoC without VUnit for simulating and building systems and cores. The FuseSoC standard core library that can be downloaded as part of the FuseSoC installation contains about 60 cores, and there are several more core libraries on the internet that can be combined with this.

Happy hacking!