pcb-rnd knowledge pool
Import_sch rewrite, switching over to tEDAx netlists
| import_sch by Tibor 'Igor2' Palinkas on 2018-08-17 | Tags: roadmap, tEDAx, import, schematics, install, import_sch, gsch2pcb, gsch2pcb-rnd, gschem, gnetlist, lepton, lepton-eda |
Abstract: Majority of pcb-rnd users use the gschem->pcb-rnd flow, either with import schematics, or less frequently with gsch2pcb-rnd. The current implementation is fragile. This article describes how it is going to be fixed using tEDAx.
1. The problem
Both of gschem and lepton use a scheme script with gnetlist to get an action script that loads the footprints and fills in the netlist. This sort of worked, but recently lepton-eda had API changes that broke this script. But the real root cause of the problem is that having that scheme script in the first place was bad design:
- we either have the scheme script in pcb-rnd's repo (current situation) and then it needs to know a lot about internals of a 3rd party software, gnetlist API (of which we have two: one in gschem, one in lepton); this will break from time to time, either with new versions of gschem or lepton, or if we update the script, it will break with old version of gschem/lepton
- or we put the scheme script next to gnetlist, a copy in gschem, another in lepton (the pcb forward annotation script is like that); this is the same mistake, just the other way around: a gnetlist scheme script that needs to know a lot about pcb or pcb-rnd internals (the actions API).
The common pattern is that we have a glue layer, which needs to know some internals of two (or four) projects, so wherever it lives, it will be easy to update for API changes of one half of the projects, but will easily get out-of-sync with the other half of the projects. Thus it will break from time to time if the 3rd party software changes.
2. The solution - theory
When I first encountered this design flaw, I concluded the only resolution is to define a transport format (protocol) that does not depend on the internals of any of the software, but is independent and is task-oriented instead of implementation/API oriented (not depending on pcb-rnd actions or gnetlist scheme API). My proposal for the format is tEDAx netlist 1 , designed to be cheap to implement. Using such a format solves the problem by allowing us having a simple netlist export script within gschem and another for lepton; these scripts can be tied together with gnetlist/xorn API, and their output is generic, non-pcb-rnd-specific tEDAx. In other words, the script depends on the internals of only one half of the projects, not both, so it won't bitrot if that other half changes.
All we need to have for this is: support for tEDAx netlist in gschem/xorn, then in lepton, import on pcb-rnd's side and a rewrite of import_sch/gsch2pcb-rnd.
3. The solution - practice
- pcb-rnd tEDAx netlist import is available for more than 1.5 years by now; it's stable, production code
- new gschem has a xorn plugin for exporting a tEDAx netlist block; it is available only in the git version and needs extra manual tweaking on the install 2 , but it is available
- old gschem will require a gnetlist backend script to be manually installed 3
- new lepton-eda (git master) has the tEDAx exporter written by Bdale; it will probably feature it in the next release; the lepton-eda Debian package being prepared by Bdale will probably have it too
- old lepton-eda will require a gnetlist backend manually installed^3
- after releasing pcb-rnd 2.0.1, I will rewrite the import_sch plugin from scratch to work better with different transport formats (see below); in short, this will help you get a one-click forward annotation with tEDAx (or any other format)
- automating the process ("gsch2pcb-rnd") will change a bit long term (see below)
4. upcoming import_sch rewrite
The changes on this side won't really affect the end user for existing designs, but will affect the user for new designs.
Import_sch does a few rather unrelated things:
- remember sch names
- remembers whether the result is to be generated running gnetlist directly or a Makefile
- can run gnetlist (or a Makefile) and pick up the resulting action script
- can load and execute the resulting action script
The general approach is correct, but the implementation is pretty much tied to the assumption that the netlister will emit a pcb-rnd action script, which forces committing the mistake described above in chapter 1.
To avoid that, import_sch will be rewritten as a more generic implementation, a superset of the original implementation. As a superset, it will still support the old mechanism. The rewrite will do the same points, just with slight differences to avoid the bad assumptions:
- remember flow-specific data (whether it's sch names or anything else)
- remember how to get the netlist generated; but the actual commands for this should be in format/schematics-editor specific plugins, not in import_sch
- execute the necessary actions to (get other plugins to) generate the netlist
- execute the necessary actions to load the resulting netlist, whatever format it is in
5. upcoming gsch2pcb-rnd deprecation
By now gsch2pcb-rnd does the same thing as import schematics 4 , except it does not automatically execute the resulting action script but asks the user to do so. The very same thing can be achieved without using gsch2pcb-rnd, just running gnetlist and then loading the resulting cmd file.
Or by running pcb-rnd in batch mode to automate the 'loading of cmd file' step.
By switching from gsch2pcb-rnd to running pcb-rnd in batch mode, the user gains more control over the process, being able to run more actions if needed, being able to use other transport formats (importing from different schematics editors).
Thus gsch2pcb-rnd will be removed long term. For gsch2pcb-rnd users the upgrade path should be:
- preferrably switch to the tEDAx transport as described in point 3.; run gnetlist/xorn instead of gsch2pcb-rnd and import the tEDAx netlist (using an action or the menu) instead of executing a cmd file. Other than "typing different commands" or "clicking another menu", nothing really changes.
- much less stable way: run gnetlist directly, without gsch2pcb-rnd, with the pcb-rnd forward annotation scm script, to get the same cmd file gsch2pcb-rnd generates. This will not work in lepton since a few releases and has a chance to break with future gschem releases as well. (No, keeping gsch2pcb-rnd wouldn't fix these problems, as it is just running the same scm script using the same gnetlist).
Footnotes and references:
1: tEDAx spec available at http://repo.hu/projects/tedax
2: Alain's gaf tedax howto
3: contributed by Bdale, available from svn://repo.hu/tedax/trunk/src/reference/geda