pcb-rnd knowledge pool
Long term plans: export option cleanup (summary)
| export_cleanup by Tibor 'Igor2' Palinkas on 2019-08-15 | Tags: roadmap, pcb-rnd, 3.0.0, 3.0, hidlib, rndlib, ringdove, export, png, ps, eps, gerber, as-shown |
Abstract: There will be some gradual changes to many of our exporters. These will be slow changes but will definitely break some fine details of some existing Makefiles and export scripts. If you are exporting using scripts, with the old, non-cam syntax, you might be affected.
(Updated in 2024)
I tried to avoid these changes, and I managed to do it for years, but finally we arrived to a point where there is no other choice than to fix this up. It won't happen overnight, you have the time to switch over your scripts.
Reasons and schedule below. What you can do to avoid scripts breaking is in the last part of this mail.
There is also a detailed list of export options that are likely to change .
1. What's wrong now
At the moment a some of exporters have their own idea, their own re-implementation about how some common details are done. These details are not really specific to the export format, but was once a workaround for someone for something, introducing unnecessary special cases.
A typical example is the "as shown" flag and mask handling in png. For some reason the default png export would render a board that is neither the complete stackup, nor the stackup that's currently visible on screen. Instead, it goes and semi-randomly enables and disables layer groups. For example by default it disables mask and paste and far side silk. Then if the --as-shown flag is true, it will allow mask if it is visible on screen, but not paste. And "far side" silk is not allowed either, regardless of --as-shown.
"Fortunately" the eps exporter, that also has an --as-shown parameter, acts differently, in both as-shown and normal mode.
On the one hand, it's hard to predict what such an exporter will do and why it does that and applying very similar export settings will behave differently per exporter. On the other hand some user scritps and Makefiles may depend on the specific broken behavior.
2. Why do we need to change it
This, of course, can be worked around using the CAM sytnax. But it'd be nicer if the plugins showed a much simpler and more unified behavior. It would make the code simpler and would make user's life much easier learning the exporters.
And it would break old scripts.
That's why I didn't really consider changing these things so far. But now, that I am working on the hidlib ( librnd ), for camv-rnd and sch-rnd , I do have to reconsider this: we want to reuse the same plugins with those applications, to export in the same formats. This implies two things.
First, the weaker reason: we simply do not want to reproduce/spread these old mistakes in new applications.
Second, strong reason: to make these unfortunate decisions about mask and far-side silk and such, the export plugin needs to understand PCB stackup - which is simply not available in non-pcb applications.
So at the end it's just ugly, unwanted code that also converts an oterhwise generic, "usable in any context" exporter plugin into a "works for PCB only" one.
Thus we need to get rid of such code, even if it breaks existing scripts (sorry).
3. schedule
My current roadmap for pcb-rnd is releasing one or a few more 2.1.x minor releases. These won't make huge changes in existing features but slowly mature the hidlib (and add unrelated pcb-rnd features).
Then when the hidlib is mature enough to start its own life, that will be called pcb-rnd 2.2.x (hidlib still living in pcb-rnd repo, still installed from pcb-rnd for other apps to use). Then later on the hidlib will be split out (into librnd) to an independent set of plugins that all pcb-rnd and other ringdove apps can use - and that's when pcb-rnd 3.0.0 happens.
The latest date I need to fix the export layer mess is release of 2.2.0. This practically means I need to start doing this early/mid next year.
4. what the solution will probably look like
There are mainly 2 classes of exporters:
- A. per layer exporting: multiple pages (files, images) are exported from a board file - typical examples are ps and gerber
- B. composite exporting: when all (or at least many) layers are put together on a single output page (file, image)
There are 3 ways we can normally export:
- I. direct exporting, "board mode": we don't care what's on screen, we want the board data exported; typical examples are gerber, ps and openscad
- II. direct exporting, "screen mode": the output depends on what's shown on the screen, including which layers are visible, what settings are active (e.g. thin draw mode); sort of a screenshot, except the exporters will override a lot of visibility and settings randomly. Typical examples are png and eps.
- III. CAM exporting: the user specified what layers to combine and what settings to use for exporting a specific file. The only thing coming from core is the layer order (which is pretty much the physical stackup): if you export top copper and top silk togehter, top silk will be above top copper, no matter in what order you specified your layers in the CAM syntax.
Our problem is that the code we inherited hardwired random combinations of A and B with I and II. The png exporter is always a B and --as-shown tried to choose between I and II to a limited extent. Then gerber exporter is A/I, without a good option for II. The ps exporter is mostly A/I, but --single-page can turn it into B/I.
With CAM you can already force both A and B on any exporter, but choosing some aspects of I vs. II is not accessible (e.g. whether selection color is shown on the export, as on screen).
The soluton will look like this: I will remove per exporter --as-showh and --single-page and anything similar settings. Each in direct (noncam) exporting, each exporter will be either A or B. And B mode will be more purely B (more closely matching a "screenshot") and there won't be special cases: what layers you see on the screen will show up on the export.
This means I will _remove_ a lot of exporter-specific options. That's where old scripts may break. Then I will also change some details on how some exporters work, especially what layers they do export (e.g. png will start exporting paste, when visible). This may also break some old scripts that depended on these special, hardwired behaviors.
5. What you can do to avoid problems
Whenever you don't like how a specific exporter behaves now or later, I strongly recommend using the CAM syntax. Either using the exporter directly with a --cam argument, or use the cam exporter with a job set up to your needs.
Especially for automation: the CAM syntax provides a generic, flexible mechanism that works with all relevant exporters. If you switch your scripts over once, you will never have to worry again because the CAM syntax doesn't have hidden hardwired defaults that we may need to change later on.
Direct (non-cam-job) exporting will mainly be kept for quick GUI and interactive command line exports. This will work best for B/II type exporting and will work relatively well with A/I too. Wherever possible we will optimize direct exporting for either of those two use cases and will ask users to use CAM if anything else is needed.