pcb-rnd knowledge pool


element-resrtictions: why is it not a file format issue?

data2 by Tibor 'Igor2' Palinkas on 2017-07-11

Tags: insight, element, limitation, file format, file, format, subcircuit

node source



Abstract: Busting a myth: the file format was not the issue that caused all the element limitations.

  Because this topic comes up from time to time, and by now I have actual data (implementation costs) on it, I decided to write a short analysis on:

Note: A solution (including a global data srtuct cleanup) is being implemented in pcb-rnd .

A random quote from the geda-user mailing list:

 Maybe you composed the footprint entirely of atoms that are not elegible
 to be included in a pcb footprint. Due to restrictions in the file

It is a common pattern that users express element restrictions are really file format restrictions. Unfortunately this is not the case. Unfortunately, because fixing a mere file format would be much cheaper.

The problem

Elements contain element-specific drawing primitives and the code has special casing for these in every little corner. Extending/replacing the file format is a relatively small effort, affecting a small part of the code base. Adding support for polygons, copper lines, vias, arbitrary text, etc. in elements is a huge effort because it really touches everything from the famous find.c through all drawing pimitive holding data structures to draw.c.

When you see a silk line or silk arc on screen, produced by an element, it is not really just a line or arc. It is really an element-line and an element-arc. Yes, these are separate, special drawing primitives that work only in elements; elements can't consists of plain line or arcs. Majority of the code simply has an "if" or a separate "case" branch or a separate callback function for handling each of these element-* object. The data structs don't hold a list of drawing primitives; they hold a list and an rtree of lines, another list and rtree of arcs, etc. And yes, this means a specific element-line and element-arc list within elements. Plus these element-objects ignore layers: instead of being on a layer, they are either on the top or on the bottom side. For silk and pads it's fine, but if you want outline, paste, inner copper layers, that's 1000 other special casing.

For example if you wanted to have non-pad copper lines in a pcb footprint, extending the existing code base, you'd need to:

(Or you can chose to rather implement a generic object grouping feature of the existing primitives: line, arc, text, poly, via, and say element-* special casing should be just removed as groups can replace elements. This is what I am doing in pcb-rnd with subcircuits, see below)

This is why, contrary to the popular belief, simply changing the file format to xml or json or whatever-else won't magically fix the problem. Instead rewriting 1/3 of the code base would. Then whether you keep the .fp/.pcb format (and upgrade it to handle the new situation) or introduce a new file format is a tiny implementation detail.

Note: I'm saying this after implementing a new native file format (lihata), working on half dozen alien board/footprint file formats _and_ being halfway through resolving the element restrictions by replacing elements with subcircuits in pcb-rnd.

evidences: data

Below I'm sharing objective data on this topic. Talking in pcb-rnd net development times, implementing support for a new footprint file format is ranging from a few hours to 1..2 days, if the format is reasonable. For example implementing a working plugin for tEDAx that loads netlist, loads and saves footprints took less than 14 hours total. Note: this is the time spent on a new, non-native format _and_ a netlist load. Extending the .fp/.pcb format would be way cheaper, so you can take this as a very permissive upper estimation.

Implementing half of the subcircuits took me 85 hours so far: we have generic grouping, with dynamic layer binding; we can save and load the result; we can have any object on any layer; we keep track of rotation/mirroring/scaling; we have and unique ID to keep track on what subcircuit is cloned from what other subcircuit. What's still missing for replacing elements, and we will have in the next development cycle: "terminals" (turning any copper object into a pin or pad).

By the time subcircuits will be able to fully replace footprints/elements, I estimate total expense between 180..200 hours. And these "low" numbers are possible only because I had already invested more than 130 hours in the layer rewrite before I started on subcircuits. If we account the the element-restriction-resolving parts of the layer rewrite in this effort, the final "element rewrite" cost would be over 250 hours (from which more than half are already spent and only the other part is estimation).


250 hours of subcircuits+layers (half spent, half estimated) vs. 14 hours of a file format (spent).

So a [new footprint file format + new netlist format] was less than 6% of the half-measured-half-estimated cost of solving the element restriction problem in case of pcb-rnd.

Of course different projects and different developers would make different choices, and with subcircuits I am really doing the most flexible thing possible: i just decided to really clean up the data model, not just hack in polygons and copper lines.

But I still say that with even a cheaper, more hackish element upgrade the file format issue is only 5..15% of the whole problem.