pcb-rnd knowledge pool

 

Anyload

anyload by Tibor 'Igor2' Palinkas on 2020-02-06	Tags: insight, anyload
node source

 

Abstract: Anyload is a small but flexible infrastructure in librnd. Plugins can register in it to offer loading various (usually lihata based) files by type (root node). This allows users to prepare easy-to-use, easy-to-install addons that can be shared on edakrill. The gain is especially high when a single addon requires to use multiple different resources, e.g. drc rules and vendor drill mapping.

 

1. Motivation

First two case studies are presented to describe the motivation behind the anyload infrastructure.

1.1. Case study: vendordrill

The vendor drill plugin remembers a drill map so it can round random drilled hole diameters to a fixed set of drill diameters. The original intention was to comply with fabs that require picking the drill tool from a fixed list.

Unfortunately the vendor drill plugin does one more thing: it has an option to set DRC values. This used to work with the old, hardwired DRC but it is doomed to fail with the scriptable DRC as it is impossible to map a fixed set of DRC settings (known to the vendor drill file format) onto an arbitrary set of settings of DRC scripts.

Despite the name of the plugin is vendor drill and most of the DRC settings have nothing to do with holes, the motivation for mixing in DRC is still clear. They wanted to have a single config file that holds "everything" for a given fab: both vendor drill mapping and drc settings at that time.

And the problems with that design is clear too:

• it hurts modularity by building in DRC assumptions in the vendor drill plugin
• it hurts simplicity by making the vendor drill plugin deal with two, really unrelated topics
• it does not really solve the problem: it will not let anything else, e.g. CAM job configuration be included, thus it fails to bea complete solution for the "vendor specific configure pack problem".

Conclusion: we need a generic way to specify configuration of multiple different plugins in a single, easy to share package. This must not depend on any specific workflow and must be future proof: instead of hardwiring a few settings assumed to be useful for a specific workflow, it should work with any data/state/config of any plugin that is willing to implement the hook for loading its own section of the package.

In practice, an anyload file can be crafted to contain vendor drill mapping, DRC scripts and pcb-rnd config subtrees to specify DRC settings. The vendor drill map is loaded by the vendor drill plugin, the DRC script section is interpreted by the drc_query plugin and the config subrtee is loaded by the config infrastructure. No more unnecessary assumptions and inter-dependencies.

1.2. Case study: edakrill user scripts

Users can share drc scripts and user scripts on edakrill. A minor hassle using scripts from edakrill is that different scripts need to be loaded or installed differently. This is going to be the case with other resources, such as menu patches, vendor drill maps and whatever else future plugins will support.

For advanced users this is okay: the installation process is usually easy, copying a few files here and there, maybe editing a config file entering the name of the script. Plus edakrill tarballs have instructions on this. However, for less experienced users, especially for single use or evaluation this may be a burden.

Conclusion: we need a generic way to give pcb-rnd instructions on how to load a resource (or multiple resources bundled), especially for tarballs downloaded from edakrill.

In practice, anyload does exactly that: it specifies lihata subtrees and external files to load in a certain order.

2. Implementation

There are two ways to use anyload: direct loading of a lihata document (e.g. a menu patch or a vendor drill file) or indirect loading using an anyload lihata file.

2.1. Direct loading

In direct loading the anyload() action will load a lihata document, look at the root node and look for a plugin that understands that root node. The only one thing this feature does is simplify the loading process/UI: instead of having to select a load feature (action or menu) associated with the given file type (e.g. menu patch or vendor drill map file), the user can use a single entry point for the load and the code will figure what specific load feature to use.

2.2. Indirect loading

If there are multiple resources to load and/or there are non-lihata files to load, the indirect method needs to be used. In the indirect method a short anyload lihata file is created, called anyload.lht which contains instructions on what resources to load, in order.

A resource can be:

• a lihata subtree, in-line (copied into anyload.lht)
• reference to an external file (that must be in the same directory tree as anyload.lht, for security reasons)

For example the following anyload.lht changes the background color using an in-line pcb-rnd-conf-v1 subtree after which it loads a vendor drill file from foo.lht:

ha:pcb-rnd-anyload-v1 {
  li:resources {
    li:pcb-rnd-conf-v1 {
      ha:overwrite {
        ha:appearance {
          ha:color {
            background = {#101010}
          }
        }
      }
    }
    ha:file {
      path = foo.lht
      type = vendor_drill_map
    }
  }
}

Each resource has a type. For inline lihata subtrees, the type is the node name under the resources subtree. For external file references with ha:file, it may be a type node. In both cases the type is used to select the plugin. When loading a lihata document, the type is the name of the root node, without the two letter "type:" prefix. If the external file is a lihata document, the type field is optional; when not specified, the root node of the external file will be used as type.

For lihata document resources inlining or using external file reference are interchangable. For non-lihata files an external file reference must be used. With non-lihata external files some plugins will require extra fields, for example user scripts need an id and a lang (see below).

2.3. Plugin support

You can try these examples by downloading the file(s), unpacking when needed and using the anyload GUI from the menu.

2.3.1. Config subtrees

The resource is a lihata document with root node li:pcb-rnd-conf-v1. Config subtrees are loaded on user role by default. Examples:

• single file, in-line (load ex_conf.lht)
• external file (unpack, load ex_conf/anyload.lht or ex_conf/conf.lht)

2.3.2. DRC scripts and settings

The resource is a lihata document with root node li:pcb-rnd-drc-query-v1 or a tEDAx document with a drc_query_rule block. Only the lihata variant can be in-lined. Examples:

• single file, in-line, DRC block in lihata (load ex_drc.lht)
• external file, DRC block in lihata (unpack, load ex_drc_l/anyload.lht or ex_drc_l/floating.lht)
• external file, DRC block in tEDAx (unpack, load ex_drc_t/anyload.lht)

2.3.3. Menu patches

The resource is a lihata document with root node ha:rnd-menu-v1. Examples:

• single file, in-line (load ex_menu.lht)
• external file (unpack, load ex_menu/anyload.lht or ex_menu/mp.lht)

2.3.4. User scripts

Requires anyload.lht with external file reference with the following extra nodes:

• type=user_script
• lang= fungw language, e.g. fawk or tcl or lua
• id= arbitrary, unique script ID (alnum, no space); the user can refer to the script by the ID

Example:

• external file (unpack, load ex_menu/anyload.lht)

2.3.5. Vendor drill mappings

The resource is a lihata document with root node ha:vendor_drill_map. Examples:

• single file, in-line (load ex_vendor.lht)
• external file (unpack, load ex_vendor/anyload.lht or ex_vendor/vendor.lht)

2.3.6. Persistent anyload

It is possible to install an anyload package so that pcb-rnd will load it on once on every startup. To do this, create directory ~/.pcb-rnd/anyload and copy the file (e.g. ex_menu.lht ) or the directory that contains anyload.lht (e.g. ex_menu/ ) under ~/.pcb-rnd/anyload with any file or directory name. Files in that directory must be lihata files with root ha:pcb-rnd-anyload-v1. Directories in that directory must have an anyload.lht file with root ha:pcb-rnd-anyload-v1.

The order of reading the anyload files on startup and directories is arbitrary.

Note: since these files are loaded only once, on startup:

• if new persistent anyload files are installed, pcb-rnd will need to be restarted
• any change they do in global pcb-rnd states can be overridden by further anyload files or user actions