The EDA ecosystem is booted up and is called
coralEDA.
History and background: there is a lot happening in pcb-rnd these days. As
of early 2017, a set
of these seem to nicely connect up and form something that's a big portion
of the edacore idea, just implemented in a different way. We did not have
a grand edacore-like plan behind these, they mostly started to happen
independently. But they really happen to connect that way. In the same time
edacore seems to be in hibernation. When I realized this, I contacted
the edacore people to find out if they are interested in a reboot. But we
decided to walk separate ways.
This is not an edacore reboot, just happens to have similar goals. This
effort is called "EDA ecosystem".
This article focuses on how the ecosystem idea differs from edacore and
other similar ideas floating around these days.
Acceptance and mutual respect
Before we start, we must accept that we have alternative implementations,
different projects going in different directions. Although this means
code and effort duplication, this is not a bad thing, this is a good
thing. Most of us don't want to live in a world where there's only
one EDA software available, even if every development effort is concentrated
there. (If you think you do, then rethink it with substituting your favorite
package with one you really dislike and imagine that's the only one available.)
Respecting the choices of another project seems to be simple in theory, but
it is important to see a few examples of what we need to accept in practice.
Else we make decisions that are unacceptable for one project or another
too easily. An incomplete list of examples of what differs in another EDA project:
- It has different goals. Aspects that are extremely important core
concepts here are something to be avoided there.
- Different data model. Different way the tool models the world. Even,
different subset of the world is modelled by the tool.
- Obviously, different code. This doesn't seem to be a big issue
until a common effort accidentally starts to depend on it, e.g. because
of the lib API.
- Different programming languages. Again shouldn't be a big deal, until
you try to provide common code. As edacore spotted too, a plain C lib
seems to be a common minimum for most of the world, but even
there it might be something alien and can easily become a
"why couldn't we just reimplement in OOP/python/ruby/java" question.
- Different design choices, even beyond the programming language.
For example pcb-rnd walks the minilib way while gEDA/PCB goes for glib.
This has many consequences: pcb-rnd is more portable, less dependent
on GNU, compiles faster. In return we need to ship the minilibs with it
and if someone joins the project they need to learn the APIs. Both choices
are valid in their domain. If we want do anything common, we must make
it neutral to such choices, else it's automatically locked out from
some projects.
In practice this means if we are to do anything common, shared, we either
accept the ways of the other projects and come up with a minimal common
denominator or the common/shared effort will be refused by some (or even most)
projects we target. This has a few consequences, some requirements we
must obey while designing the common:
- It should not interfere with any existing program, code, project
- It should not try to push excess amount of new concepts and ways onto
existing projects; there obviously will be some new concepts, as we
are trying to introduce a new shared feature, but it should be actively
kept minimal.
- Thus we have to live with the idea of code/effort duplication. E.g.
we implement a common library of footprints - it may be a good idea
to also implement a way to index them and store the result in some
complex data format. But what if the other program already has a
complex data format for this? What if our hash table is an incompatible
idea with their lists? It's better to step back and rethink:
do we really have to do the indexing in order to achieve
our goals? If the answer is not "absolutely yes!", we just need to
skip it. Less is more.
- This obviously includes dependencies. If our shared solution depends
on qt, programs using gtk will hate it. If it depends on glib,
programs using minilibs will hate it. If all documentation and comments
are written in Italian, everyone but Italian projects will hate it.
Just reduce such external dependencies, even if this means less
features. Go for a common minimum instead of "would remove some
code duplications and costs only an extra dependency".
- Free/open source means open from ground up, no exceptions.
An open source hardware is not really free if the schematics
comes in a format that's readable only by a proprietary CAD package.
A library or a file format is not free if it is specified by a standard
that can't be included in the same repository for whatever legal
reasons. Yes, this excludes a lot of ISO and STEP and IPC stuff,
and results in effort duplication while we design our own formats.
But there's no point in making a compromise on this and building on
non-free things then see some projects refusing to depend on it
because of the legal implications.
What we could do within these limits?
These problems are not new. There are good examples on how people
have solved these in the past - some examples implemented decades
ago and are still widely used in one form or another! The most trivial
of these are network protocols: SMTP, ftp, HTTP, TCP/IP, the IRC protocol,
etc. What's common in them:
- They specify a protocol and not the actual implementation.
- They are freely accessible as RFCs - no registration, no shopping
carts, no much restrictions; if I wanted to copy the RFC next to my
implementation as part of the documentation and distribute the pack, I am free
to do so.
- They try to concentrate on one thing, specify all relevant details
and leave everything up to the user.
This is a pretty important difference compared to the original edacore
idea (of the 2015 FOSDEM talk: "we should try to write a lib and give
that to the common"). Instead, in the ecosystem idea, we should:
- Define free, open file formats and mechanisms;
- Design for the common minimum, not for the fancy extras and completeness.
This format is for interchange, and we want to reach every project, not
just the big, complex ones. If the format requires you to implement
splines, we lock out a bunch of small players with no curve support.
Don't expect interchange formats to be lossless. Don't expect interchange
formats would replace per project native formats.
- Preferably design even the file formats and mechanisms in a modular
way, expecting some projects would want to implement only parts of it;
make it work as good as possible in such partial implementations!
- As an optional extra we may provide a reference implementation
(code) lib, or even libs. Some projects would use these, embed these, others
would read the source and yet others would totally ignore them. It's
all fine as long as the lib is not the common but is just an appendix.
- Provide optional, but out-of-the-box tools. Small, simple, but
useful ones. If we specify a footprint format, provide tools
that parses it, makes checks on it ("lint"), tools that visualize
the footprint (e.g. generate a png). This can be anything from a
web service to command line tools. Best if there are even multiple
interfaces and APIs. This helps projects to adopt the file formats
as they can try it in advance and they see they have debug tools
readily available, still these tools won't interfere with their projects.
In other words: solve the interchange problem, not the code
duplication problem!
What we don't need to do?
We could also do any of these, but to get the project started and be
useful, we are not required to do them. It's an important consideration:
we don't have much resources, we should spend it where it's spent the best.
I believe we don't need to do the below in order to get the ecosystem idea work:
- Common code lib implementations; it's a nice extra, but it's the file
format specs that would make it work
- Some centralized repository, web service, etc. for sharing user content:
once a few tools already have support for the ecosystem, this will
automatically happen. Someone, or hopefully even multiple people will
make their public services. Or people will just use github, torrent
or whatever techniques. Of course we can also run
our own repository service but
that should not be tied together with the ecosystem idea, it should just
be one user of the formast (that happens to be maintained by the same people who
made up the specs).
- "Fancy features". Provide the bare minimum. It's a tradeoff between
the "works everywhere" vs. "looks perfect in all little details". If
the user wants the former, an interchange format is for that; for the
latter, use a native format.
What we could expect from such a project?
Assume we designed a set of interchange formats and maybe mechanisms and
we optionally provided some libs and tools.
- If we can get at least 2 projects using the new format for exchanging
data, we are at the same point as with the original n*n converter
EDA ecosystem idea. We just
wasted some more time on a yet-another-format.
- But if 3 or more projects are using it, we may saved some effort already!
- If that happens, users can build and share libraries more easily.
- Users will find new ways to combine our tools in the ecosystem - even
unusual ways developers didn't think of, e.g. using schematics cap from
one project and PCB editor from another, even if both projects were
"integrated".
What we should NOT expect from such a project?
- A common interchange format replacing local native formats.
- A common interchange format capturing all tiny details that a native
format can.
- Proprietary EDA tools cooperating on this on the export-side - they
don't want their users to be able to get the design out of their
program, especially not if it's clearly for importing in another.
- Everyone's support - even with the most careful design and the lowest
barriers there will be project that won't like it and won't join. Some
projects will hate it exactly because it's not restrictive enough, e.g.
that it doesn't mandate OOP or SQL or other specific complexities.
Implementation
We are trying to show good example with
pcb-rnd, which is compatible
with a wide variety of open source and proprietary EDA systems and file
formats.
I've designed a minimalistic interchange format called
tEDAx .
We are also running edakrill , an EDA
and format agnostic data sharing service.