This is the start of a discussion around external documents. I want to add this do the SBOM design doc at some point. But want to open it up to a broader discussion before.

Both SPDX and CycloneDX support referencing nodes in external documents. So far we ignore those, but in a recent
discussion, this topic came up. We need to tackle that issue (#533)
anyway.

Spec

For SPDX, external documents are listed in the header of the document. They are defined with:

• An ID string
• A document namespace (URI)
• A checksum/digest/hash

This basically provides a mapping table from an internal ID to an external (document namespace), plus a safeguard
with the digest.

The document namespace should be unique for each document created. Changes to the document must result in a new
namespace.

The ID string has the format of DocumentRef-<id>, where <id> is some unique identifier.

When using in a relationship, it will be combined with a node id: Document-Ref-<id>:<node-id>.

Implementation

We already have the digest of the target document. We also have the document namespace.

We cannot use the document namespace to locate a document, as it might be a URI, but the spec says:

Although it is not required, the URI can be constructed in a way which provides information on how the SPDX document can be found.

So there's no guarantee that the URI actually points to the document.

We would need to store the "ID" of the external reference, which is only valid in the context of a single SPDX SBOM.

Resolving packages

When querying today, we return a single hierarchy by default. That shouldn't be a different when adding those external
references.

When resolving transient dependencies, that might be different. One way to deal with this could be to stop resolving
when an external reference is encountered. Similar to "symlinks" on a Unix system. Traditionally, operations recursing
into a directory stop with a symlink, but report the symlink itself. Unless the user requests to follow symlinks.

Foreign keys

Currently, we ingest the relationships with a foreign key in the node IDs. That won't work for the external references,
as that would require ingesting the referenced document first. Also, it would create issues when deleting such
documents.

One way to deal with this would be to create a second relationship table. One that allows one side of the relationship
to be external, not enforcing any foreign key.

That would duplicate things. But it might also be helpful in lookups, or transient resolve operations, where we might
want to opt out of processing external references.

I don't see an alternative other than giving up foreign keys, which I'd like to avoid if possible.

What would be possible is to create additional "non-foreign key" fields for the reference in the same table. However,
that would basically do the same (two tables) but squeeze them into one, and feels quite messy.

Conflicting documents

I am 100% sure that we will encounter the issue that there's a conflict for the combination of document namespace and
digest. Which comes from updating a document, without updating the document namespace. Which is what we do at RH.

I don't think this should become a problem though. Since we would find multiple (possible) targets for the reference,
but could then eliminate others due to the mismatched digest.

Also, we need to be careful to store the original digest of the document, not the one after applying any fixes (like
license expressions).