Namespace Resolution Rules

This page defines the formal rules for resolving references to concepts and pipes across bundles, domains, and packages.

Reference Syntax Overview

All references to concepts and pipes in MTHDS follow a uniform three-tier syntax:

Parsing Rules

Splitting Cross-Package References

If the reference string contains ->, it is a cross-package reference. The string is split on the first ->:

• Left part: the package alias.
• Right part: the remainder (a domain-qualified or bare reference).

The alias MUST be snake_case. The remainder is parsed as a domain-qualified or bare reference.

Splitting Domain-Qualified References

For the remainder (or the entire string if no -> is present), the reference is parsed by splitting on the last . (dot):

• Left part: the domain path.
• Right part: the local code (concept code or pipe code).

If no . is present, the reference is a bare name with no domain qualification.

Examples:

Disambiguation: Concept vs. Pipe

When parsing a domain-qualified reference, the casing of the local code (the segment after the last .) determines whether it is a concept or a pipe:

• PascalCase ([A-Z][a-zA-Z0-9]*) → concept code.
• snake_case ([a-z][a-z0-9_]*) → pipe code.

This disambiguation is unambiguous because concept codes and pipe codes follow mutually exclusive casing conventions.

Domain Path Validation

Each segment of a domain path MUST be snake_case:

• Match pattern: [a-z][a-z0-9_]*
• Segments are separated by .
• No leading, trailing, or consecutive dots

Resolution Order for Bare Concept References

When resolving a bare concept code (no domain qualifier, no package prefix):

1. Native concepts — check if the code matches a native concept code (Dynamic, Text, Image, Document, Html, TextAndImages, Number, YesNo, Date, Time, Page, JSON, SearchResult, Anything, Composite). Native concepts always take priority.
2. Current bundle — check concepts declared in the same .mthds file.
3. Same domain, other bundles — if the bundle is part of a package, check concepts in other bundles that declare the same domain.
4. Error — if not found in any of the above, the reference is invalid.

Bare concept references do NOT fall through to other domains or other packages.

Resolution Order for Bare Pipe References

When resolving a bare pipe code (no domain qualifier, no package prefix):

1. Current bundle — check pipes declared in the same .mthds file.
2. Same domain, other bundles — if the bundle is part of a package, check pipes in other bundles that declare the same domain.
3. Error — if not found, the reference is invalid.

Bare pipe references do NOT fall through to other domains or other packages.

Resolution of Domain-Qualified References

When resolving domain_path.name (no package prefix):

1. Look in the named domain within the current package.
2. If not found: error. Domain-qualified references do not fall through to dependencies.

This applies to both concept and pipe references.

Resolution of Package-Qualified References

When resolving alias->domain_path.name:

1. Identify the dependency by the alias. The alias MUST match a key in the [dependencies] section of the consuming package's METHODS.toml.
2. Look in the named domain of the resolved dependency package.
3. If not found: error.

Visibility constraints for cross-package pipe references:

• The referenced pipe MUST be exported by the dependency package (listed in its [exports] section or auto-exported as a main_pipe).
• If the pipe is not exported, the reference is a visibility error. A compliant implementation MUST diagnose it at validation time, naming the dependency and its export surface — it MUST NOT silently omit the pipe at load time and let the reference fail later or resolve elsewhere.

Visibility for cross-package concept references:

• Concepts are always public. No visibility check is needed for cross-package concept references.

Visibility Rules (Intra-Package)

Within a package that has a METHODS.toml manifest:

• Same-domain references — always allowed. A pipe in domain legal.contracts can reference any other pipe in legal.contracts without restriction.
• Cross-domain references (within the same package) — the target pipe MUST be exported. A pipe in domain scoring referencing legal.contracts.extract_clause requires that extract_clause is listed in [exports.legal.contracts] (or is the main_pipe of a bundle in legal.contracts).
• Bare references — always allowed at the visibility level (they resolve within the same domain).

When no manifest is present (standalone bundle), all pipes are treated as public.

Reserved Domains

The following domain names are reserved at the first segment level:

Enforcement points:

• A compliant implementation MUST reject METHODS.toml exports that use a reserved domain path.
• A compliant implementation MUST reject bundles that declare a domain starting with a reserved segment when the bundle is part of a package.
• A compliant implementation MUST reject packages at publish time if any bundle uses a reserved domain.

The native domain is the only reserved domain with active semantics: it serves as the namespace for native concepts (native.Text, native.Image, etc.).

Package Namespace Isolation

Two packages MAY declare the same domain name (e.g., both declare domain = "recruitment"). Their concepts and pipes are completely independent — there is no merging of namespaces across packages.

Within a single package, bundles that share the same domain DO merge their namespace. Concept or pipe code collisions within the same package and same domain are errors.

Conflict Rules

Version Resolution Strategy

When resolving dependency versions, a compliant implementation MUST use Minimum Version Selection (MVS), following Go's approach:

1. Collect all version constraints for a given package address from all dependents (direct and transitive).
2. List all available versions (from VCS tags).
3. Sort versions in ascending order.
4. Select the minimum version that satisfies all constraints simultaneously.

If no version satisfies all constraints, the resolution fails with an error.

Constraints are read as floors: a declared constraint states the minimum version a dependent is known to work with, and for plain floor constraints the selected version is the maximum of the declared floors across the graph. A newly published version is never adopted implicitly — it becomes eligible only when some manifest raises a floor to it.

Properties of MVS:

• Deterministic — the same set of constraints always produces the same result. The build list is fully determined by the manifests and the published version tags, with no registry oracle; the lock file is a verification record, not a resolution record (see methods.lock Format).
• Reproducible — no dependency on a "latest" query or timestamp.
• Simple — no backtracking solver needed, and no conflicts by construction within an admissible range: a resolvable answer exists whenever the constraint sets overlap.

Raising versions is a manifest edit. Because re-resolution alone never moves a version, the version-raising gestures are defined as edits to METHODS.toml followed by a re-lock:

• add fetches the dependency's latest version and records it as the floor.
• update raises the floors of existing dependencies to the latest available versions, then re-locks. This is the deliberate-adoption gesture: nothing upgrades silently, and tooling SHOULD surface what moved between the old and new resolved versions (the lock's fingerprint field makes that diff semantic rather than textual).

Transitive Dependency Resolution

Dependencies are resolved transitively with the following rules:

• Remote dependencies are resolved recursively. If Package A depends on Package B, and Package B depends on Package C, then Package C is also resolved.
• Local path dependencies are resolved at the root level only. They are NOT resolved transitively.
• Cycle detection — if a dependency is encountered while it is already on the resolution stack, the resolver MUST report a cycle error.
• Diamond dependencies — when the same package address is required by multiple dependents with different version constraints, MVS selects the minimum version satisfying all constraints simultaneously.

Fetching Remote Dependencies

Package addresses map to Git clone URLs by the derivation rule of Distribution: Clone URL Derivation: the hostname and the first two path segments name the repository — prepend https://, append .git. Any further path segments are not part of the clone URL; they select a package within the clone, located by manifest identity per Distribution: Locating a Package Inside a Clone.

For example: github.com/acme/legal-tools → https://github.com/acme/legal-tools.git, and github.com/mthds/methods/documents → https://github.com/mthds/methods.git with documents locating the package inside the clone.

The resolution chain for fetching a dependency is:

1. Local path — if the dependency has a path field in METHODS.toml, resolve from the local filesystem.
2. Local cache — check ~/.mthds/packages/{address}/{version}/ for a cached copy.
3. VCS fetch — clone the repository at the resolved version tag using git clone --depth 1 --branch {tag}.

Version tags in the remote repository MAY use a v prefix (e.g., v1.0.0). The prefix is stripped during version parsing.

Cache Layout

The default package cache is located at ~/.mthds/packages/. Cached packages are stored at:

~/.mthds/packages/{address}/{version}/

For example:

~/.mthds/packages/github.com/acme/legal-tools/1.0.0/

The .git directory is removed from cached copies.

Cross-Package Reference Examples

The following examples illustrate the complete reference resolution for cross-package scenarios.

Setup: Package A depends on Package B with alias scoring_lib.

Package B (METHODS.toml):

[package]
name        = "scoring_lib"
address     = "github.com/mthds/scoring-lib"
version     = "0.5.0"
description = "Scoring utilities"

[exports.scoring]
pipes = ["compute_weighted_score"]

Package B (scoring.mthds):

domain    = "scoring"
main_pipe = "compute_weighted_score"

[concept.ScoreResult]
description = "A weighted score result"

[pipe.compute_weighted_score]
type        = "PipeLLM"
description = "Compute a weighted score"
inputs      = { item = "Text" }
output      = "ScoreResult"
prompt      = "Compute a weighted score for: $item"

[pipe.internal_helper]
type        = "PipeLLM"
description = "Internal helper (not exported)"
inputs      = { data = "Text" }
output      = "Text"
prompt      = "Process: $data"

Package A (analysis.mthds):

domain = "analysis"

[pipe.analyze_item]
type        = "PipeSequence"
description = "Analyze using scoring dependency"
inputs      = { item = "Text" }
output      = "Text"
steps = [
    { pipe = "scoring_lib->scoring.compute_weighted_score", result = "score" },
    { pipe = "summarize", result = "summary" },
]

Resolution of scoring_lib->scoring.compute_weighted_score:

1. -> detected — split into alias scoring_lib and remainder scoring.compute_weighted_score.
2. Look up scoring_lib in Package A's [dependencies] — found, resolves to github.com/mthds/scoring-lib.
3. Parse remainder: split on last . → domain scoring, pipe code compute_weighted_score.
4. Look in domain scoring of the resolved Package B — pipe found.
5. Visibility check: compute_weighted_score is in [exports.scoring] pipes — accessible.
6. Resolution succeeds.

If Package A tried scoring_lib->scoring.internal_helper:

1. Steps 1–4 as above — pipe internal_helper is found in Package B's scoring domain.
2. Visibility check: internal_helper is NOT in [exports.scoring] and is NOT main_pipe — visibility error.

Cross-package concept reference:

[concept.DetailedScore]
description = "An extended score with additional analysis"
refines     = "scoring_lib->scoring.ScoreResult"

This refines ScoreResult from Package B. Concepts are always public, so no visibility check is needed.

Validation Rule Summary

This section consolidates the validation rules scattered throughout this specification into a single reference.

Bundle-Level Validation

1. The file MUST be valid TOML.
2. domain MUST be present and MUST be a valid domain code.
3. main_pipe, if present, MUST be snake_case and MUST reference a pipe defined in the same bundle.
4. Concept codes MUST be PascalCase.
5. Concept codes MUST NOT match any native concept code.
6. Pipe codes MUST be snake_case.
7. refines and structure MUST NOT both be set on the same concept.
8. Local concept references (bare or same-domain) MUST resolve to a declared concept in the bundle or a native concept.
9. Same-domain pipe references MUST resolve to a declared pipe in the bundle.
10. Cross-package references (-> syntax) are deferred to package-level validation.

Concept Structure Field Validation

1. description MUST be present on every field.
2. If type is omitted, choices MUST be non-empty.
3. type = "dict" requires both key_type and value_type.
4. type = "concept" requires concept_ref and forbids default_value.
5. type = "list" with item_type = "concept" requires item_concept_ref.
6. concept_ref MUST NOT be set unless type = "concept".
7. item_concept_ref MUST NOT be set unless item_type = "concept".
8. default_value type MUST match the declared type.
9. If choices is set and default_value is present, default_value MUST be in choices.
10. Field names MUST NOT start with _.

Pipe Validation (Type-Specific)

1. PipeLLM: All prompt variables MUST have matching inputs. All inputs MUST be used.
2. PipeStructure: Exactly one input MUST be declared. The input concept MUST be Text or refine Text. output MUST NOT be Text or refine Text.
3. PipeFunc: function_name MUST be present.
4. PipeImgGen: prompt MUST be present. All prompt variables MUST have matching inputs.
5. PipeExtract: Exactly one input MUST be declared. output MUST be "Page[]".
6. PipeCompose: Exactly one of template or construct MUST be present. Output MUST NOT use multiplicity.
7. PipeSequence: steps MUST have at least one entry.
8. PipeParallel: output MUST be Composite or a structured concept, MUST NOT use multiplicity, and structured output fields MUST match branch result names.
9. PipeCondition: Exactly one of expression_template or expression MUST be present. outcomes MUST have at least one entry.
10. PipeBatch: input_list_name MUST be in inputs. input_item_name MUST NOT equal input_list_name or any inputs key.

Manifest Validation

1. [package] section MUST be present in METHODS.toml.
2. name MUST be present, snake_case, and 2–25 characters.
3. address MUST match the hostname/path pattern.
4. version MUST be valid semver.
5. description MUST NOT be empty.
6. main_pipe, if present, MUST be a valid snake_case pipe code.
7. All dependency aliases MUST be unique and snake_case.
8. All dependency addresses MUST match the hostname/path pattern.
9. All dependency version constraints MUST be valid.
10. Domain paths in [exports] MUST be valid domain codes and MUST NOT use reserved domains.
11. All pipe codes in [exports] MUST be valid snake_case.

Package-Level Validation

1. Bundles MUST NOT declare a domain starting with a reserved segment.
2. Cross-package references MUST reference known dependency aliases.
3. Cross-package pipe references MUST target pipes exported by the dependency — diagnosed at validation time, never silently filtered at load time.
4. Exported pipes MUST exist in the scanned bundles.
5. Same-domain concept and pipe code collisions across bundles (within one package) are errors.

Lock File Validation

1. Each entry's version MUST be valid semver.
2. Each entry's hash MUST match sha256:[0-9a-f]{64}.
3. Each entry's fingerprint MUST be a 64-character lowercase hex digest.
4. Each entry's source MUST start with https:// and end with .git.
5. Each entry's commit MUST be a 40-character lowercase hex digest.

Summary: Reference Resolution Flowchart

Given a reference string R:

1. Does R contain "->"?
   YES → Split into (alias, remainder).
         Look up alias in [dependencies].
         Parse remainder as domain-qualified or bare ref.
         Resolve in the dependency's namespace.
         For pipes: check export visibility.
   NO  → Continue to step 2.

2. Does R contain "."?
   YES → Split on last "." into (domain_path, local_code).
         Resolve in domain_path within current package.
   NO  → R is a bare name. Continue to step 3.

3. Is R a concept code (PascalCase)?
   YES → Check native concepts → current bundle → same domain.
   NO  → R is a pipe code (snake_case).
         Check current bundle → same domain.

4. Not found? → Error.