On this page
Deciding whether a built assembly tree is fit to deploy, before anything about it is committed or published, and with no flag that turns the check off.
#internal/blog/verify
#internal/blog/verify
Package verify answers whether a built assembly tree is fit to deploy.
The deploy used to be the first reader of the tree it published: the graft, the shared generator and the search index each did their part, the result was committed, and whatever was wrong with it became the live site. This package is the reading that happens before the push -- one pass over the assembled tree asserting every property the site depends on, each failure naming its offender.
#What is asserted
- Membership agrees in both directions. The declared roster, the site/ subtrees and the files under manifests/ name the same projects: no undeclared subtree, no declared project missing, no orphan manifest of any kind. - Each manifest describes the tree it sits next to. Its slug names its own directory, its version is the version the emitted pages carry (and is not sitting in the archive tree as though it were superseded), and every page and post it lists resolves to a file that exists. - The shared artifacts exist, parse, and say what they are for. Front page, blog index, project listing, nav.json, sitemap, feed, search index, robots, llms.txt and the root 404. Three of those are asserted on their content rather than their existence: the 404 body is not the front page's and offers a way back, robots.txt names the sitemap the tree actually carries, and llms.txt references every declared project's own llms.txt. - Every reference resolves. Internal links, canonicals, sitemap entries and feed links all go through the resolution package -- the same LINK001 pass a single project's build is checked with, run over the assembled tree. - Every page is addressable. A title, and a canonical under the site's canonical base. - Every page is styled by the site's own chrome. The site-level stylesheet is in the tree and every page links it. A page linking a copy inside its own subtree, or none at all, fails: the shared pages published as unstyled HTML for as long as nothing asserted this. - Nothing half-built or per-project leaked in. No unresolved directive markers, and none of the per-project routing artifacts the graft filters out. - Cross-project links land somewhere. Extracted from the emitted pages and checked against what the manifests say exists. - Every project is reachable. Following clickable links from the site root and the project listing arrives at every declared project's index page, so no project is published at an address a reader arriving at the site never sees and a crawler following links never reaches. Either arrival page may curate what it shows. The home project is exempt: the site root is its own front page. - Outbound links still answer, when the assembly declares a list of pages to check them on. See [github.com/smm-h/selfdoc/internal/blog/site.LoadOutbound].
The outbound cache is the one piece of state a verification produces. Verification itself never writes: [VerifyAssembly] returns the updated cache and the caller decides whether to keep it, which is why the "assembly verify" command is read-only and the deploy -- which does write, and commits the result -- is where the cache actually persists between runs.
#NotFoundPage
const NotFoundPage = "404.html"NotFoundPage is the page a hosting provider serves for an address that matches nothing. One per served root, which on this site means one, at the site root.
#Checks
var Checks = []string{Checks is every assertion, in the order a run reports them. A check that finds nothing still reports itself as run, so a report can tell "asserted and clean" from "never asked".
#SharedRoutingFiles
var SharedRoutingFiles = []string{"_headers", NotFoundPage}SharedRoutingFiles is what the assembly itself is allowed to serve at the site root. Every other routing artifact belongs to a single project's own standalone hosting and fights the site-wide one wherever it lands.
A redirect worker is not among them: the assembly emits none, a host that should answer somewhere else is a rule on the DNS zone, and a "_worker.js" left at the root by a deploy that predates that is refused here and deleted by the next integration's shared-files pass.
#RoutingArtifactNames
var RoutingArtifactNames = []string{RoutingArtifactNames is every routing file a per-project build emits for its own standalone hosting.
#RoutingArtifactSuffixes
var RoutingArtifactSuffixes = []string{".gz", ".br"}RoutingArtifactSuffixes are the pre-compressed copies a standalone build writes beside every asset.
#Error
type Error structError is the failure every operation in this package reports when it cannot read the tree it was asked to verify: a missing roster, a manifest that is not JSON, a file that vanished between the walk and the read.
A violated property is a Failure rather than an error: the point of a verification is to report every one of them at once.
#Fetcher
type Fetcher func(url string) (int, string)Fetcher fetches a URL and returns its status and the error text, which is empty when there was none.
It is the one seam in this package that leaves the machine, and the one a test replaces. A failed transport is status 0 with the reason, so a caller never has to tell "did not answer" from "answered badly" by inspecting an error value.
#Failure
type Failure structFailure is one asserted property, one offender.
#Skip
type Skip structSkip is one check that could not run, and why. It is never silent -- the CLI prints them and the deploy logs them.
#VerifyReport
type VerifyReport structVerifyReport is what a verification found.
#AssemblyTree
type AssemblyTree structAssemblyTree is the assembled tree, read once and handed to every check.
#CheckRosterAgreement
func CheckRosterAgreement(tree *AssemblyTree) ([]Failure, error)CheckRosterAgreement asserts that the roster, the site subtrees and the manifests name the same projects.
#CheckHomeProject
func CheckHomeProject(tree *AssemblyTree) ([]Failure, error)CheckHomeProject asserts that the home project is served at the site root, once, and only there.
Four properties, each a way the front page could quietly stop being the front page:
- No site/
#CheckManifestIdentity
func CheckManifestIdentity(tree *AssemblyTree) ([]Failure, error)CheckManifestIdentity asserts that each manifest names its own directory and the version on disk.
#CheckManifestPagesEmitted
func CheckManifestPagesEmitted(tree *AssemblyTree) ([]Failure, error)CheckManifestPagesEmitted asserts that every page a manifest lists resolves to an emitted file.
#CheckManifestPostsEmitted
func CheckManifestPostsEmitted(tree *AssemblyTree) ([]Failure, error)CheckManifestPostsEmitted asserts that every post a manifest lists resolves to an emitted file.
#CheckSharedArtifacts
func CheckSharedArtifacts(tree *AssemblyTree) ([]Failure, error)CheckSharedArtifacts asserts that the cross-project files exist, parse, and are not empty.
#CheckReferences
func CheckReferences(tree *AssemblyTree) ([]Failure, error)CheckReferences asserts that every emitted reference names a file the assembly wrote.
One LINK001 pass over the assembled tree, reported under three checks: a sitemap entry, a feed link and a link on a page are three different things to get wrong.
#CheckPageMetadata
func CheckPageMetadata(tree *AssemblyTree) ([]Failure, error)CheckPageMetadata asserts that every page has a title and a canonical under the canonical base.
The 404 is the one page with no canonical, and its absence is the assertion rather than an exemption from one: it is the answer to every address the site does not serve, so it has no address of its own to name. Its title is still required -- a browser tab and a crawler both read it.
#CheckSiteChrome
func CheckSiteChrome(tree *AssemblyTree) ([]Failure, error)CheckSiteChrome asserts that the site-level stylesheet exists and every page references it.
This is the assertion the live site went without. The blog index, the project listing and the root 404 were wrapped by a function whose stylesheet parameter no caller passed, so they published as unstyled HTML and nothing said so -- the pages had titles, canonicals and links that all resolved, and every assertion made about a page passed.
Two properties, one check. The asset has to be in the tree, and every page has to name it: a page naming its own subtree copy instead is a page a toolchain upgrade will not reach, and a page naming nothing is the defect itself. That the reference resolves is not asserted here -- it is a document-relative href like any other, and the LINK001 pass behind internal-references already measures it against the emitted tree.
#CheckUnresolvedDirectives
func CheckUnresolvedDirectives(tree *AssemblyTree) ([]Failure, error)CheckUnresolvedDirectives asserts that no page carries a directive the build did not resolve.
Code and preformatted blocks are excluded: the documentation of the directive syntax quotes every marker there is, and quoting one is not leaving one behind.
#CheckRoutingArtifacts
func CheckRoutingArtifacts(tree *AssemblyTree) ([]Failure, error)CheckRoutingArtifacts asserts that no per-project routing file survived the graft.
The assembly serves one set of headers, one worker and one not-found page for the whole site. A project's own copies -- and the pre-compressed variants its build emits for its own hosting -- fight them wherever they sit, so the graft filters them out and this is that filter, asserted.
404.html is in that set for a sharper reason than a fight: a subtree copy is not served at all. The provider answers an unmatched address from the root of what it serves, so a project's own 404 is an unreachable page that still has to satisfy every assertion made about a page, and it failed the canonical one on every project that published.
#ExtractLinkRegistry
func ExtractLinkRegistry(tree *AssemblyTree) (map[string][]string, error)ExtractLinkRegistry maps each emitted page to the other projects' addresses it links to.
This is the half ValidateCrossProjectLinks was written against and never had: the function knows what every project publishes, but nothing produced the registry of what the pages actually link to. A reference is resolved against the emitted tree, turned back into the address form the manifests speak, and kept only when it leaves the project the page belongs to -- a link inside one project is the LINK001 pass's business, not this one's.
Site-level pages are not read at all. A post has no project segment, so its first path segment is "blog" -- not a project, and treating it as one makes every link a post's own chrome writes back into the project that published it look like a cross-project link. Those links are generated by that project's build from its own addresses, and they reach pages the manifests never list (the glossary, the API and CLI indexes are published without appearing there). Whether they resolve is the LINK001 pass's question, and it answers it for every one.
#CheckCrossProjectLinks
func CheckCrossProjectLinks(tree *AssemblyTree) ([]Failure, error)CheckCrossProjectLinks asserts that every link into another project names a page that project publishes.
#CheckProjectReachability
func CheckProjectReachability(tree *AssemblyTree) ([]Failure, error)CheckProjectReachability asserts that every roster project's index page is reachable from an arrival page by following clickable links.
A project nothing links is published and unreachable: a reader arriving at the site never sees it, and a crawler that follows links never finds it. The walk starts at the two pages a reader arrives through, the site root and "/projects/", and follows every clickable link that resolves to an emitted page, so a project the front page curates away is still reached through the listing, and one the listing leaves out is still reached through the sibling block every assembled page carries. A project the walk never reaches is the finding.
The home project is not asked for: the site root is its own front page, so it is reached by being the destination rather than by being linked.
Links are read the way the resolution rule reads them: as clickable anchors resolved against the page that writes them, so a document-relative link and an index.html written out both resolve to the same target.
#FetchURL
func FetchURL(url string) (int, string)FetchURL is the default Fetcher: a GET with a timeout, mapping every transport failure to status 0 and the failure's text.
It is a GET: it changes nothing, which is why it does not go through the effects handle.
#Now
func Now() float64Now is the wall clock in the units the outbound store records, for a caller that is verifying a real tree rather than pinning an instant in a test.
It exists so the one conversion from a Go clock to the store's seconds lives in one place: the stored timestamps came from Python's time.time() and are compared against the declared cache window in seconds.
#CheckOutboundLinks
func CheckOutboundLinks(CheckOutboundLinks fetches the outbound links on the declared pages and returns the failures, the store as this run leaves it, and how many requests it made.
A cached result inside the window answers without a request, so a second run over an unchanged tree makes none at all.
#ReadTree
func ReadTree(assemblyDir, canonicalBase string) (*AssemblyTree, error)ReadTree reads everything a verification needs out of assemblyDir.
#VerifyAssembly
func VerifyAssembly(VerifyAssembly asserts every property the assembled tree has to have before a deploy.
assemblyDir is the assembly repository checkout, holding site/, manifests/ and the roster.
canonicalBase is the site's canonical base URL. Absolute references -- canonicals, sitemap entries, feed links -- are this site's when they sit under it, and somebody else's when they do not, so there is nothing to verify against without it, and an empty one is refused.
fetch is the outbound fetch layer; nil selects [FetchURL].
now is the clock the cache window is measured against, in seconds since the epoch. It is the caller's to supply -- the deploy passes the wall clock and a test passes a fixed instant -- so a run's verdict is a function of its inputs.
Verification never writes: the updated outbound store rides on the report for the caller to persist.
#Error.Error
func (e *Error) Error() string { return e.Message }Error returns the diagnostic.
#Failure.String
func (f Failure) String() stringString renders the failure as one line, the way the report's own rendering and every diagnostic that quotes a single failure spell it.
#VerifyReport.OK
func (r *VerifyReport) OK() bool { return len(r.Failures) == 0 }OK reports whether the tree passed every assertion.
#VerifyReport.FailuresOf
func (r *VerifyReport) FailuresOf(check string) []FailureFailuresOf returns the failures reported under one check, in report order.
#VerifyReport.ErrorText
func (r *VerifyReport) ErrorText() stringErrorText renders the whole report as one message, for a raise or a stderr dump.
#AssemblyTree.Read
func (t *AssemblyTree) Read(rel string) (string, error)Read returns the text of an emitted file, addressed site-relative.