On this page
Resolving selfdoc's directives against Python source with a pure-Go parse of the file, reproducing what the standard library's ast module reports.
#internal/extractors/python
#internal/extractors/python
Package python resolves selfdoc's directives against Python source.
#What the pages are built from
Every Python reference page selfdoc has ever produced was rendered from the stdlib ast module: ast.unparse decides how an annotation, a default value and a base class read, and the tree's child order decides what order the page lists symbols in. Both are reproduced here over a tree-sitter parse -- a pure-Go one, with the Python grammar's tables read from the parser library's embedded blob -- so documenting a Python project needs no interpreter on the machine and no cgo in the build.
The split inside the package follows that: parse.go reads the tree into the shape ast has (docstrings, signatures, line spans, __all__, re-export statements, the dataclass and pydantic predicates), unparse.go reproduces ast.unparse's rendering of an expression, and handlers.go does everything a reader sees -- which symbols are skipped, how the Markdown is assembled, how docstring sections are formatted, which parameters count as documented.
A parse is cached per file for the life of the extractor, because one page asks about the same module several times. A file that cannot be read or does not parse answers empty or renders an error marker, as it always has: that is a property of the file, not of the machine.
#Extractor
type Extractor structExtractor reads Python source through an in-process tree-sitter parse.
#FirstSyntaxErrorLine
func FirstSyntaxErrorLine(source []byte) (line int, failed bool, err error)FirstSyntaxErrorLine reports whether source fails to parse, and the one-based line the first failure sits on.
It is the syntax question on its own, for a caller that wants nothing else out of the file -- the documentation-example check, which asks it of a fenced code block rather than of a module.
Indentation is not a failure here. The grammar admits a fragment lifted out of a function, an unexpected indent, a dedent matching no outer level and a block that was never indented at all, where CPython raises IndentationError for each. The example check exempted those anyway, so the exemption is now structural rather than a case it has to recognize.
#New
func New() extractors.ExtractorNew builds the Python extractor. Reading a Python file is a file read and a parse, and neither spawns anything.
#Extractor.Detect
func (e *Extractor) Detect(dir string) boolDetect reports whether dir carries a Python project's marker files.
#Extractor.FileExtensions
func (e *Extractor) FileExtensions() []string { return []string{".py"} }FileExtensions is the single extension Python owns.
#Extractor.ResolvePath
func (e *Extractor) ResolvePath(pathArg string, sourcePaths []string, baseDir string) stringResolvePath resolves a dotted module path, a package path or a file path to a .py file.
#Extractor.PublicSymbols
func (e *Extractor) PublicSymbols(file string) ([]string, error)PublicSymbols lists the symbols a module exports.
A module that defines __all__ as a literal list or tuple of strings is taken at its word -- those names ARE its public API, underscore-prefixed ones included. Otherwise the heuristic applies: top-level functions and classes whose name does not begin with an underscore.
#Extractor.ModuleDocstring
func (e *Extractor) ModuleDocstring(path string) (string, error)ModuleDocstring is a module's own docstring, with soft-wrapped prose joined.
#Extractor.SymbolDetails
func (e *Extractor) SymbolDetails(file, symbol string) (*extractors.SymbolDetails, error)SymbolDetails reports one symbol's parameters and return value.
A dotted name selects a member of a class (MyClass.my_method); a plain name is looked for among the top-level functions and classes first, and then among each class's methods, in the order the file declares them.