Reorganizing tools,
buildlink & build phases

Joerg Sonnenberger

The NetBSD Foundation

Outline

• On licenses
• On patterns
• On dependencies
• On build front ends

Improve licenses

• RESTRICTED, NO_*_ON_* often useless
• Real requirement:
  • Put binary on public FTP?
  • Put source on public FTP?
  • Put binary on internal FTP?
  • Put source on internal FTP?
• Make this a per-license statement, upload can use it

Current patterns

• Different rule sets:
  • Dewey for ranges
  • csh-style {} for alternatives
  • glob-like wildcards
• No formal and precise definition for Dewey
• Alternatives are hard to grep
• Best match for alternatives?
• glob wildcards too generic

New pattern rules

• Extends Dewey
• Components:
  • Package base name
  • Pairs of operator/version
  • Special prefix match operator
  • Alternatives with |
• Formalise version comparision
• Formalise best match rules
• Mostly automatic conversion

Merging patterns

• Selecting a subset, changing order
• Idea: () makes a pattern relative
• Merging is left-to-right
• Normal patterns restrict, relative patterns don't

Sources of dependencies

• DEPENDS
• BUILD_DEPENDS
• buildlink3
• USE_TOOLS

Problems with current approach

• Different syntax
• Site local overrides?
• buildlink3.mk is slow

Solutions for dependency handling

• builtin.mk:
  • Compute builtin.mk results once, cache them
  • Dramatically reduces number of forks
• Drop separation of dependencies, follow USE_TOOLS style

Generalising USE_TOOLS

• Use modifiers:
  • :build, :run, :test
  • buildlink depends on the package
  • tool wrapper depends on the package
• DEPEND_PATHS.foo lists package locations
• DEPEND_PATTERNS.foo lists patterns
• Figure out best match

Implementation ideas

• mk vs external file not decided yet
• mk file for DEPENDS/BUILD_DEPENDS like 5k lines
• 10% parse time for minimal packages
• Build options handled by b3 subscript

The build right now

• "make depends"
• "make package"
• "make package-install"

Problems with the build

• Local knowledge: what to build/install
• Arising conflicts, mixed source/binary installation
• No good way to parallelize

Bulk builds

• Have logic to find out what is available and needed
• Could compute conflicts before
• Can parallelize
• Why not use it?

The end

Questions?