CIP-0194? | Builtin pattern matching in UPLC

[SeungheonOh]

Proposal for adding a new UPLC AST node: Match. Match enables matching complex and nested builtin value structure, namely Data values like script context, without builtin function invocation overhead.

A working prototype has been implemented: IntersectMBO/plutus#7852

Rendered

[rphair]

@SeungheonOh thanks very much for documenting your implementation with a CIP. The in-progress work I think makes it definite we would assign a CIP number in Triage at the next CIP meeting (https://hackmd.io/@cip-editors/140).

@zliu41 @ana-pantilie @colll78 @Quantumplation @fallen-icarus if you could review the CIP technical presentation any time before or after its confirmation, that would be great.

@kwxm it looks like performance is already also well documented here, but please likewise feel free to contribute about that presentation as well.

[colll78]

Strongly support this CIP, approach is sound and a massive improvement.

[michaelpj]

This is rather different to how costing works for other terms, where a cost is associated with the reduction rule for a term.

Here we either have one very big reduction rule, or we would need a way to evaluate the match incrementally. But the possibility of backtracking in the matching makes this difficult.

[SeungheonOh]

As implemented, latter is the case. Match is being costed incrementally. Backtracking doesn't really pose any issue since it also increments the costs for failed cases, so it doesn't have to do anything complicated for failed branches.

I wrote this in a confusing way. Not all four steps being proposed are CEK steps. Only one is for CEK itself, and other three are steps used within the matcher.

[michaelpj]

I would like to see a more declarative and less operational description of how this works. Typically we give a reduction rule for terms.

[michaelpj]

This is a lot of syntax. Probably this increases the size of the base language by 30%+!

[michaelpj]

Are the handlers evaluated strictly before evaluating the match?

[SeungheonOh]

No, none but the matching handler will be evaluated. Only patterns that comes before the matching case will be inspected.

I'll try to clarify this section

For example:

(match (con integer 2)
  (pattern (integer 0) (error)) -- Pattern inspected, but not evaluated
  (pattern (integer 1) <expensive work>) -- Pattern inspected, but not costed/evaluated
  (pattern (integer 2) (con integer 42))
  (pattern (integer 3) (error)))

-- > (con integer 42)

[michaelpj]

yes, this seems orthogonal

[michaelpj]

So... why is this faster? Operationally speaking. We are fundamentally doing a very similar process. Is it just that we skip some expensive parts of the builtin machinery? It seems odd that we're able to do the same thing but faster!

[SeungheonOh]

It's mainly the builtin calling overhead. Calling builtin requires several extra CEK frames. In most cases, it needs Apply and Builtin and for some it also requires extra Forces. These need to happen per each layer of nested values and these overheads turns out to be more expensive than actually deconstructing values themselves.

Match removes all of the overhead only running on very lean pattern syntax and doing value deconstruction directly removing significant amount of the overhead. This is exactly the same reason why IfThenElse is slower(almost 80% iirc!) than Case when casing on boolean.

Also, Match gives options to match pattern without capturing value, this reduces CEK steps even more. For instance, if you just want to check if D.I 10 is integer data or not, currently, you'd do chooseData (D.I 10) ... (\i -> ...) ... where builtin not only have to match on the Data constructors, but it also have to capture 10 and apply to the handler even though the value is not needed. This also incurs more extraneous cost. Match on the other hand can do (pattern (data-i (wildcard) <arity 0>) which doesn't have to dispatch apply at all. This is why the performance gap is bigger when fewer captures were performed in the benchmarks

[michaelpj]

Should we allow matching on other values? Notably, what about con values? If we're going to add pattern-matching it's a shame not to get it on datatype values.

[SeungheonOh]

Do you mean Constrs? I didn't add it because I initially thought it could break typing in TLPC/PIR. However, looking again, it seems reasonable to add something like DefaultPatternConstr Word64 (Vector DefaultBuiltinPattern).

I'm not entirely sure performance implication to adding this. This would definitely complicate the implementation(which can consequently make it slower) because having pattern for Term.Constr means now it needs to carry and match on Term not just values.

[michaelpj]

Why not?

DefaultPatternList { headPattern :: DefaultBuiltinPattern, tailPattern :: DefaultBuiltinPattern }

you'd need DefaultPatternNil, perhaps, to terminate them. But I'm a bit unsure about using this prefix/exact shape descriptor rather than a pattern AST that follows the shape of the datatypes.

[SeungheonOh]

This would make pattern interpretation slower since for matching long list, it would need to match on DefaultPatternList repeatedly. I figure it's better to prefer structure that is more optimal to run since this won't be user facing interface anyways.

[wadler]

The structure is odd compared to the usual pattern matching. Your form:

(match scrutinee (pattern pair (pair (bind) (bind)) (bind))) (lam x (lam y (lam z body))))

Usual form:

(match scrutinee (pattern pair (pair x y) z)) body)

In both cases, the body refers to x, y, z as bound variables.

The first form reuses the existing lambda machinery, while the second does not, so I guess the first is slightly easier to implement. But nowhere is the usual form mentioned or compared with. I think there is good reason to believe that the second can be implemented more efficiently (although it may take more work to do so). So a comparison is essential.

Typically, a compiler gets rid of nested pattern matching and only uses a shallow case to look at the top-level structure. Your motivation is that the existing mechanism which does this can (if misused) lead to partial applications, but an easier way to fix that is to supply an arity with each case rather than nested pattern matching. Did you compare with that alternative? You need to add this comparison the the CEP to justify the design.

Michael noted that for lists matching on cons and nil instead of prefix is standard. Sungheon responded that the prefix design is more efficient, which sounds plausible. But this needs to be documented with performance numbers in the CEP.

If match is included it should also apply to sum-of-product types. (I think Michael makes a similar point.)

[rphair]

@SeungheonOh this was declared a candidate at the CIP meeting today, continuing the prior confirmations. Going forward we will generally await the settlement of current & future review like what @michaelpj has already provided (please "resolve" the points that seem to be settled, since this might not always be clear to editors).

Once generally settled please feel free to explicitly point this out in a comment, so we can make sure it moves on to final / editorial review. In the meantime please rename the containing directory to CIP-0194 and update the "Rendered" link in your OP. 🎉

[SeungheonOh]

Thank you, @wadler and @michaelpj, for the review!

About binding captures directly

It seems that a similar design was considered when SOP terms were added in CIP-85, but was ultimately not adopted. CIP-85 mentions that this optimization initially produced an improvement of around 10%. However, another optimization reduced the realized improvement from direct binding to only around 3%, which was considered a small enough difference to justify preferring the simpler implementation.

I am not sure exactly what that other optimization was, or whether there were additional reasons for not using direct binding in CIP-85. Maybe @michaelpj can provide more context here.

I tested direct binding for Match as well, and observed an approximately 10–15% improvement in execution time. Unlike the proposal discussed in CIP-85, this does not require a new alt annotation specifying the arity of each handler, because the pattern syntax already determines how many variables are bound. This appears to be a worthwhile optimization, even considering the additional complications for TPLC and PIR. I will include the experiment in the CIP and pursue this approach.

About (re)using Case with arity/pattern attached to each handler

Initially, I found this to be the most straightforward change with the smallest UPLC footprint. It would have looked something like this:

(case (con data (Constr 0 [I 1, List [I 2, I 3]]))
  (pattern (data-constr 0 (bind) (wildcard)) (lam x ...)))

-- `pattern` itself is a new AST node: `Pattern pat term`.
-- Having `pattern` as a standalone AST node would also allow it
-- to be used independently of `Case`, as a partially defined
-- pattern-matching function:

[(pattern (data-constr 0 (bind) (wildcard)) (lam x ...))
 (con data ...)]
-- We can use direct bind here too and it would look even more natural:
[(pattern (data-constr 0 x (wildcard)) x)
 (con data ...)]

-- If we are only attaching arity, it would look like

(case (con data (Constr 0 [I 1, List [I 2, I 3]]))
  (arity 2 (lam x y ...))   -- Constr 0
  (arity 3 (lam x y z...))) -- Constr 1

-- Unlike `pattern`, this would not allow skipping constr 
-- indices (for example, only matching on `Data.Constr 2`) and 
-- would force all elements of the list to be bound. 

This would allow Case to be reused for pattern matching, while pattern could also function independently as a pattern-matching lambda.

Unfortunately, there are few issues with this approach.

First, the CEK machine does not evaluate the branch handlers of a Case node until a branch has been selected. Associating a pattern—or even only an arity—with each handler would therefore require the machine to inspect the AST of each branch before selecting one. I have not benchmarked this design(let alone implement), so I cannot make any claim, but inspecting AST nodes outside the CEK machine's main compute/return loop generally has a negative performance impact. This idea was discarded early on because of this concern.

Second, supporting general patterns through Case would conflict with the existing casing behavior for builtin constants. Cases over lists, integers, and booleans currently use fixed branch positions. Supporting examples such as the following would therefore require the existing builtin casing mechanism to be deprecated:

(case (con (list integer) [1, 2, 3])
  (pattern (list (bind) (wildcard) (bind)) ...)
  (pattern (list (bind) (wildcard)) ...))

Alternatively, case would need to determine whether it was operating in pattern-matching mode or conventional casing mode, but this sounds like a nightmare. Deprecation of existing builtin casing behavior may not be a decisive problem though, since introducing new Pattern node already requires a language-version bump, allowing the older builtin constant casing behavior to be deprecated smoothly.

Lastly, using Case for arbitrary patterns(even shallow) would need costing mechanism of Case to be changed. I don't think this is a big blocker; it should be relatively straightforward. However, it requires changing how costing behaves in existing Case node which might not be favorable.

I haven't explored this options deeply. Please let me know if this approach is more natural and is worthwhile to test out. I discarded this idea rather early on, so some(or all) of the issues I mentioned might not even be an issue at all.

About shallow matching

I implemented shallow matching and benchmarked it against matching the same nested structures directly. Contrary to my initial expectation, nested matching did not provide a significant performance improvement over shallow matching.

Neither approaches are uniformly faster. The differences were generally within approximately 10% of CPU execution time. Shallow matching performed better for wider structures, whereas nested matching performed better when matching values deeply within heavily nested structures.

I spent some time to figure out how to cost nested patterns and it turned out to be very difficult to come up with one that won't overshoot all pattern types. This is unlike Shallow matching which provides a substantially more straightforward costing model as it does not need any of the incremental costing mechanism.For simplicity of costing and competitive performance, shallow matching seems to be a better direction. I will collect additional benchmark data and update the CIP to use shallow matching instead.

[zliu41]

I need to review it in more detail, but here are some initial comments:

• It's worth elaborating why "A handler with the wrong number of arguments may therefore partially apply instead of failing" is problematic
• The baseline in your benchmark should use the new dropList builtin, if not already
• It would be interesting to know how much this approach narrows the performance gap between Data encoding and SOP encoding. Can it close the gap entirely? If not, why?

[zliu41]

Regarding the long list of patterns, I'd suggest shortening them as much as you can.

• The vast majority of data types are encoded using Constr and List. Optimizing for these two should deliver most of the value.
• DefaultPatternByteString: the usefulness is unclear. And it also needs an equalsByteString-like cost model, which adds complexity.
• DefaultPatternBool can already be expressed with casing on Bool.
• DefaultPatternInteger: why only Int64? Is it to make costing easier? But DefaultPatternByteString already requires a complex costing mechanism. And its usefulness is unclear.
• DefaultPatternDataI and DefaultPatternDataB: the usefulness is unclear.
• Your pattern language allows expressing patterns that make no sense, such as DefaultPatternDataI (DefaultPatternByteString "foo").

[ana-pantilie]

What is the reason behind delegating variable binding to the handler instead of using a matching mechanism which constructs substitution contexts?

Concretely, if I understand this correctly, in the following example:

match scrutinee
  (pair capture capture)
  handler

handler needs to be of the form \x -> \y -> body. If it's not, then by the typing rules in this CIP, the program is not a valid TPLC program. One consequence is a new class of potentially "ill-formed" programs in UPLC, since UPLC is untyped.

However, if match also introduced a substitution context, for example:

match scrutinee
  (pattern (pair (var x) (var y)))
  handler

Then it would be up to the matching algorithm to produce a valid substitution for x and y, there's no more requirement for the handler to be of a specific form. That would, of course, require the algorithm to produce fresh variables in the context in order to avoid shadowing.

I think it would also make matching more expressive:

(\scrut -> 
  match scrut
    (pattern (pair (var x) (var x)))
    handler
) (1, 2)

After beta-reduction, the matching algorithm would not be able to find a valid substitution for x, therefore the program would not continue execution at that stage. Otherwise, in the current proposed design, a user would have to manually check inside the handler that x == y, which might be more expensive since there's another builtin function call involved.

Of course, my suggestion would require a more complex costing mechanism to account for the substitution construction. But I think that using a proper matching algorithm would make it more future proof, in case we want to add new features to match in the future.