Summary

This proposal adds a set of new virtual machine (VM) operations which enable BCH contracts to efficiently access details about the current transaction like output values, recipients, and more – without increasing transaction validation costs.

Deployment

Deployment of this specification is proposed for the May 2022 upgrade.

Motivation

Since the introduction of OP_CHECKDATASIG (2018-11-15), it has been possible to implement practical Bitcoin Cash covenants – contracts which validate properties of the transaction in which the contract funds are spent.

Covenants enable a wide range of new innovation, but the strategy by which they are currently implemented is extremely inefficient: most of a transactions contents must be duplicated for each input which utilizes a covenant. In practice, this doubles or triples the size of even the simplest covenant transactions¹, and advanced covenant applications quickly exceed VM limits such as the maximum standard unlocking bytecode length (A.K.A. MAX_TX_IN_SCRIPT_SIG_SIZE) of 1,650 bytes. This severely limits the extent of current BCH covenant development and usage.

Covenants are currently implemented using a workaround in which the same signature is checked by both OP_CHECKSIG and OP_CHECKDATASIG. The OP_CHECKSIG confirms the signature is valid for the current transaction, and the OP_CHECKDATASIG allows the contract to validate that the signature covers a particular preimage provided in the unlocking bytecode. If both checks are successful, the contract can trust that the preimage provided to OP_CHECKDATASIG is the genuine signing serialization of the current transaction. By further inspecting this preimage, the contract can validate other properties of the transaction.

Because nodes are already required to have full access to a transaction’s contents and UTXO information during transaction validation, native introspection opcodes can be introduced without impacting existing transaction validation costs.

Benefits

Adding native support for introspection operations would make covenants simpler, safer, and more efficient, enabling innovative new use cases.

Simpler, Safer Contracts

While contracts using the current covenant workaround must carefully validate all provided transaction information, contracts using native introspection operations could safely rely on information returned by the VM without further validation. This significantly reduces contract size and complexity, making contract development and security review easier.

Reduced Transaction Sizes

By eliminating the need for a wasteful double-signature checking workaround, covenant transaction sizes can be reduced by hundreds of bytes, making them as efficient as other common transaction types. This would reduce wasted network bandwidth and lower transaction fees for all covenant applications.

Because covenant applications cover a large variety of common transaction types – including scheduled and recurring payments – improved efficiency in covenant transactions would significantly improve overall throughput of the BCH network.

Enable Innovation

By eliminating waste in covenant contracts, native introspection operations can extend the scope of possible applications which can be developed within current VM limits. This would enable far more innovation on Bitcoin Cash, without increasing node validation costs.

Costs & Risk Mitigation

The following costs and risks have been assessed.

Node Upgrade Costs

This proposal affects consensus – all fully-validating node software must implement these VM changes to remain in consensus.

These VM changes are backwards-compatible: all past and currently-possible transactions remain valid under these new rules.

Other Software Upgrade Costs

While most wallets, indexers, and other software will not require updates to support this proposal (beyond upgrades to any supporting node infrastructure), updates will be required to some libraries, transaction compilers, and other software which implements the BCH VM (e.g. Bitauth IDE).

Assignment of Opcodes

The specified operations would occupy 15 codepoints within the BCH VM instruction set. This represents a notable portion of the remaining 81 unused codepoints (80, 98, 101, 102, 137, 138, 176, 179-185, 189-255). However, we consider this to be a prudent use of these codepoints.

Technical Specification

A range of Bitcoin Cash VM codepoints is reserved for transaction introspection operations – 0xc0 (192) through 0xcf (207) – and a new set of 15 operations allows contracts to introspect all available transaction and evaluation state.

Nullary Operations

The following 6 operations consume no items and push a single result to the stack. If the item to be pushed to the stack is longer than the stack item length limit (A.K.A. MAX_SCRIPT_ELEMENT_SIZE), immediately fail evaluation. If the additional stack item would cause the stack to exceed the maximum stack size (A.K.A. MAX_STACK_SIZE), immediately fail evaluation.

1. OP_ACTIVEBYTECODE pushes the serialized bytecode for the instructions currently under evaluation beginning after the most recently executed OP_CODESEPARATOR and continuing through the final instruction. If no OP_CODESEPARATOR has been executed, OP_ACTIVEBYTECODE pushes the full, serialized bytecode for the instructions currently under evaluation. (In the Satoshi implementation, this is simply the CScript contents passed to the current EvalScript.)
2. This behavior matches the existing behavior of the BCH VM during P2SH evaluation – once the P2SH pattern is matched, the remaining stack is copied, and the VM begins evaluation again with the P2SH redeem bytecode set as the new active bytecode. (In the Satoshi implementation, this P2SH redeem bytecode is passed as a CScript to a new execution of EvalScript.)

Unary Operations

The following 9 operations pop the top item from the stack as an index (Script Number) and push a single result to the stack. If the consumed value is not a valid, minimally-encoded index for the operation, an error is produced. If the item to be pushed to the stack is longer than the stack item length limit (A.K.A. MAX_SCRIPT_ELEMENT_SIZE), immediately fail evaluation.

1. This is the byte order produced/required by all BCH VM operations which employ SHA-256 (including OP_SHA256 and OP_HASH256), the byte order used for outpoint transaction hashes in the P2P transaction format, and the byte order produced by most SHA-256 libraries. For reference, the genesis block header in this byte order is little-endian – 6fe28c0ab6f1b372c1a6a246ae63f74f931e8365e15a089c68d6190000000000 – and can be produced by this script: <0x0100000000000000000000000000000000000000000000000000000000000000000000003ba3edfd7a7b12b27ac72c3e67768f617fc81bc3888a51323a9fb8aa4b1e5e4a29ab5f49ffff001d1dac2b7c> OP_HASH256. (Note, this is the opposite byte order as is commonly used in user interfaces like block explorers.)

CHIP Integration: Bigger Script Integers

Because VM arithmetic operations currently limit Script Number inputs to signed 32-bit integers, the maximum results of OP_UTXOVALUE and OP_OUTPUTVALUE which can be used in practical contracts is 2147483647 satoshis (~21.47 BCH). For this reason, it is recommended that CHIP: Bigger Script Integers be deployed before or with this CHIP.

This CHIP can be deployed without CHIP: Bigger Script Integers, and no direct integration is required between the two CHIPs. If CHIP: Bigger Script Integers is not deployed with or before this proposal, the above specified operations must still behave as specified. (OP_UTXOVALUE and OP_OUTPUTVALUE may push Script Numbers to the stack which are up to 8 bytes in length and therefore could not be used as inputs to arithmetic operations. Until some arithmetic range upgrade, these operations would likely only be used in practice for values which can be encoded in 4 bytes or less.)

CHIP Integration: Version 3 Transaction Format (PMv3)

If this proposal is adopted with or after CHIP: Version 3 Transaction Format (PMv3), the OP_INPUTDETACHED unary introspection operation must also be activated:

See Interaction with PMv3 for rationale.

Additionally, the description of OP_INPUTBYTECODE can be clarified to confirm that its result is always the input’s unlocking bytecode (regardless of whether or not the proof is detached):

Rationale

This section documents design decisions made in this specification.

Ordering Operations by Arity

This proposal groups new introspection operations by arity: all operations which consume no stack values precede all operations which consume an index value from the stack.

This simplifies some implementations, allowing operations of like arities to more easily reuse logic. For example, in the Satoshi implementation, all unary numeric operations are handled in the same fallthrough switch case, allowing each operation to reuse the same stack pop and error handling logic.

Beyond arity, operations are arranged within the instruction set to match the order in which their results appear within the P2P transaction format.

Operation Set Completeness

The operations specified in this proposal expose the most raw form of every static property of transaction and evaluation state within the current VM model.

This strategy necessarily incorporates operations which may initially seem to duplicate existing functionality, notably OP_TXLOCKTIME and OP_INPUTSEQUENCENUMBER (many use cases for these operations can also be accomplished using the existing OP_CHECKLOCKTIMEVERIFY and OP_CHECKSEQUENCEVERIFY, respectively). However, contract authors are not always able to reverse the order of a construction to make use of these operations. This limitation is particularly acute in covenant use cases: where other limitations prevent a contract from inverting a construction (or even providing the expected result of that operation via unlocking bytecode), some use cases are likely impossible without these functionally-similar introspection operations.

Notably, the existing instruction set includes OP_SUB even though OP_ADD can be indirectly used (backwards) to accomplish many of the same validations. The existence of inverse and functionally-similar operations is important for both accommodating new use cases and optimizing contracts.

By implementing introspection in a single deployment as a complete, logical set of operations – rather than by piecemeal activation as contract authors lobby for operations which are important for their use cases – we allow new innovation to take place primarily in “userspace” and limit future technical debt.

Note: introspection of dynamic VM state – e.g. current instruction pointer index, the current state of various cumulative limits, etc. – is intentionally excluded from this specification.

This approach avoids consensus standardization of various implementation details which could complicate some implementations and requires further research and justification. Instead, this proposal incorporates only introspection operations for previously consensus-standardized, static properties of transaction and evaluation state which are already available to contracts via the OP_CHECKSIG + OP_CHECKDATASIG hack.

Differences Between OP_ACTIVEBYTECODE and OP_INPUTINDEX OP_UTXOBYTECODE

The operations specified in this proposal include partially-overlapping functionality in one rare case: the results of OP_ACTIVEBYTECODE and OP_INPUTINDEX OP_UTXOBYTECODE are equal in non-P2SH contracts (i.e. spending the output of a non-standard transaction). However, in the more common (standard) P2SH case, OP_ACTIVEBYTECODE produces the raw redeem bytecode, while OP_INPUTINDEX OP_UTXOBYTECODE will produce only the already-hashed result in the P2SH-template locking bytecode (OP_HASH160 OP_PUSHBYTES_20 <redeem_bytecode_hash> OP_EQUAL). This distinction is very important in covenant applications: even in simple cases, it can save many hundreds of bytes by avoiding the need to push a duplicate copy of the covenant’s redeem bytecode (to construct the covenant’s next output). While OP_ACTIVEBYTECODE is very cheap for the VM (typically a simple buffer copy), this single optimization reduces many covenant sizes by 50% or more.

To demonstrate, the following script produces a 1 for P2SH contracts, but a 0 for non-P2SH contracts (spending a non-standard transaction): <OP_HASH160 OP_PUSHBYTES_20> OP_ACTIVEBYTECODE OP_HASH160 <OP_EQUAL> OP_CAT OP_CAT OP_INPUTINDEX OP_UTXOBYTECODE OP_EQUAL.

Alternatively, this same functionality could be provided by modifying OP_UTXOBYTECODE to only return the “active” bytecode of each UTXO (i.e. for P2SH contracts, only the raw redeem bytecode), but this would require a significantly more complex implementation: a transaction’s inputs would need to be evaluated in two phases, caching each P2SH input’s redeem bytecode for access by other inputs where needed. Additionally, information would still be lost unless another introspection operation were made available for contracts to judge which inputs spend a P2SH UTXO (e.g. a unary OP_ISP2SH).

An advantage of that more complex solution over the proposed OP_ACTIVEBYTECODE solution could be more efficient access to the raw redeem bytecode of sibling inputs (other than the “active” one). While this might be useful in optimizing some highly-advanced covenants, the proposed solution offers a far simpler consensus implementation and slightly more efficient contracts in the common case (1-byte OP_ACTIVEBYTECODE vs. 2-byte OP_INPUTINDEX OP_UTXOBYTECODE).

The complexity of this issue ultimately stems from the implementation chosen for BIP16/P2SH, which bifurcated VM evaluation into two modes: standard and P2SH. This complicated the operational model of the VM and is particularly disruptive to the logical consistency of introspection operations between the two modes. The chosen solution focuses on simplicity and optimizing the most common cases, rather than a more complex solution which might optimize more cases. If usage of sibling redeem bytecode becomes common in real world applications, future upgrades could offer more targeted optimizations for those cases.

Introspection Operation Range

This proposal leaves a 3 codepoint gap between the last defined operation in the current BCH instruction set (OP_REVERSEBYTES, 0xbc/188) and the beginning of the proposed introspection operation range (OP_INPUTINDEX, 0xc0/192).

This breaks from the recent precedent (since Nov. 2018) of adding each new opcode at the next-available codepoint. (After OP_NOP10/0xb9, OP_CHECKDATASIG/0xba, OP_CHECKDATASIGVERIFY/0xbb, and OP_REVERSEBYTES/0xbc were appended to the ISA in activation order, rather than being grouped logically with similar operations). While this may appear to complicate logic for detecting invalid codepoints (from op > FIRST_UNDEFINED_OP_VALUE to isUndefined(op)), it is equivalent in practice, as validators are optimized for evaluating valid bytecode (the common case). For example, in the Satoshi implementation, a switch statement is used to switch between valid operations – invalid opcodes only return their error after the VM iterates through the full list of possible valid opcodes.

By leaving gaps in the ISA, future upgrades may group similar operations nearer to each other. In the long term, this may produce a more coherent ISA than one ordered only by opcode activation time.

In the context of this proposal, the proposed gap offers up to 3 codepoints for use either by formatting operations (logically grouped with OP_REVERSEBYTES) or for use by additional nullary introspection operations (if the VM model is extended to incorporate new transaction-level primitives or metadata). Likewise, the 2 remaining codepoints within the introspection range (0xce and 0xcf) provide space for additional unary introspection operations.

Exclusion of Aggregation Operations

With the partial exception of OP_ACTIVEBYTECODE, this proposal includes only operations which expose raw state – derived state must be computed using other VM operations. While this simplifies the proposal to a clearly defined set of “primitive” operations, certain useful, derived state elements are not yet possible to compute in all cases. For example, since the VM does not include any construction for looping or iteration, aggregate values like “total input value” and “total output value” are not always possible for contracts to compute.

While it would be possible to define aggregate operations like OP_TXINPUTVALUE and OP_TXOUTPUTVALUE to make this derived state accessible to all contracts, this proposal considers standardization of any derived state to be premature. Even with piecemeal implementation of some derived state operations, some important types of aggregated/derived state cannot be reasonably standardized, e.g. “public keys referenced by all P2PKH outputs”. For contracts to support extracting this information, it’s necessary to add support for generalized loops/iteration to the VM.

For this reason, this proposal does not include any aggregate operations; contract authors should instead look to proposals like CHIP: Bounded Looping Operations for this support.

With CHIP: Bounded Looping Operations, a OP_TXINPUTVALUE equivalent would require only 14 bytes:

<0> <0>
OP_BEGIN                            // loop (re)starts with index, value
  OP_OVER OP_UTXOVALUE OP_ADD       // Add UTXO value to total
  OP_SWAP OP_1ADD OP_SWAP           // increment index
  OP_OVER OP_TXINPUTCOUNT OP_EQUAL
OP_UNTIL                            // loop until next index would exceed final index
OP_NIP                              // drop index, leaving sum of UTXO values

While not included in this proposal, future proposals may introduce aggregate operations either as a partial alternative to looping operations or as an optimization strategy for common aggregations.

Interaction with PMv3

The PMv3 version 3 transaction format proposal would add two fields which don’t yet exist in the v1/v2 transaction formats: detached signatures and detached proofs. If adopted with PMv3, this proposal recommends only one additional introspection operation (OP_INPUTDETACHED).

Detached signatures are themselves inherently reusable across multiple inputs – rather than a specialized introspection operation, many use cases can reference a particular detached signature directly. Also, because all detached signature-covered transaction elements are themselves natively introspectable, there’s little value in an operation which makes the OP_CHECKSIG + OP_CHECKDATASIG workaround more efficient for detached signatures. As such, this proposal does not recommend any new introspection operation(s) for detached signatures.

Detached proofs add only one new bit of information: whether or not each input is detached (true or false). Access to this information is valuable to many public covenants; if a covenant requires proofs to be detached for transaction sizes to be kept in check, the covenant needs to prevent malicious users from interacting with the covenant using transactions with non-detached proofs (leading to increased covenant transaction costs and even disabling of poorly-designed public covenants).

To allow inspection of this information, the OP_INPUTDETACHED unary operation is proposed, and the OP_INPUTBYTECODE operation is clarified: (If the selected input uses a detached proof, the pushed value is the unlocking bytecode content of the detached proof.).

This strategy makes validation of transaction shapes trivial: OP_INPUTINDEX OP_INPUTDETACHED OP_VERIFY would ensure that the current contract is using a detached proof. It also maintains a consistent behavior for OP_INPUTBYTECODE (for both detached and standard proofs) – the pushed results of OP_INPUTBYTECODE would always be the unlocking bytecode regardless of whether or not the proof is detached.

Note, the un-hashed result of OP_INPUTBYTECODE is most useful to practical contracts, but it’s also trivial to reproduce any detached proof hash under this scheme: <N> OP_INPUTBYTECODE OP_HASH256.

Use of Single-Byte Opcodes

Earlier proposals attempted to use fewer opcodes by either introducing all introspection operations as a single opcode or by introducing multi-byte opcodes. However, deeper analysis indicates that the BCH instruction set might never reach 255 operations within the current VM model (future VM models are likely to have their own independent set of operations, rather than naively extending the current instruction set).

While it is likely wise to reserve both the remaining OP_NOP4-OP_NOP10 range and the range from 0xf0 to 0xff for future multi-byte opcodes (leaving room for ~6000 2-byte opcodes), jumping to multi-byte opcodes would be a premature optimization, with uncertain future benefit and immediate negative consequences.

As such – and because introspection operations are critical to all covenant use cases – this proposal specifies the proposed operations as standard, single-byte opcodes.

Push-Only Limitation of Unlocking Bytecode

For clarity, this proposal does not modify the existing limitation (present since BCH_2018-11-15) preventing transactions from including “non-push” operations – all operations at codepoints greater than 96 (0x60) – in unlocking bytecode. This limitation also prevents all operations specified in this proposal from appearing in unlocking bytecode.

This limitation is maintained to prevent a third-party malleability vector: any non-push operation can be replaced in a malleated transaction with an equivalent push operation pushing the computed result of the non-push operation. While all third-party malleability vectors would be eliminated by PMv3’s detached signatures, this CHIP leaves any loosening of the non-push limitation to future proposals.

OP_ACTIVEBYTECODE Support for OP_CODESEPARATOR

In earlier versions of this proposal, the result of OP_ACTIVEBYTECODE did not account for the last executed OP_CODESEPARATOR. However, the current behavior is considered more conservative (OP_ACTIVEBYTECODE returns the active bytecode beginning after the last evaluated OP_CODESEPARATOR). Contracts which don’t utilize OP_CODESEPARATOR are unaffected by this change, contracts which utilize OP_CODESEPARATOR can access either behavior (an earlier result of OP_ACTIVEBYTECODE can always be saved for use after OP_CODESEPARATOR is called), and some unusual contracts may be able to utilize the new behavior to optimize their bytecode size or runtime requirements.

Notably, the index of the last executed OP_CODESEPARATOR is available using the OP_CHECKSIG + OP_CHECKDATASIG workaround, so providing access to this state via OP_ACTIVEBYTECODE is also valuable for operation set completeness.

Implementations

• Bitcoin Cash Node (BCHN) Merge Request 1208

Test Cases

• Bitcoin Cash Node (BCHN) Native Introspection Tests

Evaluation of Alternatives

No alternative native introspection proposals are currently public, but alternative designs for several components of this proposal have been documented in the Rationale section.

Several past proposals have informed the design of this proposal:

• The initial OP_PUSHSTATE proposal used a single opcode and templated approach. (The proposal was withdrawn in favor of this CHIP.)
• An implementation of a closely-related OP_PUSH_TX_STATE exists for the Bitcoin Unlimited node software in the NextChain project.
• A multi-byte fork of the OP_PUSHSTATE proposal introduced the idea of unary operations which select information by input/output index.

Delay or rejection of this proposal may incur the following costs:

• Continued use of the inefficient workaround increases the size of the blockchain could negatively impact initial block download times and storage requirements.
• The complexity of the current workaround increases development costs and raises the barrier to entry for new developers.
• Because the current workaround is difficult to implement securely, implementation problems could result in loss of profits and reputational damage, both for the implementing company and the Bitcoin Cash ecosystem.
• Some applications and use cases will be impossible without native introspection, and competitors may outcompete BCH for these users.

Copyright Notice

Copyright (c) 2021 GeneralProtocols / Research

Permission is granted to copy, distribute and/or modify this document under the terms of the MIT license.