Benchmarking for your Substrate pallets | Substrate Seminar
Why Substrate and Polkadot Stack Exchange recommends it
Substrate Stack Exchange answerers and askers point people to it as the go-to long-form introduction to pallet benchmarking and weight calculation.
Recommended by 3 people across 4 different months, 2022 to 2022.
What it is
A roughly three-hour Substrate Seminar from Parity Tech on benchmarking FRAME pallets, covering how to measure extrinsic execution cost and turn it into weights. It is a long, detailed walkthrough from the Parity team that also touches on database behaviour such as RocksDB outliers.
What people said
Recommends Shawn Tabrizi · 2022-02 · Substrate and Polkadot Stack Exchange · on “How to generate `WeightInfo` for pallets?”
On a production chain, you should always re-calculate your weights by running benchmarks on the final runtime. The point of benchmarking is that your runtime can be configured in different ways, and your blockchain may have different assumptions than others. So running benchmarks gives you real results for exactly how you developed your runtime. The benchmarking tooling is enabled in Substrate based chains by compiling with the feature flag --features=runtime-benchmarks . And can be run like so: ./target/release/substrate benchmark \ --chain dev \ # Configurable Chain Spec --execution=wasm \ # Always test with Wasm --wasm-execution=compiled \ # Always used `wasm-time` --pallet pallet_balance
Recommends Shawn Tabrizi · 2022-09 · Substrate and Polkadot Stack Exchange · on “Examples of Substrate Benchmarking”
the official documentation Seems like a pretty good place to learn about benchmarking to me: [LINK] Beyond that: 3 Hour Video on Benchmarking: [this video] Nearly every pallet in FRAME folder has some benchmarks you can use as reference: [LINK]
Recommends Nuke · 2022-10 · Substrate and Polkadot Stack Exchange · on “How setup Storage for a proper benchmarking?”
The state matters primarily for reading and writing to the merkelized data structure operations database FRAME abstracts from you in familiar things like storage maps. But overhead in database operations because of that merkelized stuff is typically the largest weight contribution typically, and those operations and their weight may change significantly as the state becomes more populated. For example IO within a rather unbalanced trie (that FRAME should help you avoid but doesn't explicitly stop you from creating) could really negatively impact the weight of the same call for the same runtime. Your pallet calls may also be full of branching code where some scenarios are quite inexpensive (l
Recommends Syed Kamran · 2022-11 · Substrate and Polkadot Stack Exchange · on “How to benchmark a Pallet public functions?”
I watched polkadot seminar on benchmarking here: [this video] Can any one please guid how to benchmark a pallet public functions? more ever, it will be nice if there is some example with step wise explanation
Cites Shawn Tabrizi · 2022-08 · Substrate and Polkadot Stack Exchange · on “benchmarks: Significant different between 99th and worst case”
Rocks DB is a very complex database, which did have strange outlier behaviors we never fully understood. I actually spoke about that to some extent here: [this video] [LINK] There were certain benchmarks where Rocks DB totally went crazy and gave huge outliers, and because of that, we had to remove Rocks DB from the main benchmarking pipeline (using an in-memory db), and instead isolate the database benchmarking to a separate process. So unfortunately I do not have an answer "why exactly" these outliers exist, but it is something that is known about from our experience with the DB, and it is something reasonable to expect when using it. As Oliver mentions, your benchmark results show that the ave
Cites Daan - R0GUE · 2022-11 · Substrate and Polkadot Stack Exchange · on “Calculating actual weights for extrinsics”
First, the transaction_payment pallet [LINK] does this for you with compute_actual_fee [LINK] from the post_dispatch [LINK] and configuring it in your runtime with the ChargeTransactionPayment signed extension via SignedExtra , as done in Kusama [LINK] . Some additional info [LINK] . Going to your questions. The first thing that needs to be understood is the existence of a Storage Overlay [LINK] , which caches values which have been read or written to inside of a block. This means, if you read a storage item, the first time, it will need to navigate through the whole merkle trie, and fetch that storage item. But the second time, it just reads from the storage overlay, since that value has be