Thank you for linking ;-)
Wait a moment...
An Open Invitation To Join The Catapult Conversations
I wanted to introduce myself. I’m Greg Saive (eVias), the Head Developer for the NEM.io Foundation. This is my first post on NEMLog and will be the first of many. The NEMLog platform is a valuable opportunity to improve transparency from inside the NEM Foundation regarding ongoing projects and discussions with regards to Client Standards Definition, Integration Processes Definition, Public Network Release, Client Applications and Catapult features amongst others.
We do not intend to ask for any funds, tips or donations with this attempt to create a better operational transparency. We are using the NEMLog platform, our blog and our forum to share this article because we believe that it does permit to achieve more visibility in the overall NEM community.
With the distributed characteristics of peer-to-peer networks also come a lot of challenges including to keep everyone up-to-date on projects. It’s important to the Foundation to collaborate with the wider NEM Community so we can shape the future of NEM together. So let’s get started.
There’s a lot of topics where we’re looking for feedback including:
- A quick glance at Catapult features
- NIP-8: Catapult technology release for Public Network
- NIP-7: QR Library Standard Definition
- NIP-6: Multi-Account Hierarchy for Deterministic Wallets
- NIP-2: Transaction URI Scheme
- Client Implementation Contributions
These topics have been discussed mostly on Github or Slack but we would like to improve transparency with regards to decision-making discussions. Here’s a summary of ongoing discussions and open issues for each active NIPs.
A quick glance at Catapult features
Catapult is not just a rewrite of the previous NEM Java implementation. It comes with an improved architecture setup and many new features that I will try to summarize in this section.
Catapult network nodes are differently setup than most other distributed ledger network nodes reference implementation. In fact, Catapult features a highly scalable multi-tier architecture described more in detail here.
One of the exciting features that define the Catapult technology is its ability to communicate with other Catapult networks with the Secret Lock/Proof system. But, more than just a hybrid blockchain, Catapult comes with capabilities for interoperability with different blockchain networks.
With Catapult, NEM transaction types have evolved to be more robust and follow a plugin implementation, technically permitting anyone to create new transaction types. Aggregate transactions are one of such new transaction types introduced with Catapult which was mentioned by core developer Jaguar0625 in a tweet:
Aggregate transactions allow atomic bundling of transactions. The atomicity guarantees that either all bundled transactions are accepted by the chain or none are. Source tweet.
Additionally, Catapult provides an account restrictions feature which lets you manage exactly what can be sent to your account. These restrictions can be applied for mosaics, transaction types and addresses. In fact, as told us core developer Jaguar0625 in a tweet:
Account restrictions allow you to protect your account. For example, it's possible to configure your account to accept only a predefined list of mosaics and block all others. Source tweet.
We highly recommend following Jaguar’s latest Tweets Series (Polls) about “How well do you understand (new) catapult features” starting here. The currently active poll is about multi-level multi-signature accounts, do you know what they are? Tell us now!
A more detailed overview of those technical features available now with Catapult can be found in our NEM Developer Center.
NIP-8: Catapult technology release for Public Network
With Catapult Elephant out on Github, the public chain release has never been so near.
We are actively discussing topics around the public network migration of NEM over to Catapult technology. The latest topics include sensitive decision making about, for example, the ellected post-upgrade scenario and the dataset migrations.
Migrations working groups have been opened to handle and manage topics around the public chain release of Catapult. This includes reviewing source code available for Catapult integrations:
- Protocol Source Code: catapult-server
- REST Gateway Source Code: catapult-rest
- Serialization Tool Source Code: catbuffer
This source code peer-review initiative should be broadly available. Take part in this initiative on Slack and add your peer-review!
Other than the peer-review of protocol source code and packages, the migration process entails making decisions about desired upgrade scenarios as well as migrated datasets.
NIP-7: QR Library Standard Definition
The QR Library Standard Definition is a NIP that opened more than one year ago. Starting this year, we worked with the NEM Foundation teams to re-activate the project and it has seen a lot of improvements with a package in alpha version available now on Github:
Currently, there are open issues with this package because there is no possibility to create QR Codes for Aggregate Transactions. This is because QR Codes - with the highest version number (40) - only allow up to 1200 characters to be stored in the QR Code. This is very limited and aggregate transactions can become very much larger than that. With the current approximative minimum payload size of 142 characters for transfer transactions.
There has been the idea to use a different type of codes rather than QR Codes, but no proposal was opened for this up to now. We are still looking for alternatives that will permit using any kind of transactions of Catapult, also within the scope of encrypting content or integrating different encoding algorithms. There are several alternatives for QR Code with one downside, that QR Codes are most adopted and easy to use:
- iQR Code: https://qrcode.meetheed.com/question40.php
These alternatives are being thought of as a way to produce aggregate transaction QR Codes.
NIP-6: Multi-Account Hierarchy for Deterministic Wallets
This is something I think a lot about and want more community feedback. I opened NIP-6 a few months back to get discussions going as to how to make integration easier. I also published a beta version of the package on the NEM Foundation Github: https://github.com/nemfoundation/nem2-hd-wallets
There’s an open discussion with core developers to implement KMAC message authentication codes rather than HMAC as the SHA-3 derived function being used. This is because KMAC is more appropriate for keys derivation than HMAC. Also, the KMAC approach is the Keccak Message Authentication Code which is most compatible with SHA3 as used in Catapult.
Another important discussion thread was to allow only hardened key derivation. This effectively decreases the number of keys that can be derived by half but makes sure that keys that are generated are using the same algorithm as described in Bitcoin BIP32. It is important to note that, currently, there are no resources available for ED25519 non-hardened key derivation which have been tested and integrated enough to be peer-reviewed correctly. We have decided to implement hardened derivation only because ED25519 Curve does not permit Public Parent Key to Public Child Key derivation. This is because of a characteristic of points on the ED25519 Elliptic Curve where normal derivation is not permitted.
Additionally, base58 format is not supported as this does not serve any use case for the Catapult technology.
NIP-2: Transaction URI Scheme
This NIP aims to define a URI scheme to serve transactions which are ready to be signed. This is an important feature for clients cross-compatibility with Catapult technology. It has been worked on mostly during the beginning of this year and there is a beta package available now on the NEM Foundation Github:
A URI scheme enables communication between applications and NEM wallets. It can be recognized by desktop and mobile wallets, opening the app with the transaction ready to be signed.
The protocol adds web+ at the beginning of the scheme due to a security requirement for Firefox. An example of a valid transaction URI would be:
This opens the doors to many new integration capabilities where web application owners can integrate transaction URIs to permit people to act on their platform(s). The transaction data can be a transaction serialized in hexadecimal or it can also be formatted as DTO. This package is currently being integrated to the nem2-qr-library as well so that scanning QR Codes can directly be recognized by mobile platforms.
Client Application Contributions
Working on several client applications, we appreciate contributions on all our public repositories on Github. Projects that are actively being worked on at the time of writing include:
- Catapult Wallet: https://github.com/nemfoundation/nem2-wallet-browserextension
- QR Library Standard: https://github.com/nemfoundation/nem2-qr-library
- Hierarchical Deterministic Wallets: https://github.com/nemfoundation/nem2-hd-wallets
- Transaction URI Scheme: https://github.com/nemfoundation/nem2-uri-scheme
Of course, the foundation is not alone, contributing and adding peer-reviews to Client applications, Software Development Kits and other Software Packages that integrate Catapult. Have a look on Github to find more Source Code and Snippets related to Catapult!
We are currently defining an upgrade process for the Catapult public network release that is to happen by Q4 of this year. This requires many discussions and feedback from all around the NEM community so please join us in these conversations on Slack.
I hope you enjoy this series of articles and appreciate your continued support.
Head Developer, NEM.io Foundation
Mail: firstname.lastname@example.org | Telegram: https://t.me/evias
Work on Japanese translation available here: https://nemlog.nem.social/blog/28022