Welcome to the Streamr dev team update for April 2020. A lot has happened in the last month, so let’s kick off. Streamr has:
- Released a new version of Marketplace
- Integrated Uniswap exchange functionality
- Tested multi-tracker support for the Network
- Continued WebRTC implementation for the Network
A quick editor’s note: we are adding a new section to our monthly dev update titled Deprecations and breaking changes. As you might have guessed, it is to keep all developers building on top of Streamr ecosystem informed about upcoming deprecations (features or components not supported anymore) and breaking changes (alteration in functions, APIs, SDKs and more, that require a code update from the developer side).
The newly deployed Marketplace contains a suite of analytics that users can explore on published Data Union products — number of users belonging to a particular Data Union, aggregated earning, estimated earning potential per user and more. Here you can see the example for Swash, published on the Marketplace. Note that Swash is still in its beta phase and the Data Union Product has been migrated to a newer version of a Data Union smart contract, so the current metrics don’t show the full picture.
Additionally, we also deployed a long-awaited Uniswap integration on the Marketplace. Thanks to this decentralized exchange (DEX), data buyers now can now use either ETH or DAI to pay for a subscription, in addition to DATA. This is an important milestone because it simplifies the purchase process, which had caused some friction for new users.
Recently, the Network developer team finished testing a multi-tracker implementation. For any readers who are not yet familiar with the role a tracker plays in the Network, our core engineer Eric Andrews wrote the following in his recent blog post on the Network workshop:
An important part of the Network is how nodes get to know about each other so they can form connections. This is often referred to as ‘peer discovery’. In a centralized system, you’ll often have a predetermined list of addresses to connect to, but in a distributed system, where nodes come and go, you need a more dynamic approach. There are two main approaches to solving this problem: trackerless and tracker-based.In the tracker-based approach, we have special peers called trackers whose job it is to facilitate the discovery of nodes. They keep track of all the nodes that they have been in contact with, and every time a node needs to join a stream’s topology, they will ask the tracker for peers to connect to.
Now that we have finished testing the tracker model, the next step is to try to create an on-chain tracker registry and let Network nodes read the tracker list directly from there. This can be accomplished via a smart contract on the Ethereum network, so that this whole process of peer discovery can be handled in a decentralized way. In future, richer features could be deployed for the tracker registry, such as reputation management and staking to lower possibility of misbehavior or network attacks. The team made further progress on the Network side with the gradual implementation of WebRTC for the nodes. We recently ran an experiment, running over 70 WebRTC nodes on Linux local environment, and results were promising. That gave us additional assurance to proceed with the full implementation.
Regarding the Data Union development progress, we noticed there have been some performance issues and potential bugs on the balance calculation, due to Data Union server restarting. We sincerely apologize for the inconvenience caused, and we are working to improve the Data Union architecture to guarantee higher stability before the official public launch later this year.
Table of Contents
Deprecations and breaking changes
This section summarizes all deprecated features and planned breaking changes.
- June 1st, 2020: Support for Control Layer protocol version 0 and Message Layer protocol versions 28 and 29 will cease. This affects users with outdated client libraries or self-made integrations dating back more than a year. The deprecated protocol versions were used in JS client libraries 0.x and 1.x, as well as Java client versions 0.x. Users are advised to upgrade to newest libraries and protocol versions.
- June 1st, 2020: Currently, resources on Streamr (such as streams, canvases, etc.) have three permission levels: read, write, and share. This will change to a more granular scheme to describe the exact actions allowed on a resource by a user. They are resource-specific, such as stream_publish and stream_subscribe. The upgrade takes place on or around the above date. This may break functionality for a small number of users who are programmatically managing resource permissions via the API. Updated API docs and client libraries will be made available around the time of the change.
- Further away (date TBD): Authenticating with API keys will be deprecated. As part of our progress towards decentralization, we will eventually end support for authenticating based on centralized secrets. Integrations to the API should authenticate with the Ethereum key-based challenge-response protocol instead, which is supported by the JS and Java libraries. At a later date (TBD), support for API keys will be dropped. Instructions for upgrading from API keys to Ethereum keys will be posted well in advance.
- Further away (date TBD): Publishing unsigned data will be deprecated. Unsigned data on the network is not compatible with the goal of decentralization, because untrusted nodes could easily tamper data that is not signed. As the Streamr Network will be ready to start decentralizing at the next major milestone (Brubeck), support for unsigned data will be ceased prior to reaching that. Users should upgrade old client library versions to newer versions that support data signing, and use Ethereum key-based authentication (see above), which enables data signing by default.
Below is the more detailed breakdown of the month’s developer tasks. If you’re a dev interested in the Streamr stack or have some integration ideas, you can join our community-run dev forum here.
As always, thanks for reading.
Network
- Multi-tracker support is done. Now working on reading tracker list from smart contract
- Moving forward with WebRTC implementation after local environment testing
- Continuing fixes for Cassandra storage and long resend issues.
Data Unions
- Some Java client issues were affecting Data Union joins, but these should all be fixed now
- Improved Data Union Server monitoring. Join process is being continuously monitored in production
- Team started implementing storing state snapshots on IPFS
- JS client bugs fixes to solve problems with joins in Data Union server
- Data Union developer docs are being finalised
Core app (Engine, Editor, Marketplace, Website)
- Implementing UI for managing buyer whitelist for Marketplace
- New Marketplace version has been deployed with Data Union metrics
- New product views and Uniswap purchase flow are now live.
