Welcome to the November Developer update – our last update of 2020! It’s been another fruitful month in terms of roadmap progress and Data Union growth. We’ve seen a big uptick in developer interest towards building Data Unions, in part from our recent update to the Data Fund, and also following news that the European Union mentioned Data Unions in their latest proposal.
While our outward focus is on growing the Data Union ecosystem, we are of course also working hard to reach the next Network milestone – Brubeck. On that front I’d like to welcome Bernat Canal to the Streamr Network team. Bernat is an Ethereum and Node developer with previous experience in distributed network design; a nice addition to the team! Bernat is already hard at work helping us bring the Network into the hands of the community as soon as possible. As this milestone gets closer, these updates will begin to provide guidance on how to run your own Streamr node.
Here are the main development highlights:
- Big improvement in Network throughput. Stress tests show 240 MB/s being pushed successfully through the WebRTC connection without issue.
- Network Explorer deployed to our staging environment, quite close to release.
- Shared the first prototype of StreamrFS internally. Working well.
- Data Union 2.0 support in JavaScript & Java clients being finalised and tested.
- Data Union 2.0 support being added to the Marketplace and Core apps.
- Governance tooling established in preparation for our first governance pilot.
Table of Contents
Data Unions 2.0, Core, Client and Marketplace developments
A public Data Union 2.0 test environment is very close to release, hopefully rolling out in December. Data Unions 2.0 will be run on an EVM compatible chain. More details will be released closer to the release date.
The Core app was updated this month, the main change being that Stream IDs are now human-readable. The new format for stream IDs is an Ethereum address plus a user-given path. For example you can now identify a stream with 0x123/hello/world or even mydomain.eth/traffic/helsinki. If you haven’t used an ENS name before, you can easily create one with a small amount of ETH on ENS Domains. Your purchased addresses will automatically appear in the stream editor for immediate usage. At a future date the Stream registry will live on the blockchain. Just remember, you may need to escape the slash depending on your coding environment. Also, please check out the breaking changes section below if you are using an email address to login to your Streamr account.
Token Economics
Phase 3 has started with BlockScience, while still running tests to validate our simulation implementation. The test plan focuses on:
1) The bootstrapping phase – Streamr incentivises people to run nodes to kick off the Network.
2) System resilience – how the Network tolerates knockout shocks and finds a new equilibrium.
In short, this means gaining an understanding at an atomic level of how to best grow and sustain a vibrant network in a trustless manner using incentives.
On a separate internal track, there’s been some initial thinking about what layers or higher-level structures are needed between the user and the low-level atomic agreements in the Network. This work will probably result in a diagram or working document to be further discussed internally. Matching the various tokenomic incentive mechanisms with desired behaviour that grows the Network is the goal here.
Our first public tokenomics workshop was also recorded early this month –you can watch it here. We will be recording more tokenomics workshops as new simulations become available. The goal here is to educate, as well as to harness the insights of the community to steer some of our roadmap goals with some larger impact decisions happening early next year.
Network
The Network team had a deep dive discussion into how the node architecture is transitioning from Corea to Brubeck. Currently we have an architecture where nodes are ‘thin’ and clients are ‘heavy’. The Java and JavaScript clients have many technical responsibilities and requirements and this leads to some duplication of code, and an uphill battle to expand to new languages. We’re addressing this by shifting the heavy lifting to the Network node.
The architecture of an end-to-end encrypted and signed message may start at a simple device, communicating in a trusted environment, to a trusted network node then entering the decentralized transport network for delivery to its recipient. This will make supporting all sorts of languages and protocols a far more scalable endeavor. This work hasn’t begun yet, but will commence once Data Unions 2.0 is production ready.
Hackathons
While we have the Streamr Data Challenge happening in India, we are also participating in the Ocean Data Economy challenge, running until January. We’ve sponsored a prize for:
Best integration into Streamr real-time data streams. Utilising the Streamr JS Client to extract a static data set from a stream and forwarding it to the Ocean Marketplace to make it easy to buy & sell access as an Ocean datatoken. It makes sense for data to be sold on as many marketplaces as possible, so to that end, we’re pleased to encourage
development for a real-time to static adapter for the data generated in our streams.
Ending support for email accounts
As part of our journey towards full decentralization, we are ending support for password-based login after December 31st, 2020. After that date, the only way to authenticate to Streamr is by using an Ethereum wallet.
Not only is this method much more secure, it is also extremely convenient, and onboards you to the world of blockchain-based digital assets and digital identity.
To avoid losing access to your Streamr account, you should take the following steps by December 31st:
- Install MetaMask (available for Chrome, Firefox, Brave, Edge, Android, and iOS)
- Go to your profile page on Streamr
- Click the Connect wallet button and pair your wallet with your Streamr account
Congratulations, you are now future-proofed – not only for Streamr, but also myriad other decentralized applications, such as the “money legos” of DeFi (decentralized finance).
Deprecations and Breaking Changes
A number of API endpoints need to be retired and replaced to be compatible with our vision of decentralization. This section summarises deprecated features and upcoming breaking changes. Items marked ‘Date TBD’ will be happening in the medium term, but a date has not yet been set.
- December 31st: Email/password login will be removed
If you’re still using email/password-based login, avoid getting locked out of your account after December 31st, 2020 by doing the following: Install the MetaMask wallet, go to your Profile page, and connect your wallet to your account. From there on, you can use the “Authenticate with Ethereum” option on the login screen instead of the email/password combination. In addition to MetaMask and compatible Web3 wallets, various wallet connectivity solutions such as WalletConnect will be supported in the future. - December 31st: Support for API keys will end
The ability to create API keys has already been removed from the Core App, while previously existing API keys will continue to work until December 31st, 2020. After this date, scripts and applications still using API keys may break. To avoid disruption, simply create or connect an Ethereum account on your Profile page, and pass the private key for that account to the Streamr client library in your application:JS library:
const client = new StreamrClient({
auth: {
privateKey: ‘your-private-key’
}
})Java library:
StreamrClient client = new StreamrClient(
new EthereumAuthenticationMethod(yourPrivateKey)
); - December 31st: Storage becomes opt-in and freely choosable
So far, the storage node operated by the Streamr team has stored all messages in all streams for a period of time configurable per stream, with the default being one year. Going forward, there may be many storage nodes operated by different parties and located in different geographies, and more control over storage will be needed.As a stream owner, you now have control over which, if any, storage nodes you’d like your historical messages to be stored on. The controls for choosing storage nodes for your stream are already present in the Core application, although at the moment there is only one option shown: Streamr Germany, which is the storage node that has been storing data so far.
By default, no storage nodes are selected, and stream owners can opt-in to storage by selecting the storage nodes they wish. Starting January 1st 2020, storage nodes may only store the streams assigned to them, and purge data for any other streams. To avoid losing any important historical data in your streams, please use the Core application to assign your streams to the Streamr Germany storage node by Dec 31st to maintain status quo and continue storing your streams.
- (Date TBD): Support for unsigned data will be dropped.
Unsigned data on the network is not compatible with the goal of decentralization, because malicious nodes can tamper with data that is not signed. As the Streamr Network will be ready to start decentralizing at the next major milestone (Brubeck), support for unsigned data will be ceased as part of the progress towards that milestone. Users should upgrade old client library versions to newer versions that support data signing, and use Ethereum key-based authentication (see above).
