Welcome to the October project dev update!
This month has been busy – our long awaited Data Unions launch commenced, and we began our marketing push to developers around the world. We also announced the Streamr Data Challenge, a hackathon for India-based developers and teams. The hackathon will run over the next few months with a focus on building new data economies through Data Unions.
The Streamr tokenomics model is always a hot topic for the community, so we’re thrilled to announce we’ll be having an open session with BlockScience in November to discuss exactly that. We’ll be covering what’s been done in relation to tokenomics, where we are now, and what’s up ahead. To this end, we’re asking the community for suggestions on specific questions or topics you’d like us to cover. Please reach out with your thoughts to me on Telegram or by email.
And as usual, we’ve been busy working towards our roadmap goals. Here are the development highlights of the month:
- BlockScience team is transitioning into Phase 3 and setting up test plans to approach running the first useful simulations.
- Kicked off an experimental project StreamrFS, a file sharing extension on top of the Streamr protocol.
- Conversion of API keys to Ethereum private keys is complete, simplifying the codebase and taking a step towards decentralized identities.
- WebRTC now works reliably. Re-ran the whitepaper experiments with WebRTC, which verified correct operation.
- Working on adding the end-to-end encryption back into JS client after refactoring.
- Data Union 2.0 audits complete.
Table of Contents
The Network & Tokenomics
Two internal workshops were held to develop a precise understanding of the KPIs for each role in the Network. These exercises complete the final pieces of the model and allow us to transition into Phase 3 – running the first simulations of the Network.
The first experiments will explore 1) The bootstrapping phase (Streamr incentivises people to run nodes to kick off the network) and 2) system resilience (how the network tolerates knockout shocks and finds a new equilibrium).
Strong efficiency gains were also noticed with stream resends. If you had any issues in the past, expect it to work much better than previously with the new storage node.
Core App
Data streams are taking another step towards being crypto native. Currently streams are managed by our centralized infrastructure and are identified by machine readable IDs. We’ll be replacing these with human-readable stream IDs, with hierarchical structure based on ENS domains and a user-given path. So streams will start to look like mydomain.eth/traffic/helsinki for example. The Stream registry will live on the blockchain and there will be a sandbox domain where everyone can create streams without having to register an ENS name.
Data Unions 2.0
The team has made steadfast progress towards the second iteration of the Data Unions framework. The contracts are built, and the audits are complete. The frontend team is now exercising 2.0 functionality with the JavaScript client. The team is now working on some final tests and towards making the Java client compatible with 2.0 architecture.
We’ll be setting up a test environment for early adopters to start experimenting with some of the new features of Data Unions 2.0 later this year.
StreamrFS – An experimental file-sharing protocol built on Streamr
File sharing on Streamr? Yes, you read that right. While the Streamr protocol is built for real-time data, a common request from users is the ability to also share access to files, especially in the context of the Marketplace. This requirement has also emerged from the KRAKEN project, an EU-funded project where Streamr technology is used to share medical data in hospital environments.
Real-time data, while powerful, is still somewhat niche in the nascent data economies, while file sharing is conceptually familiar to many, potentially making it a worthwhile idea to explore. To answer this need, an internal hackathon produced a prototype called StreamrFS. By leveraging the pub/sub messaging, storage, and encryption primitives offered by the Streamr protocol, it turned out to be quite straightforward to implement a standardised higher-level mechanism for sharing files and selling access to them. Many kinds of applications can leverage the Streamr protocol – and file sharing is one such application.
In StreamrFS, files are stored in encrypted form in a 3rd party location (IPFS, Swarm, Bittorrent, etc), with encrypted Streamr messages containing a reference to that location along with the credentials needed to access and decrypt the file. By connecting access to the file with access to a stream, essentially making them the same thing, access to files can for example be sold on the Marketplace. While StreamrFS started its existence as a command-line application, it might later become natively supported by the Core and Marketplace applications.
Deprecations and Breaking Changes
A number of API endpoints need to be retired and replaced to be compatible with our vision of decentralization. This section summarises deprecated features and upcoming breaking changes. Items marked ‘Date TBD’ will be happening in the medium term, but a date has not yet been set.
- December 31st: Support for API keys will end
The ability to create API keys has already been removed from the Core App, while previously existing API keys will continue to work until December 31st, 2020. After this date, scripts and applications still using API keys may break. To avoid disruption, simply create or connect an Ethereum account on your Profile page, and pass the private key for that account to the Streamr client library in your application:JS library:
const client = new StreamrClient({
auth: {
privateKey: ‘your-private-key’
}
})Java library:
StreamrClient client = new StreamrClient(
new EthereumAuthenticationMethod(yourPrivateKey)
); - December 31st: Email/password login will be removed
If you’re still using email/password-based login, avoid getting locked out of your account after December 31st, 2020 by doing the following: Install the MetaMask wallet, go to your Profile page, and connect your wallet to your account. From there on, you can use the “Authenticate with Ethereum” option on the login screen instead of the email/password combination. In addition to MetaMask and compatible Web3 wallets, various wallet connectivity solutions such as WalletConnect will be supported in the future. - December 31st: Storage becomes opt-in and freely choosable
So far, the storage node operated by the Streamr team has stored all messages in all streams for a period of time configurable per stream, with the default being one year. Going forward, there may be many storage nodes operated by different parties and located in different geographies, and more control over storage will be needed.As a stream owner, you now have control over which, if any, storage nodes you’d like your historical messages to be stored on. The controls for choosing storage nodes for your stream are already present in the Core application, although at the moment there is only one option shown: Streamr Germany, which is the storage node that has been storing data so far.
By default, no storage nodes are selected, and stream owners can opt-in to storage by selecting the storage nodes they wish. Starting January 1st 2020, storage nodes may only store the streams assigned to them, and purge data for any other streams. To avoid losing any important historical data in your streams, please use the Core application to assign your streams to the Streamr Germany storage node by Dec 31st to maintain status quo and continue storing your streams.
- (Date TBD): Support for unsigned data will be dropped.
Unsigned data on the network is not compatible with the goal of decentralization, because malicious nodes can tamper with data that is not signed. As the Streamr Network will be ready to start decentralizing at the next major milestone (Brubeck), support for unsigned data will be ceased as part of the progress towards that milestone. Users should upgrade old client library versions to newer versions that support data signing, and use Ethereum key-based authentication (see above).
Thanks for reading!
If you’re a developer interested in contributing to the Streamr ecosystem, consider applying to the Streamr Data Fund for financial backing to fast track your plans.
