The Data Unions 2.0 upgrade is now live on Ethereum mainnet and xDai chain, and all new Data Unions deployed via the Streamr Core app will use the new architecture automatically. The upgrade brings security and robustness improvements to Data Unions, adds new features, and enables composability with DeFi. The upgrade also means that the Streamr ecosystem and DATA token now span over multiple blockchains! The Streamr team will work with existing Data Union builders to help them upgrade to the improved platform.
For those of you learning about Data Unions for the first time – the Data Union framework is a data crowdsourcing/crowdselling solution built as part of the Streamr project. Working in tandem with the Streamr Network and Ethereum, the framework powers apps that enable people to earn by sharing valuable data. Learn more about it here!
Table of Contents
What’s new?
Under the hood, everything has changed. The previous-generation Data Unions framework used the Monoplasma off-chain solution for scalability, while Data Unions 2.0 is fully on-chain and implemented as smart contracts. While the gas costs on Ethereum mainnet are currently sky-high, Data Unions 2.0 can run on any Ethereum-compatible sidechain for scalability and affordable cost of operation. The first supported sidechain is xDai, and other options may be added in the future.
The benefits of having Data Unions fully on-chain are many. First of all, the security of the system becomes as strong as the security of the underlying blockchain itself. Secondly, Data Union members and builders get to benefit from composability, network effects, and services available on the same blockchain, particularly DeFi platforms.
Previously, Data Union members needed to withdraw their earned tokens to Ethereum mainnet in order to do anything with them. With current gas prices on mainnet, this has become unfeasible, and tokens are being kept “hostage” by high transaction fees. With Data Unions 2.0, tokens can optionally be withdrawn and used on the sidechain – for example swapped to some other tokens using a DEX present on that sidechain. The transaction fees on sidechains are orders of magnitude cheaper than on Ethereum mainnet, making it much more feasible to move around even small amounts of value. On xDai, transactions currently cost only fractions of a cent, compared to transactions costing tens or even hundreds of dollars on mainnet.
In addition to the architectural changes, the upgrade also comes with some new features requested by builders. Data Unions now support gasless withdrawals via metatransactions. This means that Data Union admins can optionally pay the transaction fees on behalf of members, potentially enabling a better user experience. Another new feature enables tokens to be directly deposited to a particular user’s balance in the Data Union, allowing Data Union builders to (for example) run referral campaigns that pay out personalised rewards (as opposed to the equal revenue sharing among members, the core feature of Data Unions).
The Data Unions 2.0 architecture
Data Unions now live on a chosen sidechain. Information about its members is maintained in the state of the smart contract, and tokens flow in and out of the Data Union smart contract via normal transfers of ERC-20 tokens. As the DATA token originates from the Ethereum mainnet, a bridge such as the xDai OmniBridge is used to move tokens between chains.
Wallets and contracts on Ethereum mainnet can still interact with Data Unions over the bridge. For example, the Streamr Marketplace smart contract can transfer tokens into a Data Union then the data product is purchased. The tokens automatically cross the bridge to the sidechain before being deposited as revenue into the Data Union smart contract.
For those of you interested in seeing more of the technical details, have a look at this Miro board as well as the DU2 smart contracts repo on GitHub.
Towards cheaper gas costs
Obviously, the extreme gas costs on Ethereum mainnet can only be worked around by moving away from mainnet. Data Unions 2.0 is a first step in that direction, enabling many Data Union members to use, trade, or transfer their earned tokens without ever bridging them to mainnet. A growing amount of DeFi platforms and liquidity is available on sidechains, and in the future tokens can even be bridged between sidechains to access additional services or liquidity. For example, the xDai team has deployed a beta version of a bridge between xDai and Binance Smart Chain, allowing tokens to be withdrawn from xDai Data Unions even to Binance Smart Chain without going through the expensive Ethereum mainnet. As sidechain and bridge technologies are developing very quickly at the moment, we expect to see many new possibilities enabled over the course of this year.
That being said, any transactions on the mainnet, including bridging tokens to sidechains, are still expensive. Parts of the Streamr ecosystem, in particular the Marketplace, are still on mainnet, and purchasing access to data products can cost tens of dollars in fees, often more than the price of the data subscription itself. This severely limits the usefulness of the Marketplace at the moment, which is also the case with most other dapps on Ethereum. While Data Union teams can fall back to more traditional interactions with potential data buyers as a temporary solution, it’s clear that the Marketplace along with other smart contracts in the Streamr ecosystem will eventually need to migrate to sidechains, and the team will continue working towards this goal to completely solve the issue of unacceptable transaction fees.
Why choose xDai for Data Unions?
We looked into xDai, Matic/Polygon, Binance Smart Chain, and Avalanche as the potential first homes for the new release. While all of them are very interesting, the xDai chain ticked almost all the boxes for this use case, making it a great chain to kick off with:
- Low enough transaction fees to make Data Union member management and withdrawals feasible even for large Data Unions with hundreds of thousands of members. Swash, the first and biggest Data Union, already has over 14,000 members, and it’s likely that much larger Data Unions will exist in the future.
- A robust Arbitrary Message Bridge (AMB) implementation that is extensible enough to support ERC-677 callbacks when the recipient address on sidechain is a smart contract. The bridge also passes transactions quickly (in minutes).
- Transaction fees are more predictable thanks to using a stablecoin as the native token.
- The Graph indexes the xDai chain.
- Ramp offers direct fiat-to-xDai conversions.
- The team is responsive and welcomes the use case.
From the perspective of the xDai chain, the Streamr Data Unions framework brings 14,000+ Data Union members as well as a handful of new builder teams in touch with the chain.
To better service Data Union admins, members, and other stakeholders of the Streamr ecosystem, we are looking forward to seeing more DeFi platforms and liquidity moving to xDai in the future. As the sidechain space and especially bridges evolve over time, we’ll be keeping a close eye on other suitable chains to support. In the future, Data Unions could exist on multiple sidechains, and the preferred one would be chosen when the Data Union is created.
Manually bridging DATA to/from xDai
Users with DATA tokens in their wallets can bridge them to/from the xDai chain using the xDai OmniBridge UI, which supports Metamask and WalletConnect-enabled wallets.
To see and use tokens in your wallet on xDai chain, configure xDai chain using these instructions, switch to that, and then add a custom token 0xE4a2620edE1058D61BEe5F45F6414314fdf10548, which is the DATA smart contract on xDai.
To make transactions on the xDai chain, you’ll need some xDai for gas. You can buy some via Ramp, or transfer DAI from mainnet using the bridge.
SDK versions compatible with Data Unions 2.0
- JS SDK: version 5.0 onwards. It should be noted that 5.0 and newer versions are not backwards compatible, i.e. they only support Data Unions 2.0, while 4.x and older versions only work with Data Unions 1.0.
- Java SDK: version 2.1 onwards. Older versions did not implement Data Union interactions at all.
Migrating first-generation Data Unions to 2.0
The Streamr team will get in touch with everyone running Data Unions on the old platform, and help them upgrade. Using a utility script, a mirror copy of the old Data Union will be created on Data Unions 2.0, and the Streamr team will provide the token liquidity to create and maintain the copy. Then, the end-user application for the Data Union will be updated to use the new version of the SDK and pointed to interact with the new Data Union instead of the existing Data Union. Eventually, once all migrations are complete, the old Data Union framework will be shut down.
Join the Streamr developer ecosystem
The best place to learn about building Data Unions and working with Streamr open source technologies is the docs. To get in touch with our team for support, make sure to join our Discord server. See you there!
Learn more about Data Unions
- Discover Data Unions
- Documentation: Intro to Data Unions
- Data Union DAO
