Log Store was proposed to Streamr Governance to enable automatic decentralised permanent storage of any data transmitted over the Streamr Network.
The problem that sparked this project is that when custom data is utilised within Smart Contracts, this data is typically requested from centralised, siloed environments which exposes security risks associated to data integrity and data availability. Even when Arweave is utilised, centralised APIs that behave as gateways to this immutable data can remain points of compromise. The team behind Log Store experienced this issue first-hand whereby using custom data to manage the allocation of digital assets required trust in the entity managing the data.
The Log Store Network solves this issue by decentralising data management from the point of data collection, to storage and then to query.
By leveraging the simplicity of data transport over the Streamr Network, consensus-driven data validity managed by KYVE and guaranteed permanence and immutability of data with Arweave, the Log Store Network achieves its purpose.
Log Store alpha releases today — being appropriately named the AlphaNet.
Table of Contents
What can I do with it?
The AlphaNet will enable developers to store and query their Streamr data in a permissionless, decentralised and fully verifiable manner, but also provides practical solutions for dApps that require reliable, verifiable data storage.
Other use cases range from DeFi protocols leveraging data-driven on-chain token re-allocation, Games rewarding users based on their engagement across Web3, Marketing platforms rewarding users for their interactions and data, archiving and retrieval of data for public ledgers and Web3 infrastructure, and even long-term, easily accessible records, that can aid in decentralised identity verification. These use cases demonstrate how pure and reliable custom data can bridge the divide between heterogeneous systems. Additionally, its time-series capability prevents redundant data ingestion and even makes it an ideal fit for sectors such as financial analysis and IoT data monitoring, where trend tracking and interpretation of data are pivotal.
There are likely many other exciting use cases that we can’t wait to see the Web3 developer community explore.
Store your Streamr data
- Register your Streamr streams into the LogStoreManager Smart Contract where AlphaNet tokens can be staked to fund its storage.
- Once registered, all stakeholders of the stream can also participate in staking to fund its storage.
- As data is published to the stream, the Log Store network will be responsible for; i) firstly, caching the received data in a time-series format, as to ensure high-availability ii) secondly, bundling and storing all cached data onto Arweave.
Query your Streamr data
- Obtain some AlphaNet tokens then stake them into the LogStoreQueryManger Smart Contract.
- Use the Log Store Network’s API to fetch data between any two timestamps.
- Optionally, verify your query response. Each Query response will be accompanied by a series of signatures of each of the Nodes responsible for the response, meaning that you can verify that each response is truly the response of the entire network.
Beyond developing technology to facilitate a decentralised network of nodes that are incentivised through tokenomics to perform correctly, the team behind Log Store has also created packages such as a CLI and Client library that developers can use to simplify their interactions with our Smart Contracts and the Network.
Documentation will also be produced to support developers getting involved with this technology.
So how does it work?
The Log Store Network is comprised of two sub-networks bound by their own Blockchain systems and interlinked via a Streamr stream that behaves as a communication mesh.
The first layer is the Broker Network, where each Node is a Streamr Broker Node that has installed the logStore Plugin.
This first layer is effectively a decentralised version of the Streamr Broker Storage Plugin, whereby Nodes receive events over Streams, store those events into a local time-series database, and expose connections over HTTP for query-ability.
Log Store has taken a great effort to embed this layer within the Streamr framework. The developer experience should resemble Streamr’s quite closely i.e. the Log Store Client package shares many of the same methods as the Streamr Client, and only complements the Streamr Client with a new query method.
In order to guarantee the operational performance of the Broker layer, the second Validator layer is required.
This Validator layer not only behaves as an authority over the Broker layer — determining which Nodes are rewarded, which are penalised and how digital assets should be consumed from developers interacting with the network, but also behaves as the storage mechanism so that Streamr data is uploaded to Arweave in a fully decentralised manner.
The Validator layer builds upon KYVE, a consensus-based data lake comprised of data pools run by validators that collectively fetch, validate, and store data onto decentralised storage platforms.
Once data is pushed to Arweave, a recursive process takes place, whereby Nodes within the Broker layer are responsible for reading from this decentralised data lake, and then submitting reports to our Polygon Smart Contract. This reporting process draws on inspiration from Chainlink’s Off-Chain Reporting (OCR) strategy for minimising gas fees.
What programming language is it developed in?
The network nodes and client packages are developed in TypeScript. This ensures compatibility with Streamr and KYVE frameworks.
The Smart Contracts are to be deployed to Polygon, meaning they’re developed in Solidity.
Next Steps
The Log Store Network’s AlphaNet, is very early software and therefore we expect to encounter bugs, problems and many other hurdles on the road to a robust and production-grade platform.
For this reason, the AlphaNet will launch with its own custom ERC20 Token.
This will enable us to permission Node Operator access, while also delivering compatibility with live Streamr streams and Arweave storage.
To obtain this Token so that you can get started and store or query data, a separate process will be involved for exchanging your MATIC for AlphaNet Tokens.
This exchange is only necessary to compensate the network for storing the data!
We hope this gives you a good overview of what’s to come, and hope to see your Streamr data on the decentralised internet very soon!
Experiment with the Log Store today, dive into the dev docs, follow Usher Labs on Twitter and Discord, and to learn more, join the teams for an AMA hosted on the Streamr Discord at 16:00 CEST, 3rd July 2023.
About Usher Labs
Usher Labs is decentralizing custom data-driven digital asset management. The goal is to secure the conversion of digital interactions and signals into on-chain outcomes. The Log Store Network is the solution, offering a permanent and secure data store for event data, indexed and queryable by time. Prior to this development, Usher Labs released its Web3-integrated referral marketing platform which demonstrated the need for such infrastructure as it involved the redistribution of digital assets between parties engaged in performance-based partnerships.
About Arweave
Arweave is the first protocol that enables permanent data storage. Its design allows anyone to preserve data forever with just a single, one-time fee. The protocol is secured by an open and decentralized network of miners, who store and replicate data in exchange for AR token rewards. These rewards are funded via Arweave’s sustainable endowment, ensuring that your data will always be available, backed by a transparent and immutable risk model. Arweave empowers you to unleash your creativity and launch fully decentralized applications on the permaweb with ease. Our comprehensive ecosystem of tools and services are robust, composable, and ready to be molded to fit your vision. Learn more at arweave.org.
About Kyve
KYVE Network is revolutionizing customized access to on- and off-chain data by providing fast and easy tooling for decentralized data validation, immutability, and retrieval. Its data lake archives and validates data in a decentralized way, making trustless data easily accessible via solutions like KYVE’s Data Pipeline. KYVE has gained major support, currently backed by top VCs, including Hypersphere Ventures, Coinbase Ventures, Distributed Global, Mechanism Capital, CMS Holdings, IOSG Ventures and blockchains such as Arweave, Avalanche, Solana, Interchain, and NEAR. Join KYVE’s community: Twitter | Discord | Telegram
