From e2779990fa57034d49a76843df54cad06fc6a21d Mon Sep 17 00:00:00 2001 From: Ryan Soury Date: Sun, 31 Dec 2023 16:12:42 +1100 Subject: [PATCH 01/11] Create zkoracle-framework.md --- applications/zkoracle-framework.md | 244 +++++++++++++++++++++++++++++ 1 file changed, 244 insertions(+) create mode 100644 applications/zkoracle-framework.md diff --git a/applications/zkoracle-framework.md b/applications/zkoracle-framework.md new file mode 100644 index 00000000..76026fe1 --- /dev/null +++ b/applications/zkoracle-framework.md @@ -0,0 +1,244 @@ +# Mintbase Grant Proposal + +- **Project Name:** zkOracle Framework for Real-world Data +- **Team Name:** Usher Labs +- **Payment Address:** 5fd9395dbfffbbf9388724cf65a9f3d918e75b5115c36d2c9ac964a1a3f7bcd6 +- **Level:** 2 + +## Project Overview 📄 + +**Tag Line:** Streamlining Real-world Data Verification for Smart Contracts + +**Brief Description:** +Usher Labs is embarking on an innovative project to develop a zkOracle Framework, designed to facilitate the creation of tailored ZKOracles that authenticate real-world data. This framework aims to empower Aurora and NEAR Smart Contracts to reliably verify cryptographic proofs pertaining to private data sourced from centralised operations or TLS-enabled HTTP endpoints. To showcase the framework's capabilities, we plan to implement a simple NFT project on Mintbase, where Tweets are minted with cryptographic assurance of uniqueness, preventing duplicate minting of the same tweet. + +**Integration into the Mintbase / NEAR Ecosystem:** +Our project is intrinsically linked to the Mintbase and NEAR ecosystem. By enabling the minting of unique Tweet-based NFTs on Mintbase, we demonstrate the practical utility of our zkOracle Framework within the NEAR environment. This integration showcases how real-world data, verified via our framework, can enhance the authenticity and uniqueness of NFTs, adding tangible value to the Mintbase platform. + +**Reason for Interest:** +Our motivation stems from Usher Labs' ongoing efforts in the Transparency Network. Our Transparency (T) Network addresses transparency challenges in centralised operations through the T Node, which enriches off-chain data with cryptographic proofs. The T Node functions as a Database, HTTP request Prover, and Proxy Interface, operating in tandem with our T (Notary) Network. It securely processes first-party data, private API/HTTP data, and third-party partner data. With our interest in zero-knowledge architecture, we aim to extend this functionality, enabling the proof of diverse real-world data on-chain, such as KYC/Identity and financial information. This project is a natural extension of our existing work and aligns with our vision to enhance data integrity and privacy on the blockchain. + +### Project Details + +**UI/Architecture Documentation:** + +- **UI Mockups**: We have prepared detailed mockups for the dApp interface, designed for minting Tweet NFTs. These mockups provide a clear and intuitive user experience. + + ![151763928-ac7e9eb7-dc62-4b24-b9b4-1316e9bcbe39.png](https://user-images.githubusercontent.com/63192115/151763928-ac7e9eb7-dc62-4b24-b9b4-1316e9bcbe39.png) + +- **Solution Architecture**: A comprehensive architectural diagram is available, illustrating the integration of various components within our solution. + + https://miro.com/app/board/uXjVN-U5nDI=/?share_link_id=681340344130 + +- **Video Walkthrough**: For a deeper understanding, we offer a video walkthrough that explains the solution’s architecture and its functionalities. + + https://www.loom.com/share/2e9d3738568446a3a8511a44e35d30ca?sid=783a0a02-d77c-422f-99b7-7d26b8945827 + + +**Technology Stack:** + +- **Transparency (T) Node and Network**: At the core of our project is the T Node, an advanced version of our already deployed Log Store Node. This technology specialises in generating cryptographic proofs for data, both published to and retrieved from, the network and HTTP endpoints. +- **Notary Proof for Twitter Data**: A key feature includes generating Notary Proofs for data obtained from Twitter's V2 API, ensuring the authenticity of NFTs representing Twitter data and their uniqueness based on Tweet IDs. +- **zkOracle Framework**: The framework will utilise ZKLLVM, a circuit compiler enabling Rust programming language to be compiled into ZK Circuits. We will leverage the Placeholder Proof System for verification within EVM environments. +- **Smart Contract Integration**: Verification Smart Contracts will be deployed on Aurora, with integration into NEAR Smart Contracts for NFT minting and management via the Aurora Contracts SDK. +- **User Interface dApp**: A user-friendly Next.js based dApp will be developed, using existing open-source libraries for swift implementation. This dApp will facilitate the minting process of Twitter-based NFTs. + +**Proof of Concept and Prior Work:** +Usher Labs has a strong track record, evident in our open-source libraries showcasing expertise in Rust programming, Smart Contract deployment across various blockchains, and Node and Network architecture development, exemplified by the Log Store. + +**Project Scope and Expectations:** + +- **Primary Focus**: The project is primarily aimed at providing developers with a streamlined zkOracle to authenticate real-world data on-chain. +- **Demonstration via Twitter and Mintbase**: While the creation of unique Twitter NFTs on Mintbase serves as an intriguing use case, it is primarily a means to demonstrate the broader utility of bridging real-world data with the NEAR blockchain. +- **Limitations**: It's important to note that while our project might catalyse a novel NFT collection on Mintbase, our main focus is on enhancing developer tools for real-world data verification in the NEAR ecosystem. + +### Ecosystem Fit + +**Integration into the Mintbase and NEAR Ecosystem:** +Our project is strategically positioned to enhance the NEAR blockchain, aligning with its goal of evolving into a comprehensive Blockchain Operating System. By enabling the verification of real-world data on-chain, our solution bridges the gap between familiar web2 services and the innovative capabilities of web3. Our initial demonstration via Mintbase, involving the creation of unique Twitter-derived NFTs, is a testament to this integration. This approach not only introduces an element of novelty and scarcity in the NFT space but also paves the way for advanced crypto payments and exchange functionalities integrated with social media platforms like Twitter. + +**Target Audience:** +While our project might initially appeal to NFT enthusiasts intrigued by the novelty of Twitter-based NFTs, our core target audience extends far beyond. We are primarily focused on Web3 developers and enterprises looking to integrate real-world data verification within their blockchain applications. Our technology serves as a bridge for these entities to incorporate public and private data from traditional web2 platforms, enhancing their blockchain applications' functionality and security. + +**Meeting Market Needs:** +Our project addresses a crucial need in the blockchain ecosystem – the ability to verify real-world data in a transparent, secure, and cryptographically sound manner. This opens up new avenues for businesses to securely interact with traditional platforms like KYC, banking, finance, and ticketing, all while ensuring data integrity and security. Such capabilities are pivotal in advancing the practical use cases of blockchain technology in everyday business operations. + +**Comparison with Existing Projects:** +To our knowledge, there are no direct counterparts within the NEAR or Mintbase ecosystems that offer similar functionalities. However, in the broader blockchain and Web3 domain, projects like HyperOracle, PadoLabs, and ZKPass have ventured into related areas. What sets our project apart is our specific focus on the NEAR ecosystem and our unique approach to integrating Twitter data into NFTs, providing a practical demonstration of our technology's potential. Our solution not only adds value to the NEAR blockchain in terms of data verification capabilities but also innovates in how social media interactions can be transformed into valuable, verifiable digital assets. + +## Team 👥 + +### Team members + +**Team Lead:** Ryan Soury — Solutions Architect, Engineer, Founder + +**Team Members:** + +- Victor Shevtsov — DevOps, Infrastructure, Senior Engineer +- Shuaibu Alexander — Smart Contracts, On-chain, Research Engineer +- Raffael Campos — Full-Stack, Infrastructure, Wildcard Engineer + +### Contact + +- **Contact Name:** Ryan Soury +- **Contact Email:** ryan@usher.so +- **Website: https://www.usher.so** *(due for an update)* + +### Legal Structure + +- **Registered Address:** 8 The Green STE R, Dover, DE 19901, USA +- **Registered Legal Entity:** Usher Labs Inc. + +### Team's experience + +**Introduction to the Team:** +Our project is spearheaded by a talented and geographically diverse team, hailing predominantly from Australia but with a global outreach and influence. Each team member contributes a unique blend of skills and experiences, essential for the multifaceted demands of our innovative project. Below is an overview of our core team members: + +1. **Ryan Soury**: + - **Role**: Team Lead and Solutions Architect + - **Experience**: Ryan is the creative force behind the development of the Log Store, a decentralised time-series database crucial for securely storing and serving event data. His expertise in blockchain technology and a history of successful project implementations make him an ideal leader for our project. Ryan’s innovative approach is the driving force behind our solution's architecture and strategic direction. +2. **Victor**: + - **Role**: DevOps and Infrastructure Specialist + - **Experience**: Victor’s journey in the Web3 realm has been marked by a passion for Rust programming, which he has adeptly applied to Smart Contract development. His work includes a proprietary project that utilised the capabilities of Solana and Wormhole, showcasing his skills in infrastructure operation, uptime management, and data engineering. His expertise in DevOps is vital for the robust and efficient infrastructure of our project. +3. **Alex**: + - **Role**: Smart Contract Developer and Web3 Expert + - **Experience**: With a rich background in Web3 startups, including Push Protocol (formerly EPNS), Alex brings comprehensive expertise in Smart Contract development, review, and testing. His involvement extends to dApp development, providing invaluable insights into the Web3 landscape. Alex’s holistic understanding of Smart Contracts and dApp ecosystems is instrumental for the technical execution of our project. +4. **Raffael**: + - **Role**: Full-Stack Developer and DevOps Engineer + - **Experience**: Raffael’s proficiency spans across application engineering, DevOps, and cloud infrastructure. His rapid adaptation to Web3 technologies, combined with direct experience in protocol and cryptographic schemes at Usher Labs, makes him a versatile and key contributor to our project. His ability to manage complex systems ensures smooth operation and deployment. + +**Team's Collective Expertise**: +Our team boasts over three years of dedicated experience in the Web3 industry, enriched by a broader background in traditional Web2 development. This diverse expertise enables us to navigate the intricate requirements of our project effectively. Our deep understanding of cryptographically verifiable data engineering, coupled with extensive experience in Smart Contract development across both EVM and Rust-based chains, uniquely positions us to tackle the challenges of this project and drive it towards success. + +### Team Code Repos + +- https://github.com/usherlabs +- https://github.com/usherlabs/logstore/ +- https://github.com/usherlabs/logstore-node/ +- https://github.com/usherlabs/ccamp/ +- https://github.com/usherlabs/indexer-relayer +- https://github.com/usherlabs/verifiable-data-streams + +**GitHub accounts of all team members** + +- https://github.com/rsoury/ +- https://github.com/victorshevtsov/ +- https://github.com/Xand6r/ +- https://github.com/outerlook + +### Team LinkedIn Profiles (if available) + +- https://www.linkedin.com/in/ryan-soury-904072100/ +- https://www.linkedin.com/in/victor-shevtsov-3a772258 +- https://www.linkedin.com/in/alex-ander-shuaibu-28889b17a/ + +## Development Status 📖 + +- SIPs for Streamr Network: + - [SIP-19](https://vote.streamr.network/#/proposal/0x615d603fd1f4e5c541f282081679ec3ae359c93b533ab9874e335c3e212c3af3) + - [SIP-13](https://vote.streamr.network/#/proposal/0x87a023191e4525265c0f1c3a68b5b63e4dd758e8dbbdec2cb3ec721828773f1c) +- [Documentation](https://docs.logstore.usher.so/) +- [Tutorial](https://docs.logstore.usher.so/network/quick-start/tutorial) +- [Research Papers](https://docs.logstore.usher.so/network/overview/paper) +- [T Node for Stellar](https://www.loom.com/share/54611c7b984d412db9e3c96cde0e5281?sid=644e8832-acac-4ba7-9ddb-aeb71790732b) +- [Discussion on Asset Management in ICP](https://forum.dfinity.org/t/introducing-ccamp-unlocking-cross-chain-defi-aggregation-on-the-internet-computer/24643) +- [Discussion on Verifiability of TheGraph](https://forum.thegraph.com/t/empowering-subgraphs-with-verifiable-confirmations/4738/5) + +**Investors Materials**: +Investor Deck and Company Overview Brief can be shared. + +## Development Roadmap 🔩 + +### Overview + +- **Total Estimated Duration:** 5 months +- **Full-Time Equivalent (FTE):** 2 +- **Total Costs:** $50,000 USD + +### Milestone 1: Initial zkProof Generation and Verification — $10,000 + +- **Estimated Duration:** 1 month +- **FTE:** 2 +- **Costs:** $10,000 + +| Number | Deliverable | Specification | +| --- | --- | --- | +| 0a. | License | Apache 2.0 | +| 0b. | Documentation | Inline documentation and a basic tutorial explaining how to set up the ZK Proof Generation environment and generate sub-proofs using the Transparency Node. | +| 0c. | Testing Guide | Guide on setting up and running unit tests for the initial zkProof generation logic. | +| 0d. | Docker | Dockerfile for setting up the ZK Proof Generation environment. Docker Compose environment for zkProof Generation, including separate Dev and Prod Compose Files. The Prod Compose File includes a service for the Transparency Node. | +| 1. | ZK Proof Generation Logic | Development of boilerplate logic for ZK Proof Generation of sub-proofs produced by the Transparency Node. | +| 2. | Notary Proofs from Twitter API | Implementation of logic to generate Notary Proofs for data fetched from the Twitter API. | +| 3. | ZK Circuit Verification | Development of a ZK Circuit for verifying the Notary Proofs (Sub-proofs). | + +### Milestone 2: Smart Contract Development and Testing — $10,000 + +- **Estimated Duration:** 1 month +- **FTE:** 2 +- **Costs:** $10,000 + +| Number | Deliverable | Specification | +| --- | --- | --- | +| 0a. | License | Apache 2.0 | +| 0b. | Documentation | Documentation on deploying NEAR and Aurora Smart Contracts to respective testnets and their integration using Aurora Contract SDK. | +| 0c. | Testing Guide | Testing guide for Smart Contract functionality, with a focus on Docker operation for ZK Proof generation. | +| 0d. | Docker | No updates in Docker setup; existing environment used for test proof generation. | +| 1. | Verification Smart Contracts | ZK Verification Smart Contracts leveraging the existing EVM Placeholder Proof System Verification | +| 2. | NEAR Contract Integration | Integration of Aurora Smart Contract functionality through a NEAR Smart Contract entry point. | +| 3. | Smart Contract Deployment | Deployment of ZK Verification Smart Contracts to Aurora Testnet, and deployment of NEAR Smart Contracts to NEAR Testnet | +| 4. | Testing Framework | Development of a test-driven development framework for Smart Contract functionality. | + +### Milestone 3: **Mintbase Integration and Metadata Research** — $10,000 + +- **Estimated Duration:** 1 month +- **FTE:** 2 +- **Costs:** $10,000 + +| Number | Deliverable | Specification | +| --- | --- | --- | +| 0a. | License | Apache 2.0 | +| 0b. | Documentation | Documentation on NEAR Smart Contract callback functionality for Mintbase NFT minting integration and metadata handling research. | +| 0c. | Testing Guide | Expanded testing suite incorporating NFT creation callback mechanisms. | +| 0d. | Docker | No Docker updates required for this milestone. | +| 1. | Callback Mechanism & Mintbase Module Integration | Expansion of NEAR Smart Contract callback functionality for NFT creation on Mintbase. | +| 2. | Metadata Handling Research | Research on data availability options for NFT metadata, including on-chain, in ZK Proof, and Arweave storage. | +| 3. | Unit Testing | Comprehensive development of unit tests for the integrated NFT creation process. | + +### Milestone 4: Refinement and Modularisation — $10,000 + +- **Estimated Duration:** 1 month +- **FTE:** 2 +- **Costs:** $10,000 + +| Number | Deliverable | Specification | +| --- | --- | --- | +| 0a. | License | Apache 2.0 | +| 0b. | Documentation | Documentation detailing the separation of the zkOracle Framework and Twitter NFTs Example, including Docker service separation. | +| 0c. | Testing Guide | Distinguishing testing procedures for zkOracle Framework and Twitter NFTs Example. | +| 0d. | Docker | Docker setup update to separate services by their respective purposes. | +| 1. | ZK Circuit Refinement | Refinement of the ZK Circuit for verifying Twitter Data Sub-proofs. | +| 2. | Framework Modularisation | Modularisation of the zkOracle Framework to abstract from the Twitter API Integration. | +| 3. | Orchestration Layer Development | Development of an Orchestration Layer for the zkOracle Framework, facilitating frontend feedback. | + +### Milestone 5: NFT Minting dApp Deployment and Testing — $10,000 + +- **Estimated Duration:** 1 month +- **FTE:** 2 +- **Costs:** $10,000 + +| Number | Deliverable | Specification | +| --- | --- | --- | +| 0a. | License | Apache 2.0 | +| 0b. | Documentation | Comprehensive documentation on the modified NFT minting dApp, including integration details with NEAR/Mintbase libraries. Reference to third-party packages used. | +| 0c. | Testing Guide | End-to-end testing guide for the NFT minting process with zkProof verification. | +| 0d. | Docker | Deployment of Docker environment for the final dApp. | +| 0e. | Article | Publication detailing the project's achievements, impact on the Mintbase, NEAR, and wider blockchain ecosystems. | +| 1. | dApp Modification and Integration | Modification and deployment of an existing open-source NFT Minting dApp, incorporating NEAR/Mintbase libraries for minting NFTs with zkProof verification from the zkOracle. Reference to third-party packages like https://github.com/AnishDe12020/twnft. | +| 2. | Smart Contract Deployment | Deployment of the complete suite of Smart Contracts to Mainnets. | +| 3. | E2E Testing | Conducting end-to-end tests to ensure unique Twitter Data minting and listing on the Mintbase Marketplace. | + +## Future Plans +In the short term, Usher Labs plans to utilise and promote the zkOracle Framework as a foundational tool for enterprises integrating blockchain technology, particularly focusing on the verification of real-world data on-chain. Our promotion strategy will involve direct sales, partnerships, and highlighting the potential of NEAR as a Layer 1 blockchain. We also aim to promote the Twitter NFTs collection through co-marketing efforts with Mintbase and other NFT communities, exploring revenue-sharing models with experienced marketers. In the long term, we are committed to continually enhancing the framework, fostering a community around it, and staying at the forefront of blockchain technology advancements. We will explore expanding the integration of social, financial, or identity data into blockchain, depending on the success of our initial project with Mintbase. + +## Additional Information ➕ +Usher Labs learned about the Mintbase Grants Program through a referral from a previous NEAR Foundation Lead. As a capital-backed research and development lab, we specialise in addressing transparency challenges in centralised operations. Our team has a strong track record of successful partnerships and grant acquisitions from notable organisations in the blockchain space, such as Arweave, Streamr, Stellar, DFINITY, and Kyve. This history highlights our commitment to innovation and our capability to contribute significantly to blockchain ecosystems, including NEAR. From 95c2bfcd9f2e9d46d236c4fb9a3b1ebc385bd06d Mon Sep 17 00:00:00 2001 From: Ryan Soury Date: Tue, 7 May 2024 16:56:46 +1000 Subject: [PATCH 02/11] new submission for milestone 1 of zkoracle framework --- ...koracle-framework_milestone1_2024-05-07.md | 89 +++++++++++++++++++ 1 file changed, 89 insertions(+) create mode 100644 submissions/zkoracle-framework_milestone1_2024-05-07.md diff --git a/submissions/zkoracle-framework_milestone1_2024-05-07.md b/submissions/zkoracle-framework_milestone1_2024-05-07.md new file mode 100644 index 00000000..9e9458d0 --- /dev/null +++ b/submissions/zkoracle-framework_milestone1_2024-05-07.md @@ -0,0 +1,89 @@ +# Milestone Delivery Guidelines + +These are the guidelines to be followed for milestones submitted for evaluation. + +## Content + +The submission should contain the following information: + +- zkOracle Framework for Real-world Data +- **Open-source code/delivery**: + - Framework: [https://github.com/usherlabs/zkaf-r0](https://github.com/usherlabs/zkaf-r0) + - X (Twitter) NFTs Game: [https://github.com/usherlabs/x-twitter-nfts](https://github.com/usherlabs/x-twitter-nfts) + - Framework (on legacy proving system): [https://github.com/usherlabs/zkaf](https://github.com/usherlabs/zkaf) +- **[License](#license)** +- **[Documentation](#documentation)** +- **[Formatted code](#formatted-code), according to a set of guidelines** +- **[Testing Guide](#testing-guide)** +- **A list of the [milestone deliverables](#milestone-deliverables)** +- **Any [additional information](#additional-information)** + +## License + +Licensed under Apache 2.0. + +## Documentation + +1. Instructions for generating TLS Proofs are found in [https://github.com/usherlabs/x-twitter-nfts](https://github.com/usherlabs/x-twitter-nfts) + 1. Configure X (Twitter) API v2 Keys and Conversation/Tweet ID in `./src/twitter/.env` + 2. Start the Notary Server - *This server runs locally for testing purposes, but will be offered by Usher Labs' decentralised data security network for production environments.* + ```shell + ./start_notary_server.sh + ``` + 3. Generate Twitter TLS Proof + ```shell + ./generate_twitter_proof.sh + ``` +2. Clone the framework - `git clone https://github.com/usherlabs/zkaf-r0` +3. If you've created a new TLS Proof, load it into the `./host/fixtures/` directory +4. Execute the ZK Circuit locally with `RISC0_DEV_MODE=1 cargo run --release` + +## Formatted code + +The [ZKAF R0 (Framework) codebase](https://github.com/usherlabs/zkaf-r0) and the X (Twitter) NFTs Game which involves the TLS Proof generation are both programmed in strictly typed Rust, for the most part. Comments are included among various steps in the logic to aid in understand the flow and processing of data. + +The currently archived/deprecated [ZKAF (on zkLLVM) codebase](https://github.com/usherlabs/zkaf) includes Typescript for handling Smart Contract dependencies issues, however, is no longer relevant to demonstrate the working deliverables for this milestone. + +## Testing Guide + +Currently, tests are performed rudamentarily by executing the programs and ensure that there is a valid response through human evaluation. + +### A step-by-step guide demonstrating how your code achieves the milestones + +Two videos have been created to walk you through the project, and how it achieves the outlined milestones. + +1. [A quick overview](https://www.loom.com/share/e5875772d9dd460facacd8daf5f7f8bf?sid=ad222b18-a9a7-42b4-a845-76c8b7a9bd65) +2. [An in-depth explainer](https://www.loom.com/share/5cdf166a728f43a995eecd79fe1e716d?sid=e2ee0a55-180e-40e8-b7ca-58629b25cfe0) + 1. The in-depth video explains why we switched from zkLLVM to RiscZero, for now. + +### Unit tests + +Unit test boilerplate has been established and leveraged where components are complex - specifically [within the custom `tls-substrings-verifier` module](https://github.com/usherlabs/zkaf-r0/blob/8f54c1c563220db80bf07620d11d46fa1a8fe988/tlsn-substrings-verifier/src/proof/substrings.rs#L331) that we developed to be compatibile with the ZK proving systems. + +## Milestone Deliverables + +Please provide a list of milestone deliverables. This list should closely reflect the list of deliverables agreed on in the Pull Request for the public **Grants Program** application or in Annex 1 of the grant contract for the private applications. + +Each item in the list should include a link to the deliverable itself, e.g.: + +- Google Doc link - make sure anyone with the link has View access +- GitHub repository - include the appropriate file/folder in the link + +**Please highlight anything that deviates from the contract** and include further information that you think is relevant to the deliverable, either alongside the appropriate deliverable or under [Additional Information](#additional-information). + +Please ensure the repo has the correct open-source license. + +| Number | Deliverable | Links & Notes | +| --- | --- | --- | +| 0a. | License | Apache 2.0 | +| 0b. | Documentation | [LICENSE](https://github.com/usherlabs/zkaf-r0/blob/master/LICENSE) | +| 0c. | Testing Guide | [ZKAF Testing Guide](https://github.com/usherlabs/zkaf-r0/blob/master/README.md) and [TLS Proof Guide](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md) | +| 0d. | Docker | [Docker setup guide for deprecated ZKAF](https://github.com/usherlabs/zkaf?tab=readme-ov-file#build-the-image). Docker is incompatible with newer ZK proving systems due to GPU accelerated hardware requirements. | +| 1. | ZK Proof Generation Logic | [Development of boilerplate logic for ZK Proof Generation](https://github.com/usherlabs/zkaf-r0/blob/master/host/src/main.rs) of [sub-proofs produced by the Transparency Node](https://github.com/usherlabs/zkaf-r0/blob/master/methods/guest/src/main.rs). | +| 2. | Notary Proofs from Twitter API | [Implementation of logic](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/twitter/src/main.rs) to generate [Notary Proofs](https://github.com/usherlabs/zkaf-r0/blob/master/host/fixtures/twitter_proof.json) for data fetched from the Twitter API. | +| 3. | ZK Circuit Verification | [Development of a ZK Circuit](https://github.com/usherlabs/zkaf-r0/blob/master/methods/guest/src/main.rs) for [verifying the Notary Proofs (Sub-proofs)](https://github.com/usherlabs/zkaf-r0/blob/master/host/src/main.rs#L39). | + +## Additional Information + +1. The project has been renamed to ZK Attestation Framework, from ZKOracle Framework, as this better depicts the value and purpose of the technology - in that it works to succinctly prove processing over verified attestations of real-world data. +2. The architecture of the project was designed with zkLLVM proving system in mind, but limitations of this nascent technology have forced us to explore alternatives to deliering on the same zero-knowledge protocol to converting proofs about the real-world into verified data points for verification on the Near / Aurora blockchains. \ No newline at end of file From d4c32a9ab2308c3a03f4a1095a42f7f53eff14f1 Mon Sep 17 00:00:00 2001 From: Ryan Soury Date: Tue, 7 May 2024 17:01:55 +1000 Subject: [PATCH 03/11] update docs steps in submission --- submissions/zkoracle-framework_milestone1_2024-05-07.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/submissions/zkoracle-framework_milestone1_2024-05-07.md b/submissions/zkoracle-framework_milestone1_2024-05-07.md index 9e9458d0..b9800715 100644 --- a/submissions/zkoracle-framework_milestone1_2024-05-07.md +++ b/submissions/zkoracle-framework_milestone1_2024-05-07.md @@ -35,7 +35,7 @@ Licensed under Apache 2.0. ./generate_twitter_proof.sh ``` 2. Clone the framework - `git clone https://github.com/usherlabs/zkaf-r0` -3. If you've created a new TLS Proof, load it into the `./host/fixtures/` directory +3. If you've created a new TLS Proof, load it into the `./host/fixtures/twitter_proof.json` directory 4. Execute the ZK Circuit locally with `RISC0_DEV_MODE=1 cargo run --release` ## Formatted code @@ -73,6 +73,8 @@ Each item in the list should include a link to the deliverable itself, e.g.: Please ensure the repo has the correct open-source license. + + | Number | Deliverable | Links & Notes | | --- | --- | --- | | 0a. | License | Apache 2.0 | From a3ceeadb8355c2b046248be96182dcbb61f45f44 Mon Sep 17 00:00:00 2001 From: Ryan Soury Date: Tue, 7 May 2024 17:04:28 +1000 Subject: [PATCH 04/11] links --- submissions/zkoracle-framework_milestone1_2024-05-07.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/submissions/zkoracle-framework_milestone1_2024-05-07.md b/submissions/zkoracle-framework_milestone1_2024-05-07.md index b9800715..e4b7e48b 100644 --- a/submissions/zkoracle-framework_milestone1_2024-05-07.md +++ b/submissions/zkoracle-framework_milestone1_2024-05-07.md @@ -40,7 +40,7 @@ Licensed under Apache 2.0. ## Formatted code -The [ZKAF R0 (Framework) codebase](https://github.com/usherlabs/zkaf-r0) and the X (Twitter) NFTs Game which involves the TLS Proof generation are both programmed in strictly typed Rust, for the most part. Comments are included among various steps in the logic to aid in understand the flow and processing of data. +The [ZKAF R0 (Framework) codebase](https://github.com/usherlabs/zkaf-r0) and the [X (Twitter) NFTs Game](https://github.com/usherlabs/x-twitter-nfts) which involves the TLS Proof generation are both programmed in strictly typed Rust, for the most part. Comments are included among various steps in the logic to aid in understand the flow and processing of data. The currently archived/deprecated [ZKAF (on zkLLVM) codebase](https://github.com/usherlabs/zkaf) includes Typescript for handling Smart Contract dependencies issues, however, is no longer relevant to demonstrate the working deliverables for this milestone. From 7e92c9c48da1b3ccfb5b74a502fbdeed9de53a40 Mon Sep 17 00:00:00 2001 From: Ryan Soury Date: Mon, 20 May 2024 20:18:59 +1000 Subject: [PATCH 05/11] update reference to specific commit relevant to the milestone --- .../zkoracle-framework_milestone1_2024-05-07.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/submissions/zkoracle-framework_milestone1_2024-05-07.md b/submissions/zkoracle-framework_milestone1_2024-05-07.md index e4b7e48b..edfe416c 100644 --- a/submissions/zkoracle-framework_milestone1_2024-05-07.md +++ b/submissions/zkoracle-framework_milestone1_2024-05-07.md @@ -78,12 +78,12 @@ Please ensure the repo has the correct open-source license. | Number | Deliverable | Links & Notes | | --- | --- | --- | | 0a. | License | Apache 2.0 | -| 0b. | Documentation | [LICENSE](https://github.com/usherlabs/zkaf-r0/blob/master/LICENSE) | -| 0c. | Testing Guide | [ZKAF Testing Guide](https://github.com/usherlabs/zkaf-r0/blob/master/README.md) and [TLS Proof Guide](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md) | +| 0b. | Documentation | [LICENSE](https://github.com/usherlabs/zkaf-r0/blob/8f54c1c563220db80bf07620d11d46fa1a8fe988/LICENSE) | +| 0c. | Testing Guide | [ZKAF Testing Guide](https://github.com/usherlabs/zkaf-r0/blob/8f54c1c563220db80bf07620d11d46fa1a8fe988/README.md) and [TLS Proof Guide](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md) | | 0d. | Docker | [Docker setup guide for deprecated ZKAF](https://github.com/usherlabs/zkaf?tab=readme-ov-file#build-the-image). Docker is incompatible with newer ZK proving systems due to GPU accelerated hardware requirements. | -| 1. | ZK Proof Generation Logic | [Development of boilerplate logic for ZK Proof Generation](https://github.com/usherlabs/zkaf-r0/blob/master/host/src/main.rs) of [sub-proofs produced by the Transparency Node](https://github.com/usherlabs/zkaf-r0/blob/master/methods/guest/src/main.rs). | -| 2. | Notary Proofs from Twitter API | [Implementation of logic](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/twitter/src/main.rs) to generate [Notary Proofs](https://github.com/usherlabs/zkaf-r0/blob/master/host/fixtures/twitter_proof.json) for data fetched from the Twitter API. | -| 3. | ZK Circuit Verification | [Development of a ZK Circuit](https://github.com/usherlabs/zkaf-r0/blob/master/methods/guest/src/main.rs) for [verifying the Notary Proofs (Sub-proofs)](https://github.com/usherlabs/zkaf-r0/blob/master/host/src/main.rs#L39). | +| 1. | ZK Proof Generation Logic | [Development of boilerplate logic for ZK Proof Generation](https://github.com/usherlabs/zkaf-r0/blob/8f54c1c563220db80bf07620d11d46fa1a8fe988/host/src/main.rs) of [sub-proofs produced by the Transparency Node](https://github.com/usherlabs/zkaf-r0/blob/8f54c1c563220db80bf07620d11d46fa1a8fe988/methods/guest/src/main.rs). | +| 2. | Notary Proofs from Twitter API | [Implementation of logic](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/twitter/src/main.rs) to generate [Notary Proofs](https://github.com/usherlabs/zkaf-r0/blob/8f54c1c563220db80bf07620d11d46fa1a8fe988/host/fixtures/twitter_proof.json) for data fetched from the Twitter API. | +| 3. | ZK Circuit Verification | [Development of a ZK Circuit](https://github.com/usherlabs/zkaf-r0/blob/8f54c1c563220db80bf07620d11d46fa1a8fe988/methods/guest/src/main.rs) for [verifying the Notary Proofs (Sub-proofs)](https://github.com/usherlabs/zkaf-r0/blob/8f54c1c563220db80bf07620d11d46fa1a8fe988/host/src/main.rs#L39). | ## Additional Information From 27c3694b499f407416554ea0ea99b8072b79164b Mon Sep 17 00:00:00 2001 From: Ryan Soury Date: Mon, 10 Jun 2024 02:34:46 +1000 Subject: [PATCH 06/11] draft of milestone2 for zkoracle framework --- ...koracle-framework_milestone2_2024-06-10.md | 113 ++++++++++++++++++ 1 file changed, 113 insertions(+) create mode 100644 submissions/zkoracle-framework_milestone2_2024-06-10.md diff --git a/submissions/zkoracle-framework_milestone2_2024-06-10.md b/submissions/zkoracle-framework_milestone2_2024-06-10.md new file mode 100644 index 00000000..18ef75c9 --- /dev/null +++ b/submissions/zkoracle-framework_milestone2_2024-06-10.md @@ -0,0 +1,113 @@ +# Milestone Delivery Guidelines + +These are the guidelines to be followed for milestones submitted for evaluation. + +## Content + +The submission should contain the following information: + +- zkOracle Framework for Real-world Data +- **Open-source code/delivery**: + - X (Twitter) NFTs Game: [https://github.com/usherlabs/x-twitter-nfts](https://github.com/usherlabs/x-twitter-nfts) + - - Framework (currently moved into the X NFTs Repo): [https://github.com/usherlabs/x-twitter-nfts/tree/main/src/zkaf](https://github.com/usherlabs/x-twitter-nfts/tree/main/src/zkaf) +- **[License](#license)** +- **[Documentation](#documentation)** +- **[Formatted code](#formatted-code), according to a set of guidelines** +- **[Testing Guide](#testing-guide)** +- **A list of the [milestone deliverables](#milestone-deliverables)** +- **Any [additional information](#additional-information)** + +## License + +Licensed under Apache 2.0. + +## Documentation + +This milestone encapsulates the interation of zkProofs into the Near Blockchain, using Aurora as a proxy for EVM (Groth16) SNARK verification. +An EVM platform is necessary, as currently, the ZK Prover natively generates parameters for Groth16 SNARKs on EVM blockchains. +While zkProof verification directly on Near is viable, and is currently being researched, this is yet to be implemented successfully. +Therefore, as per the original milestone deliverables, the Aurora EVM Blockchain that includes an SDKs for Aurora <> Near communication is used as a means to verify zkProofs, and establish a registry for verified zkProofs that encapsulate verification of social data from X (Twitter). + +While the full documentation, detailing deployment of the various contracts and operation of the ZK Prover can be found in the [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md), in this documentation we will highlight steps to: + +1. generate a zkProof of X (Twitter) data +2. verify the zkProof in an Aurora transaction +3. cross-reference this verification in a Near Smart Contract transaction, in order to authenticate subsequent steps - such as minting an NFT as per standards established by Mintbase. + +### Execution + +1. Navigate to the zkaf repo + Moved into X (Twitter) NFTs — https://github.com/usherlabs/twitter_notary/blob/feature/integrate-smart-contracts/src/zkaf + +2. (Optional) [Follow the deployment process outlined in the README.md](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md#smart-contracts) + +3. Install pre-requisite software libraries and technologies + 1. Install foundry [here](https://book.getfoundry.sh/getting-started/installation) + 2. Install the [necessary tool-chain to build the program](https://dev.risczero.com/api/zkvm/install) + 3. Install Docker + *To deterministically build the ZK Circuit / Guest, Docker must be running — [Learn more](https://dev.risczero.com/terminology#deterministic-builds)* + +4. Provide all environment variables and install prerequisites. + 1. See all requires env variables: https://github.com/usherlabs/x-twitter-nfts/blob/main/src/zkaf/.env.sample + 2. If you have not deployed the Smart Contracts using your own wallets, and do not have access to RiscZero's Bonsai managed ZK compute environment, Usher Labs can provide testnet Near and Aurora private keys and associated env variables on an ad-hoc basis, or provide a live demo of this execution. + +5. Run the publisher to generate and verify the proof. + + ```jsx + cargo run --bin publisher + ``` + +## Formatted code + +Originally, the [ZKAF R0 (Framework) codebase](https://github.com/usherlabs/zkaf-r0) and the [X (Twitter) NFTs Game](https://github.com/usherlabs/x-twitter-nfts) were separate repositories, establishing the fact that the framework for processing ZK Attestation is modularised and allow applications like the X (Twitter) NFT Game to adopt and use these modules to deliver a end-to-end verifiable data protocol that verifies social data within the Near Blockchain. + +However, for the sake of simplified development, the ZK AF has temporarily been merged into the X (Twitter) NFTs repository allowing for streamlined dependency management and operation. The goal of separating the framework into a standalone dependency of the X (Twitter) NFT Game will be achieved in a future milestone. + +**The code distribution:** + +1. Solidity Smart Contracts are designed to deploy to Aurora to manage the registry of verified zkProofs +2. Near Smart Contracts, and the ZK AF logic are programmed in strictly typed Rust. +3. Comments are included among various steps in the logic to aid in understand the flow and processing of data. + +## Testing Guide + +### Testing Near Smart Contracts + +1. `cd ./src/near/integration-tests` +2. `cargo test` + +### Testing Aurora Smart Contracts + +1. `cd ./src/zkaf` +2. `cargo build` +3. `forge build` +4. `forge test` + +### A step-by-step guide demonstrating how your code achieves the milestones + +A video has been created to walk you through execution of the project for this milestone. +Explaination is provided on how the code/logic achieves the outlined deliverables. + +[👉 See Demonstration]() + +### Unit tests + +Unit test have been developed for both the Near and Aurora Smart Contracts. +Follow the [Testing Guide](#testing-guide) to execute these unit tests. + +## Milestone Deliverables + + + +| Number | Deliverable | Specification | Links & Notes | +| --- | --- | --- | --- | +| 0a. | License | Apache 2.0 | [LICENSE](https://github.com/usherlabs/zkaf-r0/blob/8f54c1c563220db80bf07620d11d46fa1a8fe988/LICENSE) | +| 0b. | Documentation | Documentation on deploying NEAR and Aurora Smart Contracts to respective testnets and their integration using Aurora Contract SDK. | [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md) | +| 0c. | Testing Guide | Testing guide for Smart Contract functionality, with a focus on Docker operation for ZK Proof generation. | ZKAF Testing Guide is outlined in the [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md) both for unit tests of Smart Contracts across Near and Aurora, the deployment guide for this Smart Contracts, and TL;DR covers execution of logic that generates zkProof, verifies it, and uses verification as authentication for subsequent onchain process within Near. | +| 0d. | Docker | No updates in Docker setup; existing environment used for test proof generation. | Unlike the deliverable specification, Docker is now reincorporated as a primary mechanism through which zkProofs are built, as it ensures deterministic builds. This is a [paradigm set by RiscZero](https://dev.risczero.com/terminology#deterministic-builds), and this limitation is specific to the RiscZero ZK Co-processor. | +| 1. | Verification Smart Contracts | ZK Verification Smart Contracts leveraging the existing EVM ~~Placeholder~~ Proof System Verification | Placeholder replace with [R0's Groth16 Verification](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/zkaf/contracts/Verifier.sol) | +| 2. | NEAR Contract Integration | Integration of Aurora Smart Contract functionality through a NEAR Smart Contract entry point. | [Near Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/near/contract/src/lib.rs) | +| 3. | Smart Contract Deployment | Deployment of ZK Verification Smart Contracts to Aurora Testnet, and deployment of NEAR Smart Contracts to NEAR Testnet | The ["Publisher"](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf/apps/src/bin/publisher.rs#L11) encapsulates the zkProof generatation, [verification on Aurora](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf/apps/src/aurora.rs), and [cross-referencing the verification within Near](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf/apps/src/near.rs). Deployment of EVM Contracts is managed by [Forge](https://book.getfoundry.sh/). Near Smart Contracts are deployed using the Near CLI. This is detailed in the documentation. | +| 4. | Testing Framework | Development of a test-driven development framework for Smart Contract functionality. | Unit Tests are developed for both [Aurora Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/zkaf/tests/Verifier.t.sol), where execution is managed by [Forge](https://book.getfoundry.sh/), and [Near Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/near/integration-tests/src/lib.rs), where unit tests are essentially Rust Unit Tests | + +## Additional Information From 19e1f2a32dbcc4cf51ac974f4ff3c464fc8ed270 Mon Sep 17 00:00:00 2001 From: Ryan Soury Date: Mon, 10 Jun 2024 12:45:35 +1000 Subject: [PATCH 07/11] finalise milestone submission for zkoracle framework --- ...koracle-framework_milestone2_2024-06-10.md | 98 +++++++++---------- 1 file changed, 44 insertions(+), 54 deletions(-) diff --git a/submissions/zkoracle-framework_milestone2_2024-06-10.md b/submissions/zkoracle-framework_milestone2_2024-06-10.md index 18ef75c9..eb9e568c 100644 --- a/submissions/zkoracle-framework_milestone2_2024-06-10.md +++ b/submissions/zkoracle-framework_milestone2_2024-06-10.md @@ -1,21 +1,18 @@ -# Milestone Delivery Guidelines - -These are the guidelines to be followed for milestones submitted for evaluation. +# Milestone Delivery ## Content -The submission should contain the following information: +The submission includes the following information: -- zkOracle Framework for Real-world Data +- zkOracle Framework for Real-World Data - **Open-source code/delivery**: - X (Twitter) NFTs Game: [https://github.com/usherlabs/x-twitter-nfts](https://github.com/usherlabs/x-twitter-nfts) - - - Framework (currently moved into the X NFTs Repo): [https://github.com/usherlabs/x-twitter-nfts/tree/main/src/zkaf](https://github.com/usherlabs/x-twitter-nfts/tree/main/src/zkaf) + - Framework (integrated into the X NFTs Repo): [https://github.com/usherlabs/x-twitter-nfts/tree/main/src/zkaf](https://github.com/usherlabs/x-twitter-nfts/tree/main/src/zkaf) - **[License](#license)** - **[Documentation](#documentation)** -- **[Formatted code](#formatted-code), according to a set of guidelines** +- **[Formatted Code](#formatted-code), adhering to set guidelines** - **[Testing Guide](#testing-guide)** -- **A list of the [milestone deliverables](#milestone-deliverables)** -- **Any [additional information](#additional-information)** +- **List of [Milestone Deliverables](#milestone-deliverables)** ## License @@ -23,51 +20,50 @@ Licensed under Apache 2.0. ## Documentation -This milestone encapsulates the interation of zkProofs into the Near Blockchain, using Aurora as a proxy for EVM (Groth16) SNARK verification. -An EVM platform is necessary, as currently, the ZK Prover natively generates parameters for Groth16 SNARKs on EVM blockchains. -While zkProof verification directly on Near is viable, and is currently being researched, this is yet to be implemented successfully. -Therefore, as per the original milestone deliverables, the Aurora EVM Blockchain that includes an SDKs for Aurora <> Near communication is used as a means to verify zkProofs, and establish a registry for verified zkProofs that encapsulate verification of social data from X (Twitter). +This milestone encapsulates the integration of zkProofs into the Near Blockchain, using Aurora as a proxy for EVM (Groth16) SNARK verification. An EVM platform is necessary because the ZK Prover currently generates parameters for Groth16 SNARKs on EVM blockchains. + +While zkProof verification directly on Near is feasible and under research, it has not yet been successfully implemented. Therefore, as per the original milestone deliverables, the Aurora EVM Blockchain, which includes SDKs for Aurora <> Near communication, is used to verify zkProofs and establish a registry for verified zkProofs that encapsulate the verification of social data from X (Twitter). -While the full documentation, detailing deployment of the various contracts and operation of the ZK Prover can be found in the [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md), in this documentation we will highlight steps to: +The full documentation, detailing the deployment of various contracts and the operation of the ZK Prover, can be found in the [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md). In this documentation, we will highlight the steps to: -1. generate a zkProof of X (Twitter) data -2. verify the zkProof in an Aurora transaction -3. cross-reference this verification in a Near Smart Contract transaction, in order to authenticate subsequent steps - such as minting an NFT as per standards established by Mintbase. +1. Generate a zkProof of X (Twitter) data. +2. Verify the zkProof in an Aurora transaction. +3. Cross-reference this verification in a Near Smart Contract transaction to authenticate subsequent steps, such as minting an NFT according to standards established by Mintbase. ### Execution 1. Navigate to the zkaf repo - Moved into X (Twitter) NFTs — https://github.com/usherlabs/twitter_notary/blob/feature/integrate-smart-contracts/src/zkaf - -2. (Optional) [Follow the deployment process outlined in the README.md](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md#smart-contracts) - -3. Install pre-requisite software libraries and technologies - 1. Install foundry [here](https://book.getfoundry.sh/getting-started/installation) - 2. Install the [necessary tool-chain to build the program](https://dev.risczero.com/api/zkvm/install) - 3. Install Docker - *To deterministically build the ZK Circuit / Guest, Docker must be running — [Learn more](https://dev.risczero.com/terminology#deterministic-builds)* + - Moved into X (Twitter) NFTs — [https://github.com/usherlabs/twitter_notary/blob/feature/integrate-smart-contracts/src/zkaf](https://github.com/usherlabs/twitter_notary/blob/feature/integrate-smart-contracts/src/zkaf) + +2. *(Optional)* [Follow the deployment process outlined in the README.md](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md#smart-contracts) + +3. Install pre-requisite software libraries and technologies: + 1. Install Foundry [here](https://book.getfoundry.sh/getting-started/installation) + 2. Install the [necessary toolchain to build the program](https://dev.risczero.com/api/zkvm/install) + 3. Install Docker: + - To deterministically build the ZK Circuit / Guest, Docker must be running — [Learn more](https://dev.risczero.com/terminology#deterministic-builds) -4. Provide all environment variables and install prerequisites. - 1. See all requires env variables: https://github.com/usherlabs/x-twitter-nfts/blob/main/src/zkaf/.env.sample - 2. If you have not deployed the Smart Contracts using your own wallets, and do not have access to RiscZero's Bonsai managed ZK compute environment, Usher Labs can provide testnet Near and Aurora private keys and associated env variables on an ad-hoc basis, or provide a live demo of this execution. +4. Provide all environment variables + 1. See all required environment variables: [https://github.com/usherlabs/x-twitter-nfts/blob/main/src/zkaf/.env.sample](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/zkaf/.env.sample) + 2. If you have not deployed the Smart Contracts using your own wallets, and do not have access to RiscZero's Bonsai managed ZK compute environment, Usher Labs can provide testnet Near and Aurora private keys and associated environment variables on an ad-hoc basis, or provide a live demo of this execution. -5. Run the publisher to generate and verify the proof. +5. Run the publisher to generate and verify the proof: - ```jsx + ```sh cargo run --bin publisher ``` -## Formatted code +## Formatted Code -Originally, the [ZKAF R0 (Framework) codebase](https://github.com/usherlabs/zkaf-r0) and the [X (Twitter) NFTs Game](https://github.com/usherlabs/x-twitter-nfts) were separate repositories, establishing the fact that the framework for processing ZK Attestation is modularised and allow applications like the X (Twitter) NFT Game to adopt and use these modules to deliver a end-to-end verifiable data protocol that verifies social data within the Near Blockchain. +Originally, the [ZKAF R0 (Framework) codebase](https://github.com/usherlabs/zkaf-r0) and the [X (Twitter) NFTs Game](https://github.com/usherlabs/x-twitter-nfts) were separate repositories, demonstrating that the framework for processing ZK Attestation is composable. This allows applications like the X (Twitter) NFT Game to adopt and use these modules to deliver an end-to-end verifiable data protocol that verifies social data within the Near Blockchain. -However, for the sake of simplified development, the ZK AF has temporarily been merged into the X (Twitter) NFTs repository allowing for streamlined dependency management and operation. The goal of separating the framework into a standalone dependency of the X (Twitter) NFT Game will be achieved in a future milestone. +However, for simplified development, the [ZKAF has temporarily been merged into the X (Twitter) NFTs repository](https://github.com/usherlabs/x-twitter-nfts/tree/main/src/zkaf), allowing for streamlined dependency management and operation. The goal of separating the framework into a standalone dependency of the X (Twitter) NFT Game will be achieved in a future milestone. **The code distribution:** -1. Solidity Smart Contracts are designed to deploy to Aurora to manage the registry of verified zkProofs -2. Near Smart Contracts, and the ZK AF logic are programmed in strictly typed Rust. -3. Comments are included among various steps in the logic to aid in understand the flow and processing of data. +1. Solidity Smart Contracts are designed to deploy to Aurora to manage the registry of verified zkProofs. +2. Near Smart Contracts and the ZKAF logic are programmed in strictly typed Rust. +3. Comments are included among various steps in the logic to aid in understanding the flow and processing of data. ## Testing Guide @@ -83,31 +79,25 @@ However, for the sake of simplified development, the ZK AF has temporarily been 3. `forge build` 4. `forge test` -### A step-by-step guide demonstrating how your code achieves the milestones +### A Step-by-Step Guide Demonstrating How Code Achieves the Milestone -A video has been created to walk you through execution of the project for this milestone. -Explaination is provided on how the code/logic achieves the outlined deliverables. +A video has been created to walk you through the execution of the project for this milestone. The explanation is provided on how the code/logic achieves the outlined deliverables. -[👉 See Demonstration]() +[👉 See Demonstration](https://www.loom.com/share/09ab894833584744aefb58349b7659f4?sid=e56d8cc7-1722-4a02-bc32-e119b5263b9d) -### Unit tests +### Unit Tests -Unit test have been developed for both the Near and Aurora Smart Contracts. -Follow the [Testing Guide](#testing-guide) to execute these unit tests. +Unit tests have been developed for both the Near and Aurora Smart Contracts. Follow the [Testing Guide](#testing-guide) to execute these unit tests. ## Milestone Deliverables - - | Number | Deliverable | Specification | Links & Notes | | --- | --- | --- | --- | | 0a. | License | Apache 2.0 | [LICENSE](https://github.com/usherlabs/zkaf-r0/blob/8f54c1c563220db80bf07620d11d46fa1a8fe988/LICENSE) | | 0b. | Documentation | Documentation on deploying NEAR and Aurora Smart Contracts to respective testnets and their integration using Aurora Contract SDK. | [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md) | -| 0c. | Testing Guide | Testing guide for Smart Contract functionality, with a focus on Docker operation for ZK Proof generation. | ZKAF Testing Guide is outlined in the [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md) both for unit tests of Smart Contracts across Near and Aurora, the deployment guide for this Smart Contracts, and TL;DR covers execution of logic that generates zkProof, verifies it, and uses verification as authentication for subsequent onchain process within Near. | -| 0d. | Docker | No updates in Docker setup; existing environment used for test proof generation. | Unlike the deliverable specification, Docker is now reincorporated as a primary mechanism through which zkProofs are built, as it ensures deterministic builds. This is a [paradigm set by RiscZero](https://dev.risczero.com/terminology#deterministic-builds), and this limitation is specific to the RiscZero ZK Co-processor. | -| 1. | Verification Smart Contracts | ZK Verification Smart Contracts leveraging the existing EVM ~~Placeholder~~ Proof System Verification | Placeholder replace with [R0's Groth16 Verification](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/zkaf/contracts/Verifier.sol) | -| 2. | NEAR Contract Integration | Integration of Aurora Smart Contract functionality through a NEAR Smart Contract entry point. | [Near Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/near/contract/src/lib.rs) | -| 3. | Smart Contract Deployment | Deployment of ZK Verification Smart Contracts to Aurora Testnet, and deployment of NEAR Smart Contracts to NEAR Testnet | The ["Publisher"](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf/apps/src/bin/publisher.rs#L11) encapsulates the zkProof generatation, [verification on Aurora](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf/apps/src/aurora.rs), and [cross-referencing the verification within Near](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf/apps/src/near.rs). Deployment of EVM Contracts is managed by [Forge](https://book.getfoundry.sh/). Near Smart Contracts are deployed using the Near CLI. This is detailed in the documentation. | -| 4. | Testing Framework | Development of a test-driven development framework for Smart Contract functionality. | Unit Tests are developed for both [Aurora Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/zkaf/tests/Verifier.t.sol), where execution is managed by [Forge](https://book.getfoundry.sh/), and [Near Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/near/integration-tests/src/lib.rs), where unit tests are essentially Rust Unit Tests | - -## Additional Information +| 0c. | Testing Guide | Testing guide for Smart Contract functionality, with a focus on Docker operation for ZK Proof generation. | The ZKAF Testing Guide is outlined in the [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md) and covers unit tests for Smart Contracts on NEAR and Aurora, the deployment guide for these Smart Contracts, and a TL;DR on executing logic to generate zkProofs, verify them, and use verification as authentication for subsequent on-chain processes within NEAR. | +| 0d. | Docker | No updates in Docker setup; existing environment used for test proof generation. | Docker is now reincorporated as a primary mechanism through which zkProofs are built, ensuring deterministic builds. This is a [paradigm set by RiscZero](https://dev.risczero.com/terminology#deterministic-builds) and is specific to the RiscZero ZK Co-processor. | +| 1. | Verification Smart Contracts | ZK Verification Smart Contracts leveraging the existing EVM Proof System Verification. | Placeholder replaced with [R0's Groth16 Verification](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/zkaf/contracts/Verifier.sol) | +| 2. | NEAR Contract Integration | Integration of Aurora Smart Contract functionality through a NEAR Smart Contract entry point. | [NEAR Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/near/contract/src/lib.rs) | +| 3. | Smart Contract Deployment | Deployment of ZK Verification Smart Contracts to Aurora Testnet and deployment of NEAR Smart Contracts to NEAR Testnet. | The ["Publisher"](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf/apps/src/bin/publisher.rs#L11) encapsulates zkProof generation, [verification on Aurora](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf/apps/src/aurora.rs), and [cross-referencing the verification within NEAR](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf/apps/src/near.rs). Deployment of EVM Contracts is managed by [Forge](https://book.getfoundry.sh/). NEAR Smart Contracts are deployed using the NEAR CLI, as detailed in the documentation. | +| 4. | Testing Framework | Development of a test-driven development framework for Smart Contract functionality. | Unit tests are developed for both [Aurora Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/zkaf/tests/Verifier.t.sol), where execution is managed by [Forge](https://book.getfoundry.sh/), and [NEAR Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/near/integration-tests/src/lib.rs), where unit tests are written in Rust. | From 804a2971eb9ce24d5db2ae1bb782755d4472d843 Mon Sep 17 00:00:00 2001 From: Ryan Soury Date: Mon, 10 Jun 2024 12:48:41 +1000 Subject: [PATCH 08/11] finalise milestone submission for zkoracle framework --- ...zkoracle-framework_milestone2_2024-06-10.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/submissions/zkoracle-framework_milestone2_2024-06-10.md b/submissions/zkoracle-framework_milestone2_2024-06-10.md index eb9e568c..ad0ca097 100644 --- a/submissions/zkoracle-framework_milestone2_2024-06-10.md +++ b/submissions/zkoracle-framework_milestone2_2024-06-10.md @@ -24,7 +24,7 @@ This milestone encapsulates the integration of zkProofs into the Near Blockchain While zkProof verification directly on Near is feasible and under research, it has not yet been successfully implemented. Therefore, as per the original milestone deliverables, the Aurora EVM Blockchain, which includes SDKs for Aurora <> Near communication, is used to verify zkProofs and establish a registry for verified zkProofs that encapsulate the verification of social data from X (Twitter). -The full documentation, detailing the deployment of various contracts and the operation of the ZK Prover, can be found in the [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md). In this documentation, we will highlight the steps to: +The full documentation, detailing the deployment of various contracts and the operation of the ZK Prover, can be found in the [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/README.md). In this documentation, we will highlight the steps to: 1. Generate a zkProof of X (Twitter) data. 2. Verify the zkProof in an Aurora transaction. @@ -33,9 +33,9 @@ The full documentation, detailing the deployment of various contracts and the op ### Execution 1. Navigate to the zkaf repo - - Moved into X (Twitter) NFTs — [https://github.com/usherlabs/twitter_notary/blob/feature/integrate-smart-contracts/src/zkaf](https://github.com/usherlabs/twitter_notary/blob/feature/integrate-smart-contracts/src/zkaf) + - Moved into X (Twitter) NFTs — [src/zkaf](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf) -2. *(Optional)* [Follow the deployment process outlined in the README.md](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md#smart-contracts) +2. *(Optional)* [Follow the deployment process outlined in the README.md](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/README.md#smart-contracts) 3. Install pre-requisite software libraries and technologies: 1. Install Foundry [here](https://book.getfoundry.sh/getting-started/installation) @@ -44,7 +44,7 @@ The full documentation, detailing the deployment of various contracts and the op - To deterministically build the ZK Circuit / Guest, Docker must be running — [Learn more](https://dev.risczero.com/terminology#deterministic-builds) 4. Provide all environment variables - 1. See all required environment variables: [https://github.com/usherlabs/x-twitter-nfts/blob/main/src/zkaf/.env.sample](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/zkaf/.env.sample) + 1. See all required environment variables: [src/zkaf/.env.sample](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf/.env.sample) 2. If you have not deployed the Smart Contracts using your own wallets, and do not have access to RiscZero's Bonsai managed ZK compute environment, Usher Labs can provide testnet Near and Aurora private keys and associated environment variables on an ad-hoc basis, or provide a live demo of this execution. 5. Run the publisher to generate and verify the proof: @@ -94,10 +94,10 @@ Unit tests have been developed for both the Near and Aurora Smart Contracts. Fol | Number | Deliverable | Specification | Links & Notes | | --- | --- | --- | --- | | 0a. | License | Apache 2.0 | [LICENSE](https://github.com/usherlabs/zkaf-r0/blob/8f54c1c563220db80bf07620d11d46fa1a8fe988/LICENSE) | -| 0b. | Documentation | Documentation on deploying NEAR and Aurora Smart Contracts to respective testnets and their integration using Aurora Contract SDK. | [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md) | -| 0c. | Testing Guide | Testing guide for Smart Contract functionality, with a focus on Docker operation for ZK Proof generation. | The ZKAF Testing Guide is outlined in the [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/main/README.md) and covers unit tests for Smart Contracts on NEAR and Aurora, the deployment guide for these Smart Contracts, and a TL;DR on executing logic to generate zkProofs, verify them, and use verification as authentication for subsequent on-chain processes within NEAR. | +| 0b. | Documentation | Documentation on deploying NEAR and Aurora Smart Contracts to respective testnets and their integration using Aurora Contract SDK. | [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/README.md) | +| 0c. | Testing Guide | Testing guide for Smart Contract functionality, with a focus on Docker operation for ZK Proof generation. | The ZKAF Testing Guide is outlined in the [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/README.md) and covers unit tests for Smart Contracts on NEAR and Aurora, the deployment guide for these Smart Contracts, and a TL;DR on executing logic to generate zkProofs, verify them, and use verification as authentication for subsequent on-chain processes within NEAR. | | 0d. | Docker | No updates in Docker setup; existing environment used for test proof generation. | Docker is now reincorporated as a primary mechanism through which zkProofs are built, ensuring deterministic builds. This is a [paradigm set by RiscZero](https://dev.risczero.com/terminology#deterministic-builds) and is specific to the RiscZero ZK Co-processor. | -| 1. | Verification Smart Contracts | ZK Verification Smart Contracts leveraging the existing EVM Proof System Verification. | Placeholder replaced with [R0's Groth16 Verification](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/zkaf/contracts/Verifier.sol) | -| 2. | NEAR Contract Integration | Integration of Aurora Smart Contract functionality through a NEAR Smart Contract entry point. | [NEAR Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/near/contract/src/lib.rs) | +| 1. | Verification Smart Contracts | ZK Verification Smart Contracts leveraging the existing EVM Proof System Verification. | Placeholder replaced with [R0's Groth16 Verification](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf/contracts/Verifier.sol) | +| 2. | NEAR Contract Integration | Integration of Aurora Smart Contract functionality through a NEAR Smart Contract entry point. | [NEAR Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/near/contract/src/lib.rs) | | 3. | Smart Contract Deployment | Deployment of ZK Verification Smart Contracts to Aurora Testnet and deployment of NEAR Smart Contracts to NEAR Testnet. | The ["Publisher"](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf/apps/src/bin/publisher.rs#L11) encapsulates zkProof generation, [verification on Aurora](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf/apps/src/aurora.rs), and [cross-referencing the verification within NEAR](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf/apps/src/near.rs). Deployment of EVM Contracts is managed by [Forge](https://book.getfoundry.sh/). NEAR Smart Contracts are deployed using the NEAR CLI, as detailed in the documentation. | -| 4. | Testing Framework | Development of a test-driven development framework for Smart Contract functionality. | Unit tests are developed for both [Aurora Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/main/src/zkaf/tests/Verifier.t.sol), where execution is managed by [Forge](https://book.getfoundry.sh/), and [NEAR Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/near/integration-tests/src/lib.rs), where unit tests are written in Rust. | +| 4. | Testing Framework | Development of a test-driven development framework for Smart Contract functionality. | Unit tests are developed for both [Aurora Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf/tests/Verifier.t.sol), where execution is managed by [Forge](https://book.getfoundry.sh/), and [NEAR Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/9a10a819d982c6ff0d90238f636051cf8d069f13/src/near/integration-tests/src/lib.rs), where unit tests are written in Rust. | From b7a1e931188eb0c596d1d74e87819de09422690a Mon Sep 17 00:00:00 2001 From: Ryan Soury Date: Mon, 10 Jun 2024 12:53:22 +1000 Subject: [PATCH 09/11] finalise milestone submission for zkoracle framework --- submissions/zkoracle-framework_milestone2_2024-06-10.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/submissions/zkoracle-framework_milestone2_2024-06-10.md b/submissions/zkoracle-framework_milestone2_2024-06-10.md index ad0ca097..fbe7f015 100644 --- a/submissions/zkoracle-framework_milestone2_2024-06-10.md +++ b/submissions/zkoracle-framework_milestone2_2024-06-10.md @@ -57,7 +57,7 @@ The full documentation, detailing the deployment of various contracts and the op Originally, the [ZKAF R0 (Framework) codebase](https://github.com/usherlabs/zkaf-r0) and the [X (Twitter) NFTs Game](https://github.com/usherlabs/x-twitter-nfts) were separate repositories, demonstrating that the framework for processing ZK Attestation is composable. This allows applications like the X (Twitter) NFT Game to adopt and use these modules to deliver an end-to-end verifiable data protocol that verifies social data within the Near Blockchain. -However, for simplified development, the [ZKAF has temporarily been merged into the X (Twitter) NFTs repository](https://github.com/usherlabs/x-twitter-nfts/tree/main/src/zkaf), allowing for streamlined dependency management and operation. The goal of separating the framework into a standalone dependency of the X (Twitter) NFT Game will be achieved in a future milestone. +However, for simplified development, the [ZKAF has temporarily been merged into the X (Twitter) NFTs repository](https://github.com/usherlabs/x-twitter-nfts/tree/9a10a819d982c6ff0d90238f636051cf8d069f13/src/zkaf), allowing for streamlined dependency management and operation. The goal of separating the framework into a standalone dependency of the X (Twitter) NFT Game will be achieved in a future milestone. **The code distribution:** From 1349dd99f4e677fbbc6f99a2a5b71895e94fad0f Mon Sep 17 00:00:00 2001 From: Ryan Soury Date: Thu, 11 Jul 2024 15:48:53 +1000 Subject: [PATCH 10/11] new milestone3 for zkoracle framework --- ...koracle-framework_milestone3_2024-07-11.md | 108 ++++++++++++++++++ 1 file changed, 108 insertions(+) create mode 100644 submissions/zkoracle-framework_milestone3_2024-07-11.md diff --git a/submissions/zkoracle-framework_milestone3_2024-07-11.md b/submissions/zkoracle-framework_milestone3_2024-07-11.md new file mode 100644 index 00000000..42c2ee2c --- /dev/null +++ b/submissions/zkoracle-framework_milestone3_2024-07-11.md @@ -0,0 +1,108 @@ +# Milestone Delivery + +## Content + +The submission includes the following information: + +- Mintbase Integration and Metadata Research +- **Open-source code/delivery**: + - X (Twitter) NFTs Game: [https://github.com/usherlabs/x-twitter-nfts](https://github.com/usherlabs/x-twitter-nfts) + - Framework (integrated into the X NFTs Repo): [https://github.com/usherlabs/x-twitter-nfts/tree/1b5f84536baf1de04b4cb58090ba762cdbbead8a/src/zkaf](https://github.com/usherlabs/x-twitter-nfts/tree/1b5f84536baf1de04b4cb58090ba762cdbbead8a/src/zkaf) + - Near NFT Contract: [https://github.com/usherlabs/x-twitter-nfts/tree/1b5f84536baf1de04b4cb58090ba762cdbbead8a/src/near_nft](https://github.com/usherlabs/x-twitter-nfts/tree/1b5f84536baf1de04b4cb58090ba762cdbbead8a/src/near_nft) + - Near Verifier Contract: [https://github.com/usherlabs/x-twitter-nfts/tree/1b5f84536baf1de04b4cb58090ba762cdbbead8a/src/near_verifier](https://github.com/usherlabs/x-twitter-nfts/tree/1b5f84536baf1de04b4cb58090ba762cdbbead8a/src/near_verifier) +- **[License](#license)** +- **[Documentation](#documentation)** +- **[Formatted Code](#formatted-code), adhering to set guidelines** +- **[Testing Guide](#testing-guide)** +- **List of [Milestone Deliverables](#milestone-deliverables)** + +## License + +Licensed under Apache 2.0. + +## Documentation + +This milestone focuses on the integration of Mintbase for NFT creation and the research on metadata handling. The NEAR Smart Contract has been expanded to include callback functionality for Mintbase NFT minting. Additionally, research has been conducted on various data availability options for NFT metadata, including on-chain storage, ZK Proof, and Arweave. + +The full documentation, detailing the deployment of various contracts and the operation of the ZK Prover, can be found in the [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/1b5f84536baf1de04b4cb58090ba762cdbbead8a/README.md). In this documentation, we will highlight the steps to: + +1. Integrate verification for NFT creation. +2. Handle metadata using ZK enabled hash check verification, with on-chain metadata storage. Media to be stored off-chain. +3. Verify ZK Proof and metadata hash before minting NFT +4. Integration and unit testing. + +### Execution + +1. Navigate to the zkaf repo - [src/zkaf](https://github.com/usherlabs/x-twitter-nfts/blob/1b5f84536baf1de04b4cb58090ba762cdbbead8a/src/zkaf) + +2. *(Optional)* [Follow the deployment process outlined in the README.md](https://github.com/usherlabs/x-twitter-nfts/blob/1b5f84536baf1de04b4cb58090ba762cdbbead8a/README.md#smart-contracts) + +3. Install pre-requisite software libraries and technologies: + 1. Install Foundry [here](https://book.getfoundry.sh/getting-started/installation) + 2. Install the [necessary toolchain to build the program](https://dev.risczero.com/api/zkvm/install) + 3. Install Docker: + - To deterministically build the ZK Circuit / Guest, Docker must be running — [Learn more](https://dev.risczero.com/terminology#deterministic-builds) + +4. Provide all environment variables + 1. See all required environment variables: [src/zkaf/.env.sample](https://github.com/usherlabs/x-twitter-nfts/blob/1b5f84536baf1de04b4cb58090ba762cdbbead8a/src/zkaf/.env.sample) + 2. If you have not deployed the Smart Contracts using your own wallets, and do not have access to RiscZero's Bonsai managed ZK compute environment, Usher Labs can provide testnet Near and Aurora private keys and associated environment variables on an ad-hoc basis, or provide a live demo of this execution. + +5. Run the publisher to generate and verify the proof: + + ```shell + cargo run --bin publisher + ``` + +6. Response will indicate a ZK verification and NFT token ID. + + +## Formatted Code + +**The code distribution:** + +1. Solidity Smart Contracts are designed to deploy to Aurora to manage the registry of verified zkProofs. +2. Near Smart Contracts and the ZK logic are programmed in strictly typed Rust. +3. Comments are included among various steps in the logic to aid in understanding the flow and processing of data. + +## Testing Guide + +### Testing Near NFT Smart Contract + +1. `cd ./src/near_nft` +2. `cargo test` + +### Testing Near Verifier Smart Contract + +1. `cd ./src/near_verifier/integration-tests` +2. `cargo test` + +### Testing Aurora Smart Contracts + +1. `cd ./src/zkaf` +2. `cargo build` +3. `forge build` +4. `forge test` + +### A Step-by-Step Guide Demonstrating How Code Achieves the Milestone + +A video has been created to walk you through the execution of the project for this milestone. The explanation is provided on how the code/logic achieves the outlined deliverables. + +- [🌁 See Architecture Walkthrough](https://www.loom.com/share/846468ac308d4fe5badc96ca9d93c6ea?sid=9e635dd5-08eb-428d-bc1d-4c67a87db760) +- [🤖 See Code Walkthrough](https://www.loom.com/share/4f77e44f93d6487fa28fac13d90c2cec?sid=8d823fde-a9e9-4f50-8141-c093e5f8bc36) +- [🦾 See Terminal Demonstration](https://www.loom.com/share/05daa7ac3996442082405ac05db31dec?sid=6b3e246b-d8c9-4bb7-bd65-9192c6a1532d) + +### Unit Tests + +Unit tests have been developed for both the Near and Aurora Smart Contracts. Follow the [Testing Guide](#testing-guide) to execute these unit tests. + +## Milestone Deliverables + +| Number | Deliverable | Specification | Links & Notes | +| --- | --- | --- | --- | +| 0a. | License | Apache 2.0 | [LICENSE](https://github.com/usherlabs/zkaf-r0/blob/1b5f84536baf1de04b4cb58090ba762cdbbead8a/LICENSE) | +| 0b. | Documentation | Documentation on NEAR Smart Contract callback functionality for Mintbase NFT minting integration and metadata handling research. | [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/1b5f84536baf1de04b4cb58090ba762cdbbead8a/README.md?tab=readme-ov-file#near-smart-contract-callback-functionality-for-nfts) | +| 0c. | Testing Guide | Expanded testing suite incorporating NFT creation callback mechanisms. | The ZKAF Testing Guide is outlined in the [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/1b5f84536baf1de04b4cb58090ba762cdbbead8a/README.md) and covers unit tests for Smart Contracts on NEAR and Aurora, the deployment guide for these Smart Contracts, and a TL;DR on executing logic to generate zkProofs, verify them, and use verification as authentication for subsequent on-chain processes within NEAR. | +| 0d. | Docker | No Docker updates required for this milestone. | Docker is now reincorporated as a primary mechanism through which zkProofs are built, ensuring deterministic builds. This is a [paradigm set by RiscZero](https://dev.risczero.com/terminology#deterministic-builds) and is specific to the RiscZero ZK Co-processor. | +| 1. | Callback Mechanism & Mintbase Module Integration | Expansion of NEAR Smart Contract callback functionality for NFT creation on Mintbase. | [NEAR Verifier Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/tree/1b5f84536baf1de04b4cb58090ba762cdbbead8a/src/near_verifier) | +| 2. | Metadata Handling Research | Please refer to the ZK proof of metadata hash and hash check verification to understand how metadata is directly and verifiably associated with the X Post from the API. This metadata is deemed appropriate for on-chain storage according to this verified paradigm, and Near storage supports such a paradigm. Furthermore, all media items will remain off-chain with references to Arweave or IPFS, to be re-incorporated in the final milestone. | [Research Outcome](https://github.com/usherlabs/x-twitter-nfts/blob/1b5f84536baf1de04b4cb58090ba762cdbbead8a/src/near_verifier/contract/src/lib.rs#L63) | +| 3. | Unit Testing | Comprehensive development of unit tests for the integrated NFT creation process. | Unit tests are developed for both [Aurora Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/1b5f84536baf1de04b4cb58090ba762cdbbead8a/src/zkaf/tests/Verifier.t.sol), where execution is managed by [Forge](https://book.getfoundry.sh/), and [NEAR Smart Contracts](https://github.com/usherlabs/x-twitter-nfts/blob/1b5f84536baf1de04b4cb58090ba762cdbbead8a/src/near/integration-tests/src/lib.rs), where unit tests are written in Rust. | \ No newline at end of file From 5bc50772ecbdf2278b6607e099fd2f8642d55588 Mon Sep 17 00:00:00 2001 From: Ryan Soury Date: Mon, 30 Dec 2024 18:08:56 +1100 Subject: [PATCH 11/11] milestone4 and milestone5 submissions for the zkOracle Framework / X NFTs project --- ...koracle-framework_milestone4_2024-12-30.md | 93 +++++++++++++++++++ ...koracle-framework_milestone5_2024-12-30.md | 92 ++++++++++++++++++ 2 files changed, 185 insertions(+) create mode 100644 submissions/zkoracle-framework_milestone4_2024-12-30.md create mode 100644 submissions/zkoracle-framework_milestone5_2024-12-30.md diff --git a/submissions/zkoracle-framework_milestone4_2024-12-30.md b/submissions/zkoracle-framework_milestone4_2024-12-30.md new file mode 100644 index 00000000..a2741dbf --- /dev/null +++ b/submissions/zkoracle-framework_milestone4_2024-12-30.md @@ -0,0 +1,93 @@ +# Milestone Delivery + +## Content + +The submission includes the following information: + +- Refinement and Modularisation +- **Open-source code/delivery**: + - Verity zkTLS Data Processor Framework: [https://github.com/usherlabs/verity-dp](https://github.com/usherlabs/verity-dp) + - zkVM for TLS Proof Rollups and ZK Verifier Contracts: [https://github.com/usherlabs/x-twitter-nfts/blob/a79999a68801b8ecf2a29d2a4892cde6c5f8e933/src/zkaf/README.md](https://github.com/usherlabs/x-twitter-nfts/blob/a79999a68801b8ecf2a29d2a4892cde6c5f8e933/src/zkaf/README.md) + - X (Twitter) NFTs Project: [https://github.com/usherlabs/x-twitter-nfts](https://github.com/usherlabs/x-twitter-nfts) + - Near Core Contract: [https://github.com/usherlabs/x-twitter-nfts/blob/a79999a68801b8ecf2a29d2a4892cde6c5f8e933/src/contracts/README.md](https://github.com/usherlabs/x-twitter-nfts/blob/a79999a68801b8ecf2a29d2a4892cde6c5f8e933/src/contracts/README.md) +- **[Documentation](#documentation)** +- **[Formatted Code](#formatted-code), adhering to set guidelines** +- **[Testing Guide](#testing-guide)** +- **List of [Milestone Deliverables](#milestone-deliverables)** + +## License + +Licensed under Apache 2.0. + +## Documentation + +This milestone focuses on the refinement and modularisation of the X (Twitter) NFTs repository. In a prior milestone, the repository was disorganised, with code for various functionalities distributed across folders. Now, each folder has its own specific purpose, and any additional modules and functions that can be used across projects incorporating Usher Labs' zkTLS Protocol can be found in a [dedicated repository](https://github.com/usherlabs/verity-dp). + +The X NFTs project folder structure is as follows: + +- `src/bitte_plugin`: The Bitte Plugin includes an API that allows Bitte AI to interact with the X NFTs project. +- `src/near_indexer`: The Near Indexer orchestrates the X NFTs project, including Near NFT Contract Indexing and ZK Prover processes management. +- `src/zkaf`: The zkAF utility generates the ZK Prover ELF binary, loaded into `src/near_indexer/src/generated/methods.rs`. It includes zkVM logic and code generation for the ZK Verifier Smart Contracts. +- `src/contracts`: The Near Contracts represent the X NFTs project on the Near blockchain. + - `src/contracts/nft`: The Near NFT Contract manages mint intents, cancellations, and verifiable metadata from the ZK Prover. + - `src/contracts/verifier`: The ZK Verifier Contract verifies ZK Proofs on the Aurora blockchain. A Near <> Aurora message prootocol is used to bridge verified proofs from the EVM environment to the NEAR environment. + +### Workflow + +1. Request a Tweet to be minted via your Bitte Wallet. +2. Use AI to generate NFT art or use an automatic snapshot of the Tweet. +3. Provide a Twitter handle for notifications once the zkTLS proof is verified on-chain. +4. Wait for zkProof verification and NFT minting to complete. +5. **Brag and bet with your friends on whether your 1-of-1 X NFT will represent a Tweet that will go viral!** + +### Execution + +The full setup and execution flow for the X NFTs project is outlined in the [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/a79999a68801b8ecf2a29d2a4892cde6c5f8e933/README.md). Each component/folder within the repository also has its own README.md with instructions relative to the component. + +## Formatted Code + +**The code distribution:** + +1. Solidity Smart Contracts are designed to deploy to Aurora to manage the registry of verified zkProofs. +2. Near Core Contracts are programmed in strictly typed Rust. +3. zkVM logic for TLS proof rollup is programmed in strictly typed Rust. +4. Bitte Plugin API is developed in Rust, implemented through the Rocket.rs API framework. +5. Comments are included among various steps in the logic to aid in understanding the flow and processing of data. + +## Testing Guide + +### Testing Near NFT Smart Contract + +1. `cd ./src/near_nft` +2. `cargo test` + +### Testing Aurora Smart Contracts + +1. `cd ./src/zkaf` +2. `cargo build` +3. `forge build` +4. `forge test` + +### A Step-by-Step Guide Demonstrating How Code Achieves the Milestone + +A video has been created to walk you through the execution of the project for this milestone. The explanation is provided on how the code/logic achieves the outlined deliverables. + +- [🌁 See Workflow Walkthrough](https://www.loom.com/share/78595bceeedc4e95bac4a291588773d3?sid=a25f0721-9766-40ba-8ec3-06d7af1272d0) +- [🤖 See Code Walkthrough](https://www.loom.com/share/a808fe84f97f4b86b0ff58c32e693711?sid=80f1b04b-4df6-494c-a06d-ae8a421d4fc6) + +### Unit Tests + +Unit tests have been developed for both the Near and Aurora Smart Contracts. Follow the [Testing Guide](#testing-guide) to execute these unit tests. + +## Milestone Deliverables + +| Number | Deliverable | Specification | Links & Notes | +| --- | --- | --- | --- | +| 0a. | License | Apache 2.0 | [LICENSE](https://github.com/usherlabs/x-twitter-nfts/blob/4af72cebba82a5c73b3f3aec5448ebfeda5fac39/LICENSE) | +| 0b. | Documentation | Documentation detailing the separation of the zkOracle Framework and Twitter NFTs Example, including Docker service separation. | [README.md](https://github.com/usherlabs/x-twitter-nfts/tree/feature/x-nfts-v1?tab=readme-ov-file#getting-started) | +| 0c. | Testing Guide | Distinguishing testing procedures for zkOracle Framework and Twitter NFTs Example. | zkTLS (previously named zkOracle) tests are isolated to the separate framework covering data processing for the [Verity zkTLS protocol](https://github.com/usherlabs/verity-dp). Tests for the X NFTs project are isolated to each of the components that comprise the project. | +| 0d. | Docker | Docker setup update to separate services by their respective purposes. | Docker is now reincorporated as a primary mechanism through which zkProofs are built, ensuring deterministic builds. This is a [paradigm set by RiscZero](https://dev.risczero.com/terminology#deterministic-builds) and is specific to the RiscZero ZK Co-processor. | +| 1. | ZK Circuit Refinement | Refinement of the ZK Circuit for verifying Twitter Data Sub-proofs. | [zkVM Logic](https://github.com/usherlabs/x-twitter-nfts/blob/edca0b7636bd4efa7cae5b91d82fe279c163f4fb/src/zkaf/methods/guest/src/bin/verify.rs) | +| 2. | Framework Modularisation | Modularisation of the zkOracle Framework to abstract from the Twitter API +Integration. | [Verity DP ZK libraries](https://github.com/usherlabs/verity-dp/tree/main/examples/zktls) and X NFTs [supported binaries.](https://github.com/usherlabs/x-twitter-nfts/blob/3705f04bb385749bf3207422462f011716118026/src/zkaf/apps/src/bin/publisher.rs) | +| 3. | Orchestration Layer Development | Development of an Orchestration Layer for the zkOracle Framework, facilitating frontend feedback. | The Orchestrator is the [`near_indexer` component](https://github.com/usherlabs/x-twitter-nfts/blob/2dd039c3d0834c228fc96a9673bfa0be0a672930/src/near_indexer/src/main.rs) in the X NFTs project. | \ No newline at end of file diff --git a/submissions/zkoracle-framework_milestone5_2024-12-30.md b/submissions/zkoracle-framework_milestone5_2024-12-30.md new file mode 100644 index 00000000..fe31a37b --- /dev/null +++ b/submissions/zkoracle-framework_milestone5_2024-12-30.md @@ -0,0 +1,92 @@ +# Milestone Delivery + +## Content + +The submission includes the following information: + +- NFT Minting dApp Deployment and Testing +- **Open-source code/delivery**: + - Bitte Plugin for NFT Minting UX: [Bitte Plugin](https://github.com/usherlabs/x-twitter-nfts/blob/83de18a1291319d46a878c1395f407bb123b6e2c/src/bitte_plugin/README.md) +- **[Documentation](#documentation)** +- **[Formatted Code](#formatted-code), adhering to set guidelines** +- **[Testing Guide](#testing-guide)** +- **List of [Milestone Deliverables](#milestone-deliverables)** + +## License + +Licensed under Apache 2.0. + +## Documentation + +This milestone centres on the production readiness of the X NFTs project. It primarily focuses on the Bitte Plugin for NFT Minting UX, which is a plugin that allows Bitte AI to interact with the X NFTs project. + +### Workflow + +1. Request a Tweet to be minted via your Bitte Wallet. +2. Use AI to generate NFT art or use an automatic snapshot of the Tweet. +3. Provide a Twitter handle for notifications once the zkTLS proof is verified on-chain. +4. Wait for zkProof verification and NFT minting to complete. +5. **Brag and bet with your friends on whether your 1-of-1 X NFT will represent a Tweet that will go viral!** + +### Execution + +The full setup and execution flow for the X NFTs project is outlined in the [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/a79999a68801b8ecf2a29d2a4892cde6c5f8e933/README.md). Each component/folder within the repository also has its own README.md with instructions relative to the component. + +## Formatted Code + +**The code distribution:** + +1. Solidity Smart Contracts are designed to deploy to Aurora to manage the registry of verified zkProofs. +2. Near Core Contracts are programmed in strictly typed Rust. +3. zkVM logic for TLS proof rollup is programmed in strictly typed Rust. +4. Bitte Plugin API is developed in Rust, implemented through the Rocket.rs API framework. +5. Comments are included among various steps in the logic to aid in understanding the flow and processing of data. + +## Testing Guide + +### Testing Bitte Plugin + +1. Start the server: + + ```bash + cargo run + ``` + +2. Run the agent on the mainnet: + + ```bash + npx make-agent dev -p 8007 + ``` + +3. Run the agent on the testnet: + + ```bash + npx make-agent dev -p 8007 -t + ``` + +### A Step-by-Step Guide Demonstrating How Code Achieves the Milestone + +A video has been created to walk you through the execution of the project for this milestone. The explanation is provided on how the code/logic achieves the outlined deliverables. + +- [🌁 See Workflow Walkthrough](https://www.loom.com/share/78595bceeedc4e95bac4a291588773d3?sid=a25f0721-9766-40ba-8ec3-06d7af1272d0) +- [🤖 See Code Walkthrough](https://www.loom.com/share/a808fe84f97f4b86b0ff58c32e693711?sid=80f1b04b-4df6-494c-a06d-ae8a421d4fc6) + +## Milestone Deliverables + +| Number | Deliverable | Specification | Links & Notes | +| --- | --- | --- | --- | +| 0a. | License | Apache 2.0 | [LICENSE](https://github.com/usherlabs/x-twitter-nfts/blob/4af72cebba82a5c73b3f3aec5448ebfeda5fac39/LICENSE) | +| 0b. | Documentation | Comprehensive documentation on the modified NFT minting dApp, including integration details with NEAR/Mintbase libraries. Reference to third-party packages used. | [README.md](https://github.com/usherlabs/x-twitter-nfts/blob/83de18a1291319d46a878c1395f407bb123b6e2c/src/bitte_plugin/README.md) | +| 0c. | Testing Guide | End-to-end testing guide for the NFT minting process with zkProof verification. | [🌁 See Workflow Walkthrough](https://www.loom.com/share/78595bceeedc4e95bac4a291588773d3?sid=a25f0721-9766-40ba-8ec3-06d7af1272d0) | +| 0d. | Docker | Deployment of Docker environment for the final dApp. | No docker related changes for this deliverable. | +| 0e. | Article | Publication detailing the project's achievements, impact on the Mintbase, NEAR, and wider blockchain +ecosystems. | [Preliminary publications](https://x.com/usher_web3/status/1818285628292792577) have been made preparing for the release of the Bitte Plugin. Co-marketing initiatives are advised alongside preparation of a full-length blog post. | +| 1. | dApp Modification and Integration | Modification and deployment of an existing open-source NFT Minting dApp, +incorporating NEAR/Mintbase libraries for minting NFTs with zkProof verification from the zkOracle. Reference to +third-party packages like https://github.com/AnishDe12020/twnft. | As per discussions with the Bitte/Mintbase team, we found a better solution to integrate a new **X NFTs - Assistant** [Bitte Plugin](https://github.com/usherlabs/x-twitter-nfts/blob/f268186395302354babb08f8279b842eec06ae53/src/bitte_plugin/src/main.rs) in place of the NFT minting dApp. | +| 2. | Smart Contract Deployment | Deployment of the complete suite of Smart Contracts to Mainnets. | Near Core Contracts are deployed to Testnet successfully. + +- Near NFT Contract: [https://testnet.nearblocks.io/address/x-bitte-nft.testnet](https://testnet.nearblocks.io/address/x-bitte-nft.testnet) +- Near Verifier Contract: [https://testnet.nearblocks.io/address/local-verifier.testnet](https://testnet.nearblocks.io/address/local-verifier.testnet) +As the project is prepared with marketing for production deployment, the same guidelines for deployment can be applied.| +| 3. | E2E Testing | Conducting end-to-end tests to ensure unique Twitter Data minting and listing on the Mintbase Marketplace. | Alongside production deployment, with co-marketing initiatives, we aim to re-conduct the full e2e testing flow shown here: [https://www.loom.com/share/78595bceeedc4e95bac4a291588773d3?sid=a25f0721-9766-40ba-8ec3-06d7af1272d0](https://www.loom.com/share/78595bceeedc4e95bac4a291588773d3?sid=a25f0721-9766-40ba-8ec3-06d7af1272d0) | \ No newline at end of file