Skip to main content

CoinFabrik On Ink Integration Tests

  • Team Name: CoinFabrik (Nektra S.A)
  • Payment Address: 0xf488039EDe6B38D7689fDCC6A9FC2dd0EF39D54e (USDC)
  • Level: 2

Project Overview ๐Ÿ“„โ€‹


CoinFabrik has successfully accomplished two previous grant milestones for the development of Scout, an open source bug-detection tool for ink! targetted to developers and smart contract auditors.

We have completed a Proof of Concept and a Prototype.

During these iterations, we encountered certain challenges, especially while developing fuzzing detectors. These challenges included limitations in integration tests and differences with the ink! E2E testing environment. Specifically, we faced difficulties when working with functions in integration testing related to cross contract calls, storage, gas usage, and delegatecall.

With this grant, our objective is to conduct a comprehensive analysis to identify any other missing functionalities in integration tests, and to propose and develop new testing features based on our findings.

Our intention is to flatten the anvil of ink! integration testing. With a properly flattened anvil, quality tools can be built.

Project Detailsโ€‹

As mentioned before, we have already identified a number of E2E functionalities that are not present in integration tests.

For example:

  • A different implementation of storage in integration testing vs E2E testing. The same storage limitations present in E2E should also be present in integration testing.
  • Inability to perform delegatecall in integration tests.
  • Inability to perform cross contract calls in integration tests.
  • Gas consumption is not integrated.
  • Address inconsistencies. Alice and Bob addresses differ between integration and E2E tests environments.

This list is not exhaustive, as it simply highlights some difficulties we encountered while conducting integration tests for vulnerability examples during the development of detectors for our static analyzer, Scout.

The overall advantage of integration tests is that, since they are performed off-chain, they are significantly faster than E2E tests, which imply compiling and deploying the smart contract to a Substrate node. We believe that having a complete set of functionalities for integration tests will allow developers to thoroughly test their code more quickly. Particularly, we believe the impossibility to perform cross contract calls in integration tests is a limitation for developers trying to quickly assess interactions across different contracts. This can be done through E2E tests, but it takes more time.

Our proposal is to begin our work by making a full review of the current functionalities of integration tests and E2E tests. From this revision, we will assemble a comparative table, identifying differences and proposing improvements and missing developments to be made for integration tests. For specific cases where the enhancement or missing functionality is clear, and the implementation of the enhancement is deemed feasible, code examples could be provided to show the current limitations of integration tests.

Furthermore, we need to thoroughly analyze the complexity and feasibility of generating the necessary tests. The analysis we just mentioned will allow us to better understand which improvements will truly be possible to develop. This includes listing the missing functionalities in integration tests that are available for E2E testing, analyzing the feasibility of their implementation, and prioritizing their development order.

Ecosystem Fitโ€‹

Having a comprehensive set of functionalities available for integration tests would bring numerous benefits to the entire community, including improved reliability, code quality and maturity, and faster feedback loops.

In the context of fuzzing detectors, integration tests are useful during their development in order to identify fuzzing parameters, and they are quicker than E2E tests. We learned this while working on fuzzing detection techniques during the Proof of Concept of Scout, which we performed in collaboration with researchers from the University of Buenos Aires. We believe that, having a complete set of functionalities for integration tests would be useful for other teams working in the development of fuzzing detectors for ink! smart contracts (e.g: Fuzzland team developing Ityfuzz, Klevoya team).

Team ๐Ÿ‘ฅโ€‹

Team membersโ€‹

  • Ariel Wassbein, Head of Research.
  • Diego Kelyacoubian, Head of Program Management.
  • Valeria Caracciolo, Business Development.
  • CoinFabrik's development and auditing team.


  • Registered Address: Dr. Emilio Ravignani 2394, C1425 CABA, Argentina.
  • Registered Legal Entity: Nektra S.A.

Team's experienceโ€‹

We are a research and development company specialized in Web3, with a strong background in cybersecurity. Founded in 2014, we have worked on over 200 blockchain-related projects, EVM based and also for Solana, Algorand, and Polkadot. Beyond development, we offer security audits through a dedicated in-house team of senior cybersecurity professionals, currently working on code in Substrate, Solidity, Clarity, Rust, and TEAL.

Our team has an academic background in computer science and mathematics, with work experience focused on cybersecurity and software development, including academic publications, patents turned into products, and conference presentations. Furthermore, we have an ongoing collaboration on knowledge transfer and open-source projects with the University of Buenos Aires.

As well, CoinFabrik has been providing Quality Assurance as a service to development teams, accumulating valuable expertise in the field for a considerable period of time. Our clients highly appreciate this service, and as a result, we are eager to expand our capabilities to the ink! ecosystem.

Team Code Reposโ€‹

Team LinkedIn Profiles (if available)โ€‹

Development Status ๐Ÿ“–โ€‹

We have already identified some improvements to be made in integration tests. We also identified how they are implemented in E2E tests and provide an initial level of complexity / feasibility associated to their development:

Issue NumberIssue NameDescriptionFeasibilityReferences
1Alice and Bob's addressesAlice and Bob's addresses should match across integration and E2E tests.FeasibleAccounts on Integration Tests: Link
Accounts on E2E tests: Addresses are used from this library Link
Account addresses are different for both Alice and Bob.
2Storage LimitationsThe storage in the integration environment should have the same limitations as in the blockchain environment (end-to-end).FeasibleSet_contract_storage is implemented differently in integration tests vs E2E tests.
In integration tests, set_contract_storage is implemented this way: Link
In E2E tests, the implementation calls the ext::set_storage function, this function is implemented in a way that checks maximum value lengths: Link1 Link2
The checking of maximum value lengths through the ext::set_storage function is not implemented in integration tests.
3Delegate callAbility to use delegate calls in integration tests.Complex, yet feasible.Not implemented in integration tests: Link
Notice in line 460: โ€œoff-chain environment does not support delegated contract invocation"
In E2E tests, it is implemented here: Link
4Cross contract callsAbility to perform contract-to-contract calls during integration tests.To be evaluated.Not implemented in integration tests: Link
Notice in line 446: โ€œoff-chain environment does not support contract invocation"
In E2E tests, it is implemented here: Link
5Gas usageGas usage in the integration environment should have the same limitations as in the blockchain environment (end-to-end).To be evaluated.Not implemented in integration tests: Link
Notice in line 405: โ€œoff-chain environment does not yet support gas_left"
In E2E tests, it is implemented here: Link

We validated the idea of the analysis and development described in this application with Sam Ruberti from the ink! Ecosystem and David Hawig from Web3 Foundation, who encouraged us to apply for this grant.

Development Roadmap ๐Ÿ”ฉโ€‹


  • Total Estimated Duration: 2 weeks
  • Full-Time Equivalent (FTE): 2.5 FTE (0.25 Project Manager, 0.25 Tech Lead, 1 Full time Sr. Rust Developer, 1 Full Time SemiSr. Rust Developer)
  • Total Costs: 13,500 USD

Milestone 1: Analysisโ€‹

  • Estimated duration: 2 weeks
  • FTE: 2.5
  • Costs: 13,500 USD
0b.DocumentationCreate a comprehensive report that compares the functionalities of integration tests and E2E (End-to-End) tests. The report's focus is to identify what can be accomplished in E2E tests but not currently in integration tests, as well as any inconsistencies. If applicable, we will provide suggestions that are not covered by either test type.
0c.Testing and Testing GuideNo tests will be produced at this stage.
0d.DockerDoes not apply at this stage.
0e.ArticleWe will prepare a summary report and publish it on our blog
1AnalyzeStudy and compare Integration and E2E (End-to-End) tests in ink!.
2EvaluateAssign a complexity level to each finding based on the difficulty of implementing the missing or enhanced functionality.
3EstimateIndicate an order of priority under which the missing functionalities shall be developed during the next milestone, where the functionalities are effectively implemented for integration tests.

Future Plansโ€‹

After finishing Milestone 1: Analysis, and having a good understanding of which missing functionalities in the integration testing environment can be developed, as well as an estimation of the effort required to develop them, we will submit a new grant proposal for a second milestone. The intention of this second milestone is to effectively implement these missing features. We detail in the table bellow its deliverables; its estimated duration is to be defined upon the delivery of the initial Analysis milestone.

0b.DocumentationWe will update our previous report. This includes the current status of identified use cases.
0c.Testing and Testing GuideWe will develop the missing functionalities identified, and submit a pull request to the corresponding repository. The newly developed functionalities will be documented and a testing guide will be included.
0d.DockerDoes not apply at this stage.
0e.ArticleWe will publish an updated report in our blog at
1DevelopBuild the necessary improvements and missing tests for the identified use cases outlined in the first milestone.
2Analyze and EstimateIf applicable, suggest additional tests for this or next milestones.

Moving forward, we have two projects in mind:

  1. Research and develop an advanced testing automation solution for ink! smart contracts.
  2. Improve our open source bug-detection tool Scout

Referral Program (optional) ๐Ÿ’ฐโ€‹

Additional Information โž•โ€‹

How did you hear about the Grants Program? Richard Casey from Parity brought this program to our attention, and we have already successfully delivered two applications as a result.

During our inquiries for this application, we briefly consulted Sam Ruberti from the ink! Team and David Hawig from the Web3 Foundation. Their encouragement motivated us to proceed with this presentation.