Keep Your Private Keys Close and Keep Your Smart Contracts Closer — Introducing The Smart Contract Watch
Monitoring your servers, services or micro-services usually is considered a priority, in order to see if everything is running as it should and additionally receive instant alerts in case of any suspicious or unplanned activity.
Blockchain. As Backend?
Usually, when running a regular backend service, the idea of monitoring your servers is straightforward with many options to choose from. In the blockchain (Distributed Ledger), things run a bit differently: the whole blockchain is your backend and there are no dedicated servers you can monitor in order to check your smart contract’s activity. The only way to check how your smart contracts are interacting in the blockchain is to scan the blockchain block by block (as they come) for any transactions or events related to your smart contract.
Blockchain Explorers
This manual process of block scanning is made easier by blockchain explorers like Etherscan. They explore the whole blockchain and most of the information needed can be found using them as a platform. Blockchain explorers are a good start if you want to do quick infrequent checks but their shortcomings start to show if any serious smart contract monitoring effort is considered, for example:
- You have no control on what is scanned or what should be scanned, in terms of parameters or activity, as you only assume that these explorers scan and log everything.
- The explorers are not running locally so you are at risk if they fail, and thus have no control or guarantee that these services will keep running.
- These explorers try to tackle the problem by monitoring and logging the whole blockchain and provide it as a service to many users. This creates constraints in terms of how much data can you extract. For example, Etherscan limits your requests to only 10,000 transactions per
GETrequest when using their API. - They are not designed by nature to be a monitoring tool, some of these explorers lack good Transaction / Log decoding features.
- You will still require some programming effort in order to integrate these APIs into your own logging tools, e.g. Graylog
- They are mostly closed source so the idea of cloning these projects and tweaking them to your specific needs is out of the question.
The Smart Contract Watch
Due to these shortcomings, we decided at Neufund to write a tool. Smart Contract Watch specifically solves these problems. It has become the main tool that monitors our smart contracts activity.
We wanted a tool that can:
- Be fully controlled in terms of what information to extract from the blockchain and which blocks to scan. For example, specific period of time in the past or continuously checking new blocks as they come.
- Run in a fully controllable, consistent and reliable local environment.
- Decode all transactions and logs generated from any smart contract we decide to monitor.
- Have the option to run on different chains regardless of whether it is Mainnet, Testnet, or our own private Ethereum chain.
- Be open source for us and the community to use, test and contribute.
- Have modular output modules. For example, graylog or any database.
Smart Contract Watch is designed to either run as a command line tool for quick inquiries on your local system, or as a dockerized service that can run on your server for heavy long-term monitoring.
In our case, we use Smart Contract Watch as a dockerized service in order to monitor our ESOP as well as some ICO’s added in our ICO Monitor. It also continuously logs all generated activity to our Graylog system. Also, we are planing to monitor our ICBM when it starts soon.
Other use cases
Other interesting use cases for the Smart Contract Watch is to:
- Integrate it with your deployment and transaction signing processes. For example, By quickly running the Smart Contract Watch in “CLI” mode and typing in your smart contract address you can immediately check if for example a transaction was mined or if a transaction was successful
- Send decoded output to the console to quickly
grepfor method calls of interest. We use it a lot to verify contracts activity in ICO Transparency Monitor - Send transaction data to a transnational storage like a relational database. You can store all transactions and events and quickly do stats on them or to access historical data from your d-app
- Watch for a particular event log and act on it — for example we can detect if there is new revenue to be claimed in the token and we can alert all investors that it is the case
How to use
In order to run this tool you will need to connect to an Ethereum node like Parity or Geth and have “Node js” installed. Use “git clone” to download the project repo into a local file and run yarn start. The tool can be used either as a:
- command line interface
- config file “.watch” that should be placed in the same directory as the tool.
Parameters
The smart contract watch uses:
-a Array of addresses to monitor
-f Starting block number
-t Ending block number (keep empty for continued checking)
-q Quick mode (scans the blockchain with less RPC calls)
-s Save state mode (saves the last block scanned when service restarts again)
Environmental variables
Additionally there are environmental variables that must be taken care of:
LOG_LEVEL Sets the logging level for the app
SOCKET_IO_PORT Socket port
RPC_URL Link to your RPC node
OUTPUT_TYPE Terminal or Graylog
Access_Token Access token for Etherscan
Under the hood
The Smart Contract Watch uses web3.js to issue JSON-RPC requests to a running node. You control which chain the tools runs on depending on the connecting node.
There are two main operation modes:
- Default mode: the tool scans the blockchain block by block, transaction by transaction, and log by log for any activity related to the monitored smart contracts. This is a slow process and heavy on the node due to the high number of
eth_getTransactionByHashandeth_getTransactionReceiptRPC requests. However, this approach gives us the opportunity to add more features in the future like instrumenting the EVM or debugging transactions. - Fast mode: the tool gets all needed information from a whole block by only sending two RPC calls
eth_getBlockByHash,get_logs. This proves to be more efficient and faster.
Costume RPC calls
Because Web3.js 0.19 doesn’t provide get_logs RPC call as a native method in the library, we had to dive deep into realms of web3.js code and write our own costume RPC requester.
import Method from 'web3/lib/web3/method';
import web3Instance from './web3Provider';const initCustomRPCs = () => {
const customRpcCall = new Method({
name: 'getLogs',
call: 'eth_getLogs',
params: 1,
inputFormatter: [filter => filter],
outputFormatter: data => data,
});
customRpcCall.setRequestManager(web3Instance._requestManager);
customRpcCall.attachToObject(customRpcCall);
return customRpcCall;
};export default initCustomRPCs;
We ended up with this small piece of code to handle get_logs.
In reality this piece of code can be copied to any project that uses web3.js and accommodate any custom RPC call your node supports while taking advantage of web3.js RPC handling features. Just change the call from eth_getLogs into the JSON RPC call you need. Also, change the name field into whatever you want the returned method to be named. The only thing to keep in mind when using this approach is to format your input into the correct format before sending a request. That way you avoid unnecessary errors from the node.
In reality this piece of code can be copied to any project that uses web3.js and accommodate any costume RPC call your node supports while taking advantage of web3.js RPC handling features, just change the call from eth_getLogs into the JSON RPC call you need. Also , change the name field into what ever you want the returned method to be named. The only point to note when using this approach is to format your input into the correct format before sending a request, in order to avoid unneeded errors from the node.
Decoding transactions
Decoding transactions was an important feature we had to add, without it there was no way of knowing what was called or what is going on. abi-decoder (Initially uses web3.js for decoding) was a good start but later we realized that it didn’t fit all our needs. Constructor transaction decoding was missing and for a good reason. abi-decoder stores all smart-contract ABIs in one const instance this is not a good idea if decoding constructors is what you want. constructors in smart contract ABIs are stored without a name, only its type: constructor reveals its true purpose. In order to tackle this we had to refactor the code,add a new method decodeConstructor along the way and add the ability of creating a separate instance one for each smart contract ABI.
Every time you have a new smart contract ABI you can simply create a new instance using
import Decoder from './decoder';
const testContract = new Decoder( testContractAbi.json);Etherscan
Smart-Contract-Watch uses Etherscan’s API to import the required smart contracts ABIs for successful transaction/log decoding. This is a helpful option when we want to monitor smart contracts we have no access to their ABI.
The second option is to just paste your own smart contract ABI in the ./contracts in the project folder, this is helpful if you’re using a private chain and your smart contracts are not uploaded yet in Etherscan.
Files should be manually stored in the this format:
0x<address of smart contract>.jsonThe priority is always for the local folder, so you should make sure when you manually insert files to insert the correct abi and its address.
Output
We designed our output component to be modular where you can add you own output modules depending on your needs. Currently Smart Contract Watch supports two output modes
- Terminal output: regular output to the terminal
- Graylog output: which is more interesting for logging and in depth analysis
You can choose which options you want through the ENV variables. There is more room for additional output modules, you just need to add your own output model in the ./src/output folder and link it here:
import logger from '../logger';
import grayLogFromat from './graylogFormat';
import terminalFormat from './terminalFormat';export default (data, type = 'terminal') => {
switch (type) {
case 'terminal':
logger.log('info', terminalFormat(data));
break;
case 'graylog':
logger.log('info', JSON.stringify(grayLogFromat(data.transaction,
data.decodedInputDataResult, data.decodedLogs)));
break;
default:
throw new Error(`${type} output module is undefind`);
}
};
in the switch case.
Community
Smart Contract Watch is released by Neufund as an open source tool, issues feature requests and pull requests are encouraged in order for us to a create a tool that benefits everyone and pushes the community forward.
For technical details of how this tool works and how to run it please refer to Smart Contract Watch on GitHub
Conclusion
With the Smart Contract Watch, we now have a good grip on what is going on with our smart contracts and how are they interacting generally on the blockchain. We can check individual transaction and know exactly what is happening down to the function call and it’s parameters. This is done in preparation for our ICBM starting this autumn.
Moe Elsharif is the Embedded Developer at Neufund, a community-owned fundraising platform bridging the worlds of equity investments and blockchain.
To learn more, check out Neufund’s ICBM page or the whitepaper for a deep dive. Questions? Ask us anything on Twitter and Telegram!
