Recipe: List Tokens I Own

What do I Own?

A TrueBlocks recipe to show ERC 20 token balances for an address.

Note: The following assumes you have a copy of (either by having built it yourself or downloaded it) the TrueBlocks Appearance Index.

Preliminaries#

We start by extracting from the chain all the transactions for the address in question. We will use a randomly selected address that we know owns a number of different tokens.

chifra list 0x03fdcadc09559262f40f5ea61c720278264eb1da

This produces a list of 2,129 appearances (at the time of this writing). Each appearances is a pair of integers detailing the blockNumber and transactionIndex of the transaction. Note that other methods of getting such a list (such as EtherScan APIs) return far fewer transactions. We will explain this in a later post.

Next, we want to export (to the screen) the transactional details of these appearances. We do this in csv format.

chifra export --fmt csv 0x03fdcadc09559262f40f5ea61c720278264eb1da

The above command generates the same 2,129 records, but this time it’s showing the details of the transactions.

Note: The list command is called automatically when you run the export command. You don’t have to do it separately. We are showing it here only to make the distinction.

The --fmt option may be used by any chifra command. It instructs the tools to export data in one of three formats: txt, csv or json. We’ll intersperse this option below.

Let’s see what happens if we export event (or log) detail instead of transactional detail of these 2,129 transactions.

Focusing on Events / Logs#

We can display the events for a given address using the --logs option:

chifra export --fmt csv --logs 0x03fdcadc09559262f40f5ea61c720278264eb1da

This takes a bit longer to finish, and it generates 8,100 records.

Question: Why is there so many more events than transactions?

Answer: Because many transactions generate more than a single event.

Question: Is it possible to speed things up?

Answer: Yes. You may speed up the export tool by adding the --cache_txs option. This will put the node data into the TrueBlocks cache. This speeds up operation of TrueBlocks by 20-30x over querying the node directly, but takes up additional hard drive space.

Articulating the Result#

Before we move on, let’s talk about articulation.

We’ll add an option to our query called --articulate. The --articulate option turns “ugly blockchain data into human-readable text”. (Try the command above both with and without --articulate to see the difference.)

chifra export --fmt csv --logs --articulate 0x03fdcadc09559262f40f5ea61c720278264eb1da

Filtering the Results#

The above command generates 8,100 log entries. This includes every event of any type that our address had anything to do with. That is, this includes any event generated by any smart contract in which our address appears even if our address didn’t generate that event.

We’re going to briefly switch the commands to json output in order to see what I mean. JSON format shows all the data, whereas txt and csv show only selected (and customizable) fields.

The next command extracts the addresses of the smart contracts that emitted any of those 8,100 events:

chifra export --logs --articulate --fmt json 0x03fdcadc09559262f40f5ea61c720278264eb1da | grep -i address

You can see there are various addresses here. It shows all smart contracts that emitted any event in which our address appears. ‘Appears’ means the address either generated (emitted) the event or the address shows up in one of the fields of an event emitted by some other contract.

We want to limit the result to include only those events emitted by our smart contract.

We do this by including the --emitter option.

chifra export --logs --articulate --emitter --fmt json 0x03fdcadc09559262f40f5ea61c720278264eb1da | grep -i address

And now, we see only our address as the emitter addresses. If we remove the grep and return to using csv:

chifra export --logs --articulate --emitter --fmt csv 0x03fdcadc09559262f40f5ea61c720278264eb1da

we now only see 3,317 events (again, at the time of this writing).

The Unix Philosophy#

Following the linux philosphy of stringing together a data pipeline…

First, let’s look at the first line of the data in order to see the names of the data fields:

chifra export --logs --articulate --fmt csv 0x03fdcadc09559262f40f5ea61c720278264eb1da | head -1

This produces

"blocknumber","transactionindex","logindex","address","topic0","topic1","topic2","topic3","data","type","compressedlog"

We are interested in the blockNumber, transactionIndex, and compressLog fields, so we use the Linux command cut to extract these columns:

chifra export --logs --articulate --fmt csv 0x03fdcadc09559262f40f5ea61c720278264eb1da | cut -d, -f1,2,11-100

And now we’re starting to get somewhere. You can see we’ve extract only events for our contract and we start to see the usefulness of articulation.

Before we proceed, notice that we could build a histogram of the generated events if we wished:

chifra export --logs --articulate --fmt csv 0x03fdcadc09559262f40f5ea61c720278264eb1da | tr '"' ' ' | cut -d',' -f11-100 | cut -d'(' -f1 | sort | uniq -c | sort -n -r

which produces the following table

How Many Transfers?#

And finally, we’ve arrived where we’ve been headed.

A list of all Transfers events emitted by a given ERC20 address ON THE COMMAND LINE AGAINST OUR OWN LOCALLY RUNNING NODE:

chifra export --logs --articulate --fmt csv 0x03fdcadc09559262f40f5ea61c720278264eb1da | tr '"' ' ' | cut -d',' -f1,2,11-100 | grep Transfer

How many of them are there?

chifra export --logs --articulate --fmt csv 0x03fdcadc09559262f40f5ea61c720278264eb1da | tr '"' ' ' | cut -d',' -f1,2,11-100 | grep Transfer | wc

You figure it out!

A Note on Using the API#

You can do the same exact analysis with the chifra serve API.

First, start the API in a separate terminal window by running

chifra serve

The server serves its API on port :8080 by default.

Return to the first window to run the following curl command.

Additionally, understand that the TrueBlocks API mirrors exactly the command line options for each tool. List all the available tools with chifra --help. And for any individual tool, list its options with <tool name> --help. Call one of the tools through the API by attaching the tool name followed by any options in regular URL format.

For example, the above command-line query could be re-written as a curl command thus

curl "http://localhost:8080/export?addrs=0x03fdcadc09559262f40f5ea61c720278264eb1da&logs&articulate&fmt=csv&maxRecords=10000" \
 | tr '"' ' ' | cut -d',' -f1,2,11-100 | grep Transfer | wc

Notice (1) prepend addresses and other positional parameters with the item’s name (in this case, addrs) and (2) attach the maxRecords=10000 option, as the API only returns 250 records by default.

The API produces json data by default, so you must include the fmt=csv option to your curl command if you wish to pipe the data through a command line tool such as cut, head, sed or grep (unless you use the JSON parser jq which is highly recommended for JSON data wrangling).

Have at it and Cheerio. Here’s hoping only happy data comes your way.

Edit this page on GitHub