Historical crypto trade data contains the individual trades that were executed by an exchange over time. Each historical trade contains information about the trading pair, the trade size, the exact time the trade was executed, and the price of the assets being bought or sold.

Interestingly, this trade data is also used to construct OHLCV data, evaluate different market events, and develop analytical tools. Having accurate and reliable historical tick-by-tick trade data is vital for reconstructing what events took place in the market over time.

Throughout this article, we will provide several examples of how to use the Shrimpy Developer APIs to access the complete archive of historical trade data sourced from Kaiko. Before we get started, we should set up our Python environment by installing the Shrimpy Python Library. This will be the key library that will give us access to everything Shrimpy has to offer.

Trade Data

The exchange trade history is a way for traders to read the latest buy and sell trades that have been executed on an exchange.

Learn more

Access Historical Trade Data

To kick off the tutorial, let’s start with the most basic script we can write. This script will simply request the first 10,000 BTC/USD trades that took place after '2020-09-15T01:00:00.000Z' on Coinbase Pro.

import shrimpy

# sign up for https://developers.shrimpy.io/ to get your API key
shrimpy_public_key = '...'
shrimpy_secret_key = '...'


# create the Shrimpy client
client = shrimpy.ShrimpyApiClient(shrimpy_public_key, shrimpy_secret_key)

trades = client.get_historical_trades(
    'coinbasepro',
    'BTC',
    'USD',
    '2020-09-15T01:00:00.000Z',
    '2020-09-16T01:00:00.000Z',
    10000
)

# write the data to a text file
with open('trades.txt', 'w') as file:
     file.write(json.dumps(trades)) # use `json.loads` to do the reverse

Note: Some exchanges process a lot of trades. Make sure to monitor your data credit usage while accessing trade data since the number of data credits required over any given time period will vary.

Using Trades To Calculate Slippage

Just for fun, let’s go through an example script that will collect trade data, organize each trade by timestamp, and calculate the slippage experienced on the exchange during each 1 second time period.

import shrimpy
import csv

# sign up for https://developers.shrimpy.io/ to get your API key
shrimpy_public_key = '...'
shrimpy_secret_key = '...'


# create the Shrimpy client
client = shrimpy.ShrimpyApiClient(shrimpy_public_key, shrimpy_secret_key)

trades_dict = {}

trades = client.get_historical_trades(
    'coinbasepro',
    'BTC',
    'USD',
    '2020-09-15T01:00:00.000Z',
    '2020-09-16T01:00:00.000Z',
    10000
)

# group trades that happened at the same time
for trade in trades:
    if trade['time'] in trades_dict:
        trades_dict[trade['time']].append(trade)
    else:
        trades_dict[trade['time']] = []
        trades_dict[trade['time']].append(trade)

# calculate the slippage for trades that happened at the same time
# at the end, we save all the data to a csv file
for key, value in trades_dict.items():
    slippage = (float(value[len(value)-1]['price']) - float(value[0]['price'])) / float(value[0]['price'])
    sum = 0
    for trade in value:
        sum += float(trade['size'])
    row = [abs(slippage * 100), key, sum]
    with open('24_hour_%s_slippage_%s_%s.csv' % (exchange, quote, base), 'a') as csvFile:
        writer = csv.writer(csvFile)
        writer.writerow(row)

Note: These are not production-grade scripts. They are simply fun examples to provide information about what is possible with the data. There are infinite possibilities, so experiment with different ways to use the data!

Slippage picture.png — What is Slippage?

Slippage takes place when a single order or multiple sequential orders are placed with an exchange that consume consecutive levels of open orders on the exchange.

Learn about slippage

Data Availability

Before we start collecting data on a trading pair, there are ways to get an understanding of how much data is available for each trading pair. This can be accessed by calling the endpoint for getting historical instruments.

Retrieving the list of historical instruments for an individual exchange or for all exchanges can simply be done by using the following requests in Python.

instruments = client.get_historical_instruments() # get historical instruments for all exchanges
bittrex_instruments = client.get_historical_instruments('Bittrex') # get historical instruments for Bittrex

The result of these requests would look something like the following:

[
    {
        "exchange":"bittrex",
        "baseTradingSymbol":"LTC",
        "quoteTradingSymbol":"BTC",
        "orderBookStartTime":"2016-09-14T13:00:00.000Z",
        "orderBookEndTime":"2019-09-07T23:00:00.000Z",
        "tradeStartTime":"2016-09-12T23:00:00.000Z",
        "tradeEndTime":"2019-09-09T16:00:00.000Z"
    }
    ...
    {
        "exchange":"kucoin",
        "baseTradingSymbol":"LTC",
        "quoteTradingSymbol":"BTC",
        "orderBookStartTime":"2019-04-09T11:00:00.000Z",
        "orderBookEndTime":"2019-08-31T23:00:00.000Z",
        "tradeStartTime":"2019-04-09T10:00:00.000Z",
        "tradeEndTime":"2019-09-03T23:00:00.000Z"
    }
]

That’s not all!

If we need even more specific information about how many trade data points are available over a time period, we can use the endpoint for getting the historical count.

The following example shows how this can be done in Python.

trades = client.get_historical_count(
    'trade',
    'binance',
    'BTC',
    'USDT',
    '2019-05-01T00:00:00.000Z',
    '2019-05-02T00:00:00.000Z'
)

The response to this request will contain the count of trades that are available in that time range for the specified trading pair. This would look like the following:

{
  "count": 165012
}

Conclusions

It’s time to get building!

There are no limits to what you can build with historical trade data. Whether you want to construct machine learning models that can predict the future price of an asset or simply want to analyze an interesting black swan event, trade data is a great place to start when analyzing historical market data.

We can’t wait to hear what you’re building! Leave a comment sharing the tools you are building with the Shrimpy Developer APIs!

Additional Good Reads

How to Make a Crypto Trading Bot Using Python

How to Download Cryptocurrency Candlestick Data from Exchanges

Arbitrage Scripts for Crypto Trading Bots

Script for Bitcoin Price Live Ticker (Using Websockets)

About Shrimpy

Shrimpy’s Developer Trading API is a unified way to integrate trading functionality across every major exchange. Collect market data, access real-time websockets, execute advanced trading strategies, and manage an unlimited number of users.

Shrimpy Crypto Trading API: Shrimpy | Crypto Trading APIs for Developers

Don’t forget to follow us on Twitter and Facebook for updates, and ask any questions to our amazing Telegram community.

The Shrimpy Team

Historical Crypto Trade Data [Example Python Scripts]