The Best APIs for Cryptocurrency Historical Data
Reliable historical cryptocurrency market data is hard to find. While there are a handful of providers which had the foresight to begin collecting data from exchanges during the early days of cryptocurrency, most people still haven’t realized the importance of cryptocurrency market data.
As institutions begin to roll into the market, there is going to be a BIG problem. That problem is the utter lack of accessible historical data. In fact, there are periods of time for which no data whatsoever might be found. These data blind spots are concerning. It means events took place on exchanges which we will never be able to recall or study.
Not to worry! The future is brighter.
Several teams have taken it upon themselves to secure the future of data in the crypto market. They understand the importance of data completeness and are going to great lengths to ensure they make it happen.
One of these teams is Shrimpy.
Shrimpy is a simple API which provides a normalized set of endpoints across every major exchange. That way developers, traders, and institutions can access complete historical data going back as far as 2011.
Order Book Snapshots
Order books are one of the fundamental aspects of cryptocurrency exchanges. They are the active set of orders which are live on the exchange and represent user intent to buy or sell an asset at the specified price.
Order book snapshots are important because they provide the state of the market on a given exchange at the exact moment in time.
Access to order book snapshots provides a way for developers, institutions, and traders to backtest strategies, analyze the market, and find ways to improve their trading algorithms.
Let’s take a specific example. Imagine you want to study how using a Simple Moving Average Crossover strategy with the BTC-USD trading pair would have performed on Kraken for the last 4 years.
Without knowing the exact state of the order book at each crossover event, we would need to guess what prices we could have bought or sold each asset. This injects a level of uncertainty into mathematical models which attempt to accurately simulate trading on historical data.
Instead, we can now collect a complete history of order book snapshots for the BTC-USD trading pair on Kraken. Evaluating the actual order books, we can simulate the exact trades which would have been available at the crossover events.
How We Collect Order Book Snapshots
Unlike many other data services available in the cryptocurrency market, Shrimpy has strict requirements when it comes to the way we manage exchange order books.
Since Shrimpy originated as a trading engine which powers tens of thousands of traders, developers, and institutions, our infrastructure needs to be ready to execute trades in real-time.
That means instead of pulling for data from the exchange every time we make a trade, we need to have a local copy of the exchanges order books ready on our servers.
This is done through exchange order book WebSockets. As the order book for each asset changes, exchanges will send updates through the WebSockets to inform subscribers what has changed. Therefore allowing Shrimpy to update our local copy of the complete order book in real-time.
Once the infrastructure for real-time order book updates was in place, the ability to take snapshots of the order books on a regular interval came with ease.
On a 1 minute interval, Shrimpy takes a snapshot of every order book for every asset across each exchange we support. This snapshot is then stored in our database for later retrieval through the historical data APIs.
How You Can Access Historical Order Book Snapshots
After storing the order book snapshot, Shrimpy is now ready to serve the snapshot as a historical set of data. To access this data, you can call our REST API endpoints to quickly reconstruct the history of any order book.
The following example provides a demonstration of how you can call the endpoint and the results which are returned.
Request
GET https://dev-api.shrimpy.io/v1/historical/orderbooks
Response
[
{
"time": "2019-05-19T00:03:02.000Z",
"asks": [
{
"price": 564.002,
"size": 2354346
},
...
],
"bids": [
{
"price": 564.002,
"size": 2354346
},
...
]
},
...
]Tick-by-tick Trade Data
Tick-by-tick trade data is the detailed trades which were executed on the exchange. They provide the complete set of all trades which are being made on an exchange at any given time.
The trade data includes information about what time the order was executed, the size of the order, the price, and which side took the order.
Similar to the order books, our team required a robust strategy for maintaining the complete history of orders on any particular exchange.
In order to ensure we received trades in real-time, we implemented a websocket solution where we subscribe to every trading pair on each exchange to receive the complete updates from each trading pair.
Once we receive the trade, it is immediately placed into our database. In addition to storing each individual trade, we also construct candlestick (OHLCV) charting data based on these individual trades.
The Shrimpy developer APIs provide a normalized experience when handling historical data, so constructing each candlestick from scratch allows us to have a fine control over how each data point is processed.
How To Collect Tick-by-Tick Trade Data
Once a trade has been stored, it becomes ready to serve. Developers can then access this data through REST API calls to analyze, evaluate, or study how trades have been executed across exchanges.
These examples highlight how you can access trade data conveniently through our historical data endpoints.
Request
GET https://dev-api.shrimpy.io/v1/historical/trades
Response
[
{
"time": "2016-09-06T13:01:34.000Z",
"size": "1891.1316431",
"price": "0.00002585",
"takerSide": "buyer"
},
...
]Available Instruments
The Shrimpy developer APIs provide access to 16 different exchanges and thousands of markets. Having access to all this data can feel overwhelming if you don’t know where to start or what data periods are available.
Fortunately, we provide a simple endpoint where you can access all the data which is available across each exchange. That way you never again need to question what data you can access.
How to Get Historical Instruments Available
Simple API calls provide a complete list of data availability. This includes both historical order book snapshots and trade data points.
Use the following call to find what data we have available.
Request
GET https://dev-api.shrimpy.io/v1/historical/instruments?exchange=Bittrex
Response
[
{
"exchange":"bittrex",
"baseTradingSymbol":"LTC",
"quoteTradingSymbol":"BTC",
"orderBookStartTime":"2016-09-14T13:00:00.000Z",
"orderBookEndTime":"2019-09-07T23:00:00.000Z",
"tradeStartTime":"2016-09-12T23:00:00.000Z",
"tradeEndTime":"2019-09-09T16:00:00.000Z"
}
...
]Additional Reading
Crypto API for Smart Order Routing
Universal Websockets for Real-Time Crypto Exchange Order Book Data
Free Historical OHLCV Trade Charting Data for Crypto Exchanges
Kaiko Partners with Shrimpy - Historical Cryptocurrency Market Data
About Shrimpy
Shrimpy’s Developer Trading API is a unified way to integrating trading functionality across every major exchange. Collect historical market data, access real-time websockets, execute advanced trading strategies, and manage an unlimited number of users.
Shrimpy Crypto Trading API: Shrimpy | Crypto Trading APIs for Developers
Don’t forget to follow us on Twitter and Facebook for updates, and ask any questions to our amazing, active community on Telegram.
The Shrimpy Team