Okxlivro - Post Mortem

Hello! Today I’m sharing my thoughts on writing convenient Rust libraries. This is the first post in the “Post Mortem” series, where I’ll be dissecting different projects.

Today on our table is a technical assignment I was given at one company after an interview. I don’t like wasting my personal time, so I usually refuse to go through this stage, but this project seemed interesting to me, so I completed it. I also received a rejection and feedback. Let’s try to fix what didn’t please the reviewer and discuss the changes along the way.

As with the previous one, I’ll start this post with a brief clarification of terminology. If you’re confident in your knowledge of exchange trading specifics, feel free to skip to the core.

Portfolios, order books, and orders

If you’re not familiar with trading, I recommend studying this short video or this Wikipedia article before continuing reading. Below I’m presenting my understanding of the topic, which may be inaccurate or incomplete.

So, in Trading we have:

• Portfolio of orders
  • Sell orders (Ask side Order)
    • Trading pair identifier (what we’re exchanging for what), volume, and price
  • Buy orders (Bid side Order)

This results in an order book or book where we record all orders, and if they intersect (we have someone willing to sell one egg for a kilogram of potatoes, and another willing to buy one egg for a kilogram of potatoes), we can execute both orders.

Core

We’re trying to apply the Module Visibility-Based Structure. The idea and main goal is to write a library for working with the OKX exchange, the API should be convenient to use, and the implementation should be extensible and maintainable.

I decided it’s worth trying to write idiomatic but modern Rust code - using an asynchronous model, strict typing, and implementation hiding.

Here everything is fine, no issues were identified.

REST (in peace =)

Initially I made a fairly simple implementation:

pub async fn orderbook_snapshot(instrument_id: &str) -> Result<Orderbook, RestError> {
    let response: serde_json::Value = Client::new()
        .get(format!("{MARKET_BOOKS_REST_URL}?instId={instrument_id}&sz=5"))
        .send()
        .await?
        .json()
        .await?;
    let asks = response["data"][0]["asks"]
        .as_array()
        .unwrap_or(&vec![])
        .iter()
        .map(Order::try_from)
        .try_collect()?;
    let bids = response["data"][0]["bids"]
        .as_array()
        .unwrap_or(&vec![])
        .iter()
        .map(Order::try_from)
        .try_collect()?;
    let mut orderbook = Orderbook::default();
    orderbook.apply_snapshot(SnapshotData((asks, bids)));
    Ok(orderbook)
}

And received a remark - that this is a brute force solution and data parsing goes simply through Value. Let’s rewrite this with strict types:

pub async fn orderbook_snapshot(instrument_id: &str, client: &Client) -> Result<Orderbook, RestError> {
    let response: inner::ApiResponse = client
        .get(format!("{MARKET_BOOKS_REST_URL}?instId={instrument_id}&sz=5"))
        .send()
        .await?
        .json()
        .await?;
    let [data] = response.data;
    let asks = data.asks.into_iter().map(Ask::from).collect();
    let bids = data.bids.into_iter().map(Bid::from).collect();
    let mut orderbook = Orderbook::default();
    orderbook.apply_snapshot(SnapshotData((asks, bids)));
    Ok(orderbook)
}

Details can be viewed in the git repository, but the implementation itself is almost standard.

The difference is that I don’t do direct deserialization into public library types, but use mod inner {...}. This is done out of experience-based concerns that:

1. The exchange has an unstable API, despite versioning, documentation description differs from reality, especially when there are no formal schemas;
2. The exchange representation is not optimal, for example, an array of data is returned, but the element is always one, or the actual bids and asks are an array of four elements, one of which: "0" is part of a deprecated feature and it is always "0".

Asks & Bids

Let’s start with the simple - we use all standard library capabilities. There’s a requirement for order lists, they should be sorted from best to worst.

But depending on the order side (buying/selling), we need to sort either by ascending (better to buy cheaper) or descending (better to sell more expensive) price.

Initially I implemented them such that the price difference was in using the Reverse wrapper:

/// Low to high price ordered Map ([`Price`], [`Amount`]) of Asks.
pub struct Asks(BTreeMap<Price, Amount>);
// High to low price ordered Map ([`Price`], [`Amount`]) of Bids.
pub struct Bids(BTreeMap<Reverse<Price>, Amount>);

The remark indicated that extending and maintaining would be inconvenient due to logic duplication when there’s no difference (only Reverse) between bids and asks.

I think the reviewer wanted something else, but I decided to go all in and sometimes duplicate logic, but further separate bids and asks:

pub struct AskPrice(Price);
pub struct BidPrice(Reverse<Price>);

Let’s make different prices, as well as structures for single bids and asks that we use in the realtime module:

// lib.rs
pub struct Ask {
    pub price: AskPrice,
    pub amount: Amount,
}
pub struct Bid {
    pub price: BidPrice,
    pub amount: Amount,
}

// realtime.rs
pub struct SnapshotData(pub (Vec<Ask>, Vec<Bid>));
pub struct UpdateData(pub (Vec<Ask>, Vec<Bid>));

I’m countering the reviewer’s remark about “logic duplication” by saying that general logic needs to be implemented at the Price level, and logic differences can be implemented separately for Bid and Ask. And the presence of strict typing at the library API level provides good user experience, since it won’t be possible to accidentally confuse bids and asks.

Stream

Initially I used the approach through message sending channels, but then rewrote it to a version more ergonomic for library clients to use: changes

pub fn orderbook_updates(instrument_id: &str) -> Pin<Box<impl Stream<Item = Result<RealtimeData, RealtimeError>>>> {
  ...
  Box::pin(stream! {
    ...
    while let Some(msg) = read.next().await {
      ...
      yield Ok(data);
    }
  )
}

As we can see, this approach still requires additional macros, and async iterators are not native to the standard library. We continue to hope and wait.

The remark received here refers to the problem of connection management:

pub fn orderbook_updates(instrument_id: &str) -> Pin<Box<impl Stream<Item = Result<RealtimeData, RealtimeError>>>> {
    ..
    Box::pin(stream! {
        let (ws_stream, _) = tokio_tungstenite::connect_async(WS_URL).await?;
        let (mut write, mut read) = ws_stream.split();
        write
            .send(Message::Text(message_string.into()))
            .await?;
        while let Some(msg) = read.next().await {

It was necessary to extract this logic from the method, which is very logical. I love the pattern where all “resources” are passed as method arguments in minimal necessary forms. This of course forced me to struggle a bit with generics. But after ten minutes of jumping through the tungstenite code, I found the correct error types and brought the method signature to the following form, while splitting one method into two:

pub async fn subscribe<S>(channels_with_criterias: &[ChannelWithCriteria], write: &mut S) -> Result<(), RealtimeError>
...
pub fn transform_stream<T>(read: &mut T) -> Pin<Box<impl Stream<Item = Item>>>

I also changed the return type to support extending to all possible data types for different subscriptions:

pub enum RealtimeData {
    Books(BooksChannelData),
}

I can say that I find Rust enums convenient and practical. The only thing I’m really missing is that enum variants don’t have “first-class” support in the language’s type system.

pub enum BooksChannelDelta {
    Snapshot(SnapshotData),
    Update(UpdateData),
}
pub struct SnapshotData(pub (Vec<Ask>, Vec<Bid>));
pub struct UpdateData(pub (Vec<Ask>, Vec<Bid>));

For example, implementation of messages from the exchange - through a WebSocket connection to us can be delivered information both about portfolio updates and its snapshot state.

If we don’t use an enum, then the data itself are identical and the user can confuse them. In the implementation, I not only split them into variants, but also created new wrapper types for greater safety when using.

Error

I love the simple but incredibly useful derive_more library with procedural macros for all occasions, for example for creating error types:

#[derive(Debug, Display, Error, From)]
pub enum CompoundError {
    Realtime { source: RealtimeError },
    Rest { source: RestError },
}

Here, thanks to the macros, the enum becomes a Rust error, can be displayed as a string and converted from low-level errors - all this without extra hand-written code. It exists - but is generated automatically.

Additional Materials

If you’re interested in “playing around” with the example from this article, here’s the full project code.

I’ll also be grateful for feedback, please write your suggestions for improving the project in issues on GitLab.