Welcome back to my blog about MBot, this is part 2 where we'll be going over the finishing parts of the backend and then the rest will be how the frontend works, the design decisions made and the next steps.

API Layer

With our API layer we had a bunch of choices for the framework. I had a choice of Gin, HttpRouter and Chi. After a bunch of looking through Go's subreddit and looking through Github repo's I decided to stay with the stdlib. Go's standard library is truly one of the richest I've seen of any programming language. Go ships a highly functional http library as standard and it becomes quite easy to setup your own API.

I tried to abstract how the API's were created whilst keeping it readable. From what I've read in dev blogs and subreddits, try not to go to fancy with Go - you'll find out Go's much better when you make the code simple and predictable.

Here's how a set of endpoints look in our codebase.

tradingHandler := http.StripPrefix("/api/trading", handler.NewTradingHandler(tradingController))

mux := http.NewServeMux()
mux.Handle("/api/trading/", tradingHandler)

server := &http.Server{
	Addr:    ":9090",
	Handler: CORS(mux.ServeHTTP),
}

if err := server.ListenAndServe(); err != nil {
	logger.Error(err)
}

We create a handler for a specific endpoint in our codebase, for our example I'm showing my trading handler. As you can see so far from the code, it's so easy to read through the code and understand what's happening.

server.ListenAndServe() becomes a blocking call for our backend.

What the handler's job is to do is to add the different endpoints to a path and functionality behind each endpoint.

func NewTradingHandler(controller *controller.StrategyController) http.Handler {
	mux := http.NewServeMux()
	handler := &TradingHandler{strategyController: controller}
	handler.registerRoutes(mux)
	return mux
}

func (th *TradingHandler) registerRoutes(mux *http.ServeMux) {
	mux.HandleFunc("POST /create", th.createTask)
	mux.HandleFunc("GET /task/{id}", th.getTaskById)
	...
}

So in our handler, we create a mux which I add some routes to. Each of the routes have our functionality we need through the controller. We pass this mux back to our "main" mux to then register it behind a path.

Here's how data flows through our app

Rendering diagram…

Websockets

Websockets in our backend are quite interesting. All of our websockets do follow near enough the same pattern but we have different use cases. As I went over how WS's worked in our internal layer, i'll be going how they work in our API layer. They are quite important here as it's our main way to update the frontend with real-time data such as pricing data.

Note: I've removed the error handling to make the code a bit quicker to skim through, if you'd like to see the actual code check out the code here

Here's our setup for our subscribe function, notice how we have quite a few defers that we'd like to do before the function returns, including closing channels, connections, waiting for waitgroups to finish etc.

func (th *TradingHandler) subscribe(w http.ResponseWriter, r *http.Request) {
	var wg sync.WaitGroup
	c, err := websocket.Accept(w, r, opts)

	defer func(c *websocket.Conn) {
		err := c.CloseNow()
	}(c)

	wsWriteChan := make(chan dto.StrategyResponse, 1000)
	defer close(wsWriteChan)

	defer wg.Wait()

	defer func() {
		for strategyId := range activeSubs {
			err := th.strategyController.Unsubscribe(strategyId)
		}
	}()

	defer close(subscribers)

	ctx, cancel := context.WithCancel(r.Context())
	defer cancel()

Here we set up our fan in loop. Subscribers is a channel of subscription channels, we will be going over each subscriber chan and pushing the messages straight to our websocket channel. After messages get pushed to our websocket channel we're able to write those messages back to the websocket connection.

	//fan in loop
	go th.fanIn(subscribers, wsWriteChan, &wg)
	//ws write loop
	go th.writeToWs(ctx, wsWriteChan, c)
}

Here's our websocket reading loop. We'd read messages coming in from the WS connection and route to the appropriate place. If we have a new subscription, we'll start having messages come out to our WS write channel that was seen above.

for {
	var msg dto.StrategySubscribe
	resp := dto.StrategyResponse{}

	err := wsjson.Read(ctx, c, &msg)

	switch msg.Type {
	case dto.Subscribe:
		sub, err := th.strategyController.Subscribe(msg.Id)
		wg.Add(1)
		subscribers <- *sub
		activeSubs[sub.Sub_id] = *sub

	case dto.Unsubscribe:
		err := th.strategyController.Unsubscribe(msg.Id)
		delete(activeSubs, msg.Id)
	}
}

I've used a fan in design here with our WebSocket's in our API layer. The idea behind a fan in design is taking in messages from multiple different channels and inputting into one channel. This one channel's messages will be what is being broadcast to the frontend.

I've had to be careful with panic's here. One of our biggest risks was writing to a closed channel, so I've had to make sure to use my defer's and WaitGroups correctly.

Overall MBot's backend's been built quite well in terms of developer experience. I found adding new programs quite easy to add and it's been a joy to work with Golang. If you'd like to explore the backend's code which I highly recommend -> [Github Link] (https://github.com/mbdev0/PersonalBot/tree/main/mbot/backend)

Frontend Architecture

Mbot's frontend started off as a webapp built in React & Typescript. The genuine reasoning behind it was I wanted to see how performative I can get React to be. There's a whole notion online that React is not performative - I wanted to prove that wrong with Mbot. I also found that if things do become too intensive for React, if I used a desktop application framework I could offload certain aspects of the bot off of the webapp. This is where Wails comes in. Wails allows us to be able to use OS API's and allows us to ship our bot as an application rather than a webapp. This works well when it comes to features like dropping files into a section of the app, spinning up terminals to view logs of each task, streaming pricing data etc off the webapp and in a terminal.

The frontend is quite simple in how it works, for a majority of it, it is just a CRUD frontend, e.g. for Tasks, Wallets, RPC's we have all the basic CRUD functionality. A lot of the complexity behind the app came from performance and handling edge cases that a user might face, e.g. If they wanted to cancel an AFK task after it was in progress of purchasing 10 coins at the same time. A lot of the design decisions we took were made to support performance of the app and how quick pricing data changes; especially in Memecoin trading.

Let's go over how the data from the backend flows through the frontend:

Rendering diagram…

The API dataflow is quite simple enough and is what you'd see from a standard web application. I'll be going over the reason for a WS Provider and how that works in our frontend.

Our querying layer was built with Tanstack Query - Unfortunately Tanstack Query didn't have quite a lot docs about how to implement Websockets with their framework but a maintainer had written a blog on how to do so here. This ended up driving how we do Websockets in our frotnend.

We'd have our separate hooks such as useTaskWebsocket, useStrategyWebsocket etc. hosted at the upper most layer of the application. You can see this from the mermaid diagram above. This was so we can force one connection per domain for the life of the app, not one connection per component. If we hadn't kept the websocket providers in App.tsx, we'd have so many connections to each domain, this would've probably hurt our throughput and our response times from our backend.

All our websockets in the frontend share the same sort of structure:

We have our Websocket Provider

export function WebsocketProvider({ children }: { children: ReactNode }) {
  const { sendPositionWsMessage, isPositionWsOpen } = usePositionWebsocket();
  const { send, taskWebsocketOpen } = useTaskWebsocket(sendPositionWsMessage);
  const { sendStrategyWSMessage, isStrategyWebsocketOpen } =
    useStrategyWebsocket(sendPositionWsMessage);

  const value = useMemo(() => {
    const websocketOpen = taskWebsocketOpen && isStrategyWebsocketOpen && isPositionWsOpen;
    return { send, sendStrategyWSMessage, sendPositionWsMessage, websocketOpen };
  }, [
    send,
    sendStrategyWSMessage,
    sendPositionWsMessage,
    taskWebsocketOpen,
    isStrategyWebsocketOpen,
    isPositionWsOpen,
  ]);

  return <WebSocketContext.Provider value={value}>{children}</WebSocketContext.Provider>;
}

With our provider we're able to wrap our App.tsx with the provider which will expose our functions that we are returning in value.

<WebsocketProvider>
	<MainApp/>
</WebsocketProvider>

Lets go over how usePositionWebsocket works which is very similar to how the other 2 websockets hooks work.

Here we're setting up all of our state, note how we're using the QueryClient that Tanstack query exposes for us.

export const usePositionWebsocket = () => {
  const client = useQueryClient();
  const websocket = useRef<WebSocket | undefined>(undefined);
  const subscribedTasks = useRef<Set<number>>(new Set<number>());
  const [websocketOpen, setWebsocketStatus] = useState(false);

The frontend then connects to the websocket hosted by the backend and stores a reference to it. We then have event handlers that will listen into the messages that get sent from the backend, from there the front end would be able to do whatever it needs with the message.

  useEffect(() => {
    const ws = new WebSocket(POSITION_WS);
    websocket.current = ws;

    ws.onopen = () => {
      console.log('connected');
      setWebsocketStatus(true);
    };

    ws.onmessage = (event) => {
      const parsed = JSON.parse(event.data);
      if (!parsed.position_msg) {
        console.error('Position WS error:', parsed.error);
        return;
      }
      const data: PositionWebsocketMessage = parsed;
      positionStore.setWsMessage(
        key(data.position_msg.strategy_id, data.position_msg.buy_task_id),
        prettifyWsMessage(data)
      );
    };

    ws.onclose = () => {
      console.log('disconnected');
      setWebsocketStatus(false);
    };

    return () => {
      ws.close();
    };
  }, [client]);

We also return a send message callback, this is so that we're able to send messages on the websocket without needing to reinstate a connection.

  const sendPositionWsMessage = useCallback((msg: SendPositionWSMessage) => {
    if (msg.type === 'Subscribe' && subscribedTasks.current.has(msg.id)) {
      return;
    }

    if (websocket.current?.readyState == WebSocket.OPEN) {
      websocket.current.send(JSON.stringify(msg));
      if (msg.type === 'Subscribe') {
        subscribedTasks.current.add(msg.id);
      } else {
        subscribedTasks.current.delete(msg.id);
      }
    } else {
      console.error('unable to send message to websocket');
    }
  }, []);

  return {
    sendPositionWsMessage,
    isPositionWsOpen: websocketOpen,
  };
};

For our data structures that update quite frequently, for example our message box for tasks: ![[Pasted image 20261006193222.png]]

We actually had to update the store directly so we don't have to wait for a whole re-render of the component. This way we can keep our updates quick for the user to get the information ASAP. You can find my inspo for this approach here: link, And the code for that in Mbot: link

Data Table Usage

We use table's in each part of MBot's tabs to display data. All of them are basic ShadCN tables, except our task table. Due to the nature of tasks needing consistent data updates, retain performance, needing sub-rows etc. MBot's task table is built with TanStack Table.

We have a few cool feature's we've added in the task table that i'd like to go over

Starting off with our sub tasks which are only possible thanks to Tanstack Table. ![[Pasted image 20261007000059.png]]

We have subtasks which relate to a strategy task as talked about in our previous blog post. For example, for our buy tasks, since we have the option to sell the token we bought. MBot displays a sell task as a sub task as it can be considered as a child of the buy task. This design decision was primarily made because of our AFK tasks. I didn't want a user to run an AFK task and not be able to view the coins that they bought. It also allows extra functionality of the child tasks, like stopping the task.

We had to setup the sub-rows by first organizing the schema of the data that's coming in. To make it easier to incorporate into the table, we had to make sure parent & child rows have the same schema

interface StrategyRow {
  id: number;
  program: string;
  type: TaskRowType.Strategy;
  state: string;
  ws_message: string;
  tx_message?: string;
  data: StrategyTask;
  subRows: TaskRow[];
}

export interface TaskRow {
  id: number;
  program: string;
  type: TaskRowType.Task;
  state: string;
  ws_message: string;
  tx_message?: string;
  data: Task;
  strategyId?: number;
  subRows?: TaskRow[];
}

Then we can fit them into our data table definition by setting the getSubRows property:

const table = useReactTable({
    data,
    columns,
    meta: { rowActions, settingsData },
    getCoreRowModel: getCoreRowModel(),
    getExpandedRowModel: getExpandedRowModel(),
    getSubRows: (row) => row.subRows,
  });