Why build MBot?

Building a bot was always sort of a dream of mine. I started reselling in 2016, and from the get-go there was a thrill to making money online. I can just buy shoes for price y and sell it for price 1.5 * y, for 16 year old me this seemed like a way to make an amazing side income. I was quickly introduced into the world of sneakers by first entering a UK group called ExpressNotify. From there I was able to understand and learn about shoe botting.

Sneaker Bots would basically automate the entire process of adding to cart -> checkout -> purchase. At 16 I thought it was magic. I of course tried to make a bot back then, but with how secretive other developers were (as they should be) and how hard it was to find resources online, I basically winged it. I created selenium scripts which achieved nothing. Fast forward to my first year of university, me and a university friend decided to try and make a bot for Selfridges so we can buy restocks of some quite profitable shoes. We ended up trying to build a request based solution - however we got hit with bot protection, It took us a while to realize we can't just copy the request the browser made with Python's request libraries and be able to checkout like that. After we hit this huge wall of bot protection we thought lets just focus on University and get better at developing.

Now, after University, and getting a job at BP I thought it was the best time to start trying to build a bot again. Loads of things have changed though from University. Reselling sneakers is dead, I missed the NFT wave (botters made six figures easily), but I was in time for the memecoin rush. I began solo-building the bot to buy and sell Solana memecoins with different strategies. The bot was finished slightly too late to be used for the memecoin rush, but it was profitable & I learnt so much from it.

I wanted the project to be an entry way into profitable & interesting automation (I do not like working with non-request based solutions at all). From talking to crypto bot devs, I found out that all bots share quite a similar sort of infrastructure. I thought if i could build this, all the other things i'd like to bot will come easy. I could sort of create a codebase where I could easily add functionality for any sort of domain, whether it be retail websites, crypto, nfts - anything.

Why Crypto?

Crypto doesn't have bot protection, doesn't have reliability issues during high load, transactions that are executed are easily accessible. Under the hood, crypto bots seem the most accessible to any developer. It made sense why so many sneaker devs migrated to crypto and made a killing.

How does a bot work

Each bot has something called a task. A task's job is to buy 1 item from a certain website/place. Normally in sneaker bots, you'd have 1000's of tasks running at one time so that you have the most chances to be able to buy the most pairs of sneakers. The task would hold all the information you'd need to actually purchase a shoe, for example your name, address, card details. The bot would then do all the prerequisites, whether thats generating a cookie (to get passed bot protection), solve a captcha or wait until the product page is live. The bot would then use the details you gave it to purchase the sneaker.

Crypto bots work similar. You have a concept of a task, e.g. I'm buying a 100 of Coin X from pump.fun for example. That task's only job is to buy a hundred of that coin as quick as possible and that's it.

Languages

Back when I was in ExpressNotify I was able to talk to another developer who had just created a bot that could an easy run 10k tasks at any one time. This was crazy to me. At the time, most bots would only be able to run about 1k tasks max before crashing/having some performance issues. He told me about Golang and how he'd use GoRoutines to be able to spin up that many tasks.

I didn't want performance to be an issue for this application, and I wanted concurrency to be as easy as possible. After some more research and after confirming 2 of the bots I did own use Golang. I also decided to use it. It was also my first time using Go, but man, what an amazing journey. I really love the simplicity, performance, concurrency model and control you have with Go. My frontend - I ended up deciding on React. React is not performative at all, I know that. However, when i'm running about a hundred tasks, it will be perfectly fine. My idea is that eventually when I start doing 1000s of tasks, I can create the tasks in the UI. Then, start the tasks in a CLI for performance. (A lot of sneaker bots used to have CLI's to help with their performance, GUI's would eventually crash under the mass amount of updates)

MBot - Building it

I'm going to go over a higher level overview of how MBot was built, features of each side to it, some design decisions and some cool code snippets that i'm proud of.

Backend architecture

Transactions

Before we even get into how tasks work inside the bot, we should talk about transactions. Transactions in Mbot's can be considered a unit of work that a task must try and achieve. For example in MBot, that would be buying a token on Pumpfun. It could also be selling a token on PumpFunAMM. Each transaction has steps in how to achieve that unit of work. For crypto that's quite straight forward, we are basically gathering data, building a solana transaction, sending a solana transaction and then confirming the transaction actually went through. What nice about the architecture for MBot is that we've designed the transactions in a way that it becomes easy to swap out the implementation of a "transaction". Otherwise we'd have the issue of writing new logic to incorporate a new program every time we want to introduce a Solana launchpad.

type Transaction interface {
	BuildInstructions(ctx context.Context) error
	BuildTransaction(ctx context.Context) error
	SendTransaction(ctx context.Context) error
	ConfirmTransaction(ctx context.Context) error
	UpdatePosition(ctx context.Context) (tokenAmount, solAmount float64, pos *position.Position, err error)
}

Each launchpad's code will follow this transaction interface, quite straight forward. The update position step is to keep track of our trades positions which will help us with PnL reporting & Take profit/stop loss strategies.

We use these steps in the interface to then build steps of a task. A task will have other jobs other than actually executing a transaction, for example sending feedback to the user that the transaction has gone through, setting states of the actual task, sending webhooks etc.

For our transactions we had to reverse engineer how Solana launchpad's programs send buy's, sell's and create coins. I used pumpfun's own website's JS to kind of work this out and i'll be going over this on part 3 to this blog. Gratefully now, I found PumpFun has a Telegram where they release developer information behind their programs and updates to their programs. However you don't always have this luxury with newer launchpads. Launchpads most of the time when they first start out don't really want bot's to know how to use their site otherwise you get a lot of traders that try and buy a bunch of supply of the launchpads first few tokens and dump them on traders purchasing after them.

How do we make it plug and play?

We have a register that we create on startup with the program's names, we store the names programmatically only for now.

var (
	programMu sync.RWMutex
	programs  = make(map[string]Program)
)

// Register makes a Program available by the provided name.
// If Register is called twice with the same name or if program is nil,
// it panics.
func Register(name string, program Program) {
	programMu.Lock()
	defer programMu.Unlock()
	if program == nil {
		panic("program: Program is nil")
	}
	if _, dup := programs[name]; dup {
		panic("program: Register called twice for program " + name)
	}
	programs[name] = program
}

Here we register the names on startup, where "PumpfunNative" would equal the PumpFunNative's implementation of a program.

func registerPrograms(solanaStream datastream.DataStream) {
	program.Register(string(programnames.PumpfunNative), pumpfunnative.New(solanaStream))
	program.Register(string(programnames.PumpfunAMM), pumpfunamm.New(solanaStream))
}

Here we can see the program interface that would be the value of the register above.

type Program interface {
	NewBuyTransaction(task *tasks.BuyTask) transaction.Transaction
	NewSellTransaction(task *tasks.SellTask) transaction.Transaction
}

Each program will implement a programs interface, where each provider will expose their steps for buying and selling on that program.

This way throughout the program we can easily store the type of program the task is, resolve wherever we need to and get the implementation of the program.

I took this idea pretty much from how Go's stdlib does SQL. Where you just have to do something like:

db, err := sql.Open("sqlite", constants.Database)
if err != nil {
	return nil, err
}

So you can just easily get the implementation of the database you're using by keying in the database you'd like to use.

Tasks

So lets move onto the most fundamental part of the bot, A Task. We designed tasks to have information about what it's doing (buying/selling), crypto task information such as the token, the slippage, the node it's using. Then we have what the state of the task, which allows us to progress the task through different steps. Below is how the finite state machine works.

Rendering diagram…

We start each task within its own goroutine, where the goroutine will exit cleanly if the task's state hits a terminal state. I tried to think of all the possibilities that a normal user would face here, if you want to cancel your task half way through you should still be able to run the task after, if the task has an error before Done then we want the user to still be able to hit retry and restart the task.

func (e *Executor) Execute(ctx context.Context, done chan struct{}, tx transaction.Transaction, task tasks.Task) {
	defer close(done)

	for {
		state, ok := e.steps[task.State().TaskState]

		// terminal state
		if !ok {
			return
		}

		// if there is ctx error/ cancellation then we set
		if err := ctx.Err(); err != nil {
			e.setStateAndPublish(tasks.TaskCancel, task)
			return
		}

		if err := state.Fn(ctx, task, tx); err != nil {
			if ctx.Err() != nil {
				e.setStateAndPublish(tasks.TaskCancel, task)
			} else {
				e.setStateAndPublish(state.OnError, task, err.Error())
			}
			return
		}

		e.setStateAndPublish(state.To, task)
	}
}

This is how we execute a task given a number of steps., we go through each step, check for errors/cancellations and go from there - i've decided to hide the retry logic here just to make it a bit easier to read but the retry logic is quite straight forward. You just have to check if a task is retryable, if so then restart the loop. We spawn each execution of a task in a separate goroutine, blocking the function with the Execute function and after we clean up resources. This is a generic executor so for when we might have retail sites in the bot, we can reuse the executor.

go func() {
		m.executor.Execute(cancelCtx, done, transactionImpl, task)
		m.mu.Lock()
		delete(m.running, task.Id())
		m.mu.Unlock()
	}()

Why do we spin up a goroutine for every task? you may ask. Well goroutines are quite cheap in Go compared to other languages, we want each task to be as fast as possible and we do not want one task to block another task from running. You can almost think of it as a bot swarm, a bot swarm wouldn't really be a swarm if each bot waited for each other right? Go makes concurrency easy, but it comes at a cost, making it easy to have goroutine leaks, deadlocks and memory leaks. Ideally you'd want to handle a goroutine with a context or a done channel to make sure it closes correctly.

Strategies

Strategies are a way that we can orchestrate tasks. We currently have 3 main strategies currently in Mbot: Spam, New-Pairs Monitoring, QuickBuy/QuickSell. All 3 manage tasks in different ways and are useful in different situations. Before we go over what each strategy currently does, i should first go over the relationship between strategies and tasks.

Strategies and tasks share a many to one relationship, we can group a bunch of tasks by a strategy. Internally we do this by an ID that we give a strategy task to a regular task. We do this to allow a strategy task to orchestrate one to multiple tasks at a time. Some strategies require multiple tasks, such as Spam, New-Pairs monitoring whilst some strategies just require one task like Quick buy.

MBot made strategies easy to develop, and to implement. We have some common functions such as monitoring a coin's price action for take profit/stop loss strategies that can currently be used with any of the strategies labelled above.

How the strategies work:

Spam - Spam we are given a token address into the bot, we allow the user to choose how many "threads" they'd like to use - which creates a worker pool of the threads the user chooses. We also allow a user to have optional retries and allow them to specify a delay between each retry. How spam works is we will infinitely/finitely (depending on user's choice) spam buy transactions to a certain token address. We'd do this to try and purchase a coin as quick as possible basing on "luck" of getting a fast transaction.

func (s *Spam) Run(ctx context.Context, task *strategies.Spam) {
	select {
	case <-ctx.Done():
		return
	default:
		if task.StartTime == 0 {
			s.noTimerWork(ctx, task)
		} else {
			s.timerWork(ctx, task)
		}
		return
	}
}

//timerWork() is the same as the below but with precreation of tasks + use of time
func (s *Spam) noTimerWork(ctx context.Context, task *strategies.Spam) {
	var wg sync.WaitGroup
	for i := 0; i < int(task.NumberOfSubTasks); i++ {
		wg.Go(func() {
			s.work(task)
		})
	}

	go func() {
		wg.Wait()
		<-ctx.Done()
		s.cancelSubTasks(task)
	}()
}

New-Pairs Monitoring - This strategy tracks coins that have been freshly created on a specific launchpad. We allow the users to optionally use filters in the bot to filter down their buys. For any coin that gets freshly created we send a buy transaction for that coin.

func (afk *AFK) Run(ctx context.Context, afkTask *strategies.Afk) {
	coins := make(chan models.Coin, 100)
	defer close(coins)

	filterPipeline := filters.FilterPipeline{}
	for _, f := range afkTask.Filters {
		filterPipeline.AddFilter(f())
	}

	node, err := afk.rpcService.GetNode(afkTask.RPCGroupId())
	if err != nil {
		return
	}

	prog := program.Resolve(afkTask.Program)
	coinStream := prog.NewCoinStream(ctx, rpcgroupsModel.RPCNode{
		Http: rpc.New(node.Http),
		WS:   node.WS,
	}, filterPipeline)

	for {
		select {
		case <-ctx.Done():
			afkTask.SetStrategyState(string(strategies.CANCELLED))
			afk.strategyHub.PublishStateUpdate(afkTask.StrategyTaskId(), afkTask.StrategyState())
			return
		case coin, ok := <-coinStream:
			if !ok {
				afkTask.SetStrategyState(string(strategies.CANCELLED))
				afk.strategyHub.PublishStateUpdate(afkTask.StrategyTaskId(), afkTask.StrategyState())
				return
			}
			afk.handleNewCoin(ctx, afkTask, coin)
		}
	}
}

Quick Buy/Quick Sell - This strategy is quite basic. Given a token address, buy that token and vice versa for quick sell. The intention behind this strategy is to allow user's to use a chrome extension (not currently built) to expose a button on a launchpad to buy a coin by tunneling the request over to the bot's hosted address.

Monitoring

Monitoring is a big aspect of any bot. I remember when I was back in sneaker groups a sneaker dev mentioned that when you want to learn how to bot, you should learn how to monitor first. I took this advice with MBot and it paid dividends. The monitoring-first mentality works whether its crypto or retail.

To go through with monitoring we had to subscribe to the launchpad's contract via gRPC or WS. My node provider was Helius during development so we used WS's in MBot to stream data from the contract. I'll go into more depth about how we reverse engineered parts of PumpFun's contracts to be able to stream data in MBot.

In MBot's code we create a go pipeline that would do the following:

Monitoring Flow

Rendering diagram…

We created a cool generic function to actual stream anything we'd like from a Solana contract:

func Stream[T any](ctx context.Context, address, wsUrl string, streamer streamerFunc, parser func([]byte) (*T, error), cleanup func()) chan T {
	datastream, err := streamer(wsUrl, address)
	if err != nil {
		return nil
	}

	output := make(chan T, 100)
	go func() {
		defer close(output)
		if cleanup != nil {
			defer cleanup()
		}
		for {
			select {
			case <-ctx.Done():
				return
			case data, ok := <-datastream:
				if !ok {
					return
				}

				parsed, err := parser(data)
				if err != nil {
					logger.Error(err)
					continue
				}

				if parsed != nil {
					output <- *parsed
				}
			}
		}
	}()

	return output
}

Where we can pass in any streaming message + parsing method

e.g.

func (paccm *PumpfunAMMCoinCreationMonitor) StreamCoinCreation(ctx context.Context) chan models.Coin {
	parser := func(b []byte) (*models.Coin, error) {
		return transactiondecoder.ParseTransactionNotification(ctx, b, paccm.httpClient, paccm.filters)
	}

	cleanup := func() {
		paccm.datastream.Unsubscribe(paccm.wsUrl, paccm.monitoringAddress)
	}

	return stream.Stream[models.Coin](ctx,
		paccm.monitoringAddress, paccm.wsUrl,
		paccm.datastream.SubscribeToTransaction,
		parser,
		cleanup,
	)
}

This way we could reuse the same sort of monitoring architecture for monitoring new coins, but also monitor market cap changes for our selling strategies & for live market cap updates for the frontend. And eventually when we want to include EVM, we can still use the same streaming architecture that's implemented in MBot.

What's also pretty cool about MBot's solution is I created a multiplexer to reuse connections in case the user has multiple tasks. That way we can try to keep network throughput low and keep performance high.

Solana RPC Multiplexer Flow

Rendering diagram…

Looking back at our code we could make a nice performance upgrade by re-using output channels of the Stream function, e.g. 2 tasks with the same filters would reuse the same channel for the coin creations on a program.

However, MBot's monitoring flow i'm quite proud of how it was developed and it works well! There are new ways to monitor now on Solana that would give you a slight edge where you can parse transaction data even before it's been processed. Helius now offers it here.

In the next part of the blog I'll be going over a couple bits of our API on the backend & some cool abstractions/code snippets and flows which i'm proud of and then a deep dive into the frontend (mostly optimization techniques and reasonings behind design choices)