The self-built API gateway helped us save 38% on third-party call costs, but we ran into issues with cross-regional adaptation.

Last week, our Southeast Asian e-commerce tools team just reviewed the technical cost statements for Q1. The colleague in charge of the payment interface almost jumped for joy: after switching from a third-party gateway to a self-developed one, the monthly API call costs decreased by 38%. However, during the grayscale testing before a major promotion, the user payment success rate at the Singapore node suddenly dropped by 6 percentage points. It took us half a day to realize that the gateway’s cross-regional circuit-breaking rules had not been properly adapted.

First, let's understand what a self-built gateway actually is.

Simply put, you write a set of traffic entry programs yourself, and pass all external API requests, third-party callbacks, and cross-service calls through this layer first, replacing the third-party gateway service you bought previously. The version we are using now runs on four 2-core 4G overseas node servers. A single server can carry 1200 requests per second, fully covering our daily payment and product synchronous call needs of 1.8 million times.

The three tangible benefits you can obtain are:

• Firstly, the costs can indeed be reduced significantly. The third-party gateway we used before charged based on the number of calls, costing us $2,100 per month. Now, with our self-built server and monitoring system, the monthly cost is less than $1,300, and we're not subject to tiered price increases.
• Secondly, the rules are completely up to us to decide. Previously, the throttling by third-party gateways was uniform. During major promotional events, we had to temporarily increase the throttling quota for the payment interface, which required a three-workday process to handle each request. Now, we can modify the configuration in the backend in just 10 seconds, and the changes take effect immediately.
• Finally, we don't need to use third parties for data processing. Since we develop e-commerce tools, we often need to handle payment callbacks from many users. Previously, all requests had to go through the nodes of third-party service providers. Now, all sensitive data is processed on our own servers, which has significantly reduced our compliance costs.

Don't rush to build something; first, let's take a look at the pitfalls we've encountered.

The first issue was the problem with cross-regional network adaptation. Initially, we placed the gateway main node in Singapore. When we launched the Malaysia site, we directly rerouted all traffic through it. As a result, local users' requests had to first go through Singapore before being forwarded back, which added an additional 200 milliseconds of network latency. Many users couldn't wait and closed the payment page before the process was completed. It took us a week to set up the edge nodes for Malaysia.

The second issue is a lapse in security rules. Previously, third-party gateways came with built-in protection against DDoS attacks and malicious requests. When we set up our own system, we forgot to include this protection. On the third day after going live, our inventory API was hit by 200,000 invalid requests from crawlers, which nearly caused it to crash.

The third issue is that the operational and maintenance costs are higher than expected. Previously, when there were problems with the third-party gateway, we would simply contact customer service. Now, the three of us in the backend team have to take turns working night shifts to monitor the gateway. Last month, just troubleshooting gateway failures took up 15% of everyone's working time.

What kind of teams are suitable to work together, and which ones aren’t worth wasting time on?

If your team makes more than 1 million API calls per day on average, or if you have a large amount of sensitive data that needs to be processed, or if you're tired of the rate limits and ticketing processes of third-party gateways, then building your own gateway is definitely worth the investment.

However, if your team has fewer than 2 backends, or if your business is still in the rapid trial-and-error phase where the interface rules change every week, then it's really not necessary to spend the time on this. You can just use a third-party gateway for now; you can switch to your own solution once your business has stabilized.

3 Practical Tips for Those Building It for the First Time

1. Start with a single scenario first; don’t make a complete replacement all at once. At the beginning, we only diverted the traffic for the payment interface to our own gateway, and it worked fine for two weeks before we gradually migrated other interfaces. Even if there were issues, it would only affect the payment scenario and not cause the entire site to go down.
2. Be sure to conduct cross-regional stress testing. This is especially important for businesses operating overseas. Don't just test locally before going live; set up test nodes in your target markets and run 24-hour stress tests to check for any issues with latency and packet loss rates.
3. Don't skimp on the monitoring panel. At the very least, keep a close eye on three key indicators: the number of requests, the success rate, and the latency. Set appropriate alarm thresholds so you don’t wait until users start complaining to realize that the gateway has gone down.

Common Small Issues

Question: Is it necessary to write the code in Go or Rust?
No need. Our first version was written using Node.js, and it can handle peak loads just fine. You can use whatever language your team is familiar with; it’s not too late to switch if there are performance bottlenecks in the future.

Question: Which is better compared to the gateways of cloud providers?
The gateways provided by cloud vendors are more flexible than those from third parties, but they still don't offer the same level of freedom as self-built solutions. If you don't want to be tied to a particular cloud provider, it's more cost-effective to build your own system.