What is serverless computing?
The rapid pace at which technology is moving is both disorienting and exhilarating. Just like the applications and devices themselves, the infrastructure used to facilitate the ongoing use and development of technology has seen its fair share of change.
Initially, organisations had to own and maintain the physical servers needed to perform their required computing – then came the “Infrastructure as a Service” revolution in which servers were rented from cloud providers (Amazon Web Services, Google Cloud Platform, Microsoft Azure). This removed the need for physical maintenance however, the need for software maintenance remained. Next, the idea of serverless computing (“Function as a Service”) was introduced, with the key difference to cloud computing generally being that servers are only rented (and paid for) as they are used and maintenance and ownership in their entirety are handled by the provider.
The use of serverless technologies is fairly commonplace today and continuing to grow rapidly, with the Asia Pacific being tipped as the highest growth region for serverless computing in 2021-2026. They are used to achieve a range of outcomes, from the customer service chatbots used on websites to Internet of Things (IoT) devices like Amazon’s Alexa. Because these have a highly variable demand it makes sense to only pay a fee for when they are used.
Integration platforms are a prime example of variable demand and are a perfect, but currently underdeveloped, use case for this type of technology. In our experience companies’ integration platforms are typically run using on premise servers or a standard cloud deployment. In many cases this leads to cost inefficiencies as the servers sit idle, or increased workloads as teams are left to perform maintenance tasks to manage the infrastructure. We’ve highlighted some of the key benefits of using serverless technology for data integration.
Benefits of serverless data integration
The following list shows the most significant merits related to performing data integration using serverless technologies. The benefits are logical in that they stem from the serverless model being owned and managed by a cloud provider and payment being based on function execution.
Easy implementation – A number of the usual requirements when deploying an integration platform can be eliminated under a serverless model, including virtual machine (VM) related tasks. This means that implementation is significantly quicker as these tasks normally take around a week worth of effort.
Built in monitoring – A typical cloud deployment requires developers to set up their own monitoring. In a serverless scenario this monitoring is done through existing tools supplied by the cloud provider.
Automatic scaling – Serverless systems are described as ‘elastic’ due to their ability to seamlessly scale in both directions. The systems automatically scale server instances both up and down in 100th of a second to handle the required load with no additional effort required from users. Even amongst non-serverless solutions that can scale this takes several minutes, meaning they will always have to be overprovisioned.
Cost effectiveness -The very premise of serverless is that you only pay for the instances in which you’re executing functions as opposed to renting or owning the servers for an entire block of time. Most integration scenarios are schedule based synchronisations, where integration is executed in defined intervals (hourly, daily, monthly or weekly), therefore servers are utilised for a minimum period of time but the infrastructure is often still paid for 24/7.This often results in a vastly more cost effective deployment, as illustrated by the graphs below.
We compared a standard cloud setup with a serverless deployment for the same generic integration scenario and found that the monthly recurring cost of cloud infrastructure was reduced by 97%.
Standard Deployment vs Serverless Deployment
Only the production and general costs were considered in this example, in UAT and dev environments the cost differences between serverless and standard cloud are marginal. In this, and many other cases, the requirements of a simple integration allow some components to operate within the bounds that are still zero cost. Items marked in red are those that may incur additional costs if usage is significantly higher than the month examined for this scenario.
Standard cloud – cost breakdown
Serverless – cost breakdown
For all their positive features, there are some limitations present in a serverless system. However these shortcomings are specific and limited, so tend to only further illustrate the ongoing potential of the technology.
High and constant workloads – Integrations that require many functions to be consistently executed can dissolve any cost benefit of a serverless platform. For example 200 million API requests a month.
Long running functions – If the system involves operations or backend calls which take 15 minutes, then a typical infrastructure as a service solution is better suited.
Why consider serverless data integration?
Even when technology is doing its job and working as required, with the rapid pace of change you can’t have a set and forget attitude. Frequent innovation in the sector means that constantly reassessing your technical infrastructure can really pay dividends. It’s rare to find investments that so clearly demonstrate ongoing cost savings but companies that have legacy integration platforms running time based synchronisations, using on premise or standard cloud servers, could be looking at a real golden goose if they take the time to assess their server needs now.
Of course if you’re on the journey to establish integrations between your systems, these technologies should absolutely be considered as part of the plan.
We’re really excited to demonstrate and talk about this as an option, if you want to know more contact us.
Terry Donnelly is the Technical Services Leader at Novigi, and is based in the Sydney office.
For more information about anything you’ve read here, or if you have a more general inquiry, please contact us.