By Mark Weedon, EMEA Technical Director, MariaDB Corporation

While it’s true that the COVID-19 pandemic has generated a number of decision-making difficulties for CTOs, the uncertain business environment created by unprecedented levels of stress for existing digital infrastructure has made digital transformation a top necessity. With everyone relying on digital tools more often, making digital infrastructures work better together has been an essential survival mechanism for organisations throughout the pandemic.

Industries such as e-commerce, food delivery, streaming media and workplace collaboration tools needed to suddenly meet rapidly growing demand almost overnight. Flexibility and agility have become long term priorities as a result.

The importance of scalability 

The ‘viral’ nature of the internet means applications and video games can experience exponential growth in the numbers of users in as little as an hour. The power of the internet, the growth of datasets, and the relentless rise in the number of interconnected devices (IoT) has been known to us for years, so why did it take the pandemic to force IT decision-makers to wake up to the urgency of deploying full-spectrum scale-out solutions?

SaaS applications such as Slack saw a huge increase in demand for its services when the world moved to a work-from-home model. The platform found itself needing to handle a rise from 8m+ daily active users in 2018 to 12m+ daily active users in 2020. 

Meanwhile, WhatsApp’s changing privacy policy from last year has proved to be another indicator of how quickly things can change online. Many users abandoned WhatsApp for competitor platforms including Signal and Telegram, requiring these businesses to scale rapidly to accommodate up to 9 million additional customers in a remarkably short period of time! 

Databases are key to successfully running applications and servicing customers around the world as the mechanism that stores and processes data. As businesses scale, so too must their databases to keep up with demand. The pandemic reinforced, again, that the future of many businesses will rely on flexible and scalable database solutions that can provide support through the good (and challenging) times.

How do we achieve scale?

Relational databases made their mark following commercial success back in the 80’s, championing a ‘does-what-it-says-on-the-tin’ track record for being reliable, consistent and compliant. However, as businesses have become more global – reaching new users and more consumers, these traditional relational databases fell short to deliver the scalability needed to support increased demand. NoSQL tried to address scalability but compromised data integrity. Some companies attempted sharding as a scalability solution which quickly became complex and unmanageable. Others turned to expensive hardware solutions that only the elite could afford.

Next generation relational databases have emerged more recently leveraging distributed SQL technology that makes this possible. Distributed SQL solutions offer dynamic scalability and facilitates flexibility in the form of adding or removing infrastructure to match supply with demand, meaning such solutions will be heavily relied upon in the wake of the pandemic. These solutions are also now available as cloud database services for even easier and faster deployment. 

The pandemic has taught us that companies must be able to pivot their business models and adjust to fluctuations in the number of customers they are serving, without incurring high costs to do so. Sophisticated distributed SQL offerings address the increasing number of mission-critical applications that require both the reliability and consistency of relational databases, with the added need for scalability. Once deployed, they consist of many database nodes working together as one. Just add more to scale. And if some fail, no problem. By its very nature a distributed SQL database is fault tolerant and keeps running under any condition.

Investors are also taking notice and have invested hundreds of millions into distributed SQL technologies, making it the hottest new market for databases.

Choosing your database provider

The goal for many CTOs at present is primarily to move as many applications to the cloud as they can, but they must first find the right database to support them. Switching thinking from short to long-term and preparing a business to become vendor agnostic will better prepare business leaders for any unforeseen challenges that might arise. 

The ability to support mixed workloads, analytics, and transactions, at massive scale will also put companies in the best position to grow and be successful. Not only this, but companies are also now looking to transition to scale seamlessly, which means they need to be able to start small – with a small database size that suits their needs. Then as they grow, make use of a database with dynamic scalability which allows simultaneous growth of a database footprint, without having to migrate or make any changes to existing applications.

Getting different applications to work properly with one another in a way that is customised to a business can be extremely difficult, and scalability needs add another layer of complexity. The appetite among businesses for swapping out mundane database tasks for built-in automation which frees administrators to work on more strategic initiatives like moving to the cloud is growing.

When looking at database investments, it is a much better use of resources to select a DBaaS (Database-as-a-Service) that lets you spin up a database in minutes. Businesses want quick response times from an experienced team that acts as an extension of their own. The newest Distributed SQL offerings that come packaged as a service are solving key problems highlighted during the pandemic by providing businesses with truly flexible solutions that allow businesses to scale out and back, without breaking the bank.

Customers’ priorities have changed for good, and DBaaS providers are offering invaluable support and guidance at a time where forgoing digital transformation is not an option.