By Nick Heudecker, Senior Director at Cribl
The UK’s economic crisis has brought around a myriad of problems. The Organisation for Economic Co-operation (OECD) presents a bleak prediction that by 2023 only Russia will register weaker than the UK in the G2. As such, businesses will likely need to make efficiencies, meaning tech budgets could be slashed, jobs lost, and with spiralling inflation and energy price hikes, everyday costs on the business spiralling – including those associated with data. And it’s no secret that large amounts of data require large amounts of money to manage efficiently.
Most organisations will have their data spread around various different networks and systems, and each of these will require different applications to store, run, retrieve, transfer and monitor that data. With all this in mind, businesses are feeling the toll of the economic crisis just as fervently as everyone else. With the nation on the brink of a recession, organisations can’t afford to splurge money on a multitude of fees associated with storage, data recall and monitoring. Instead, they must look for more effective means of getting better outcomes, but how can they do this?
A shift is needed: enter observability pipelines
It’s no secret that storing and manipulating data can be expensive, and the more data there is, the more expensive it gets.The high price tag of monitoring forces teams to compromise on what exactly they’re monitoring. With tough decisions around which logs, metrics, and traces to keep whilst also staying within budget, many teams even at the largest enterprises simply can’t store everything they need to observe their environment.
For large enterprises storing masses of data, the priceof storing and moving that data can skyrocket but employing an observability pipeline can make a huge difference. Put simply, observability is the practice of interrogating your environment without knowing in advance the questions you need to ask. But what does that mean in practice?
Today, organisations want to pull data from the network, servers, endpoints and applications. They typically take those different types of data and load them into a logging analytics or security platform. However, these platforms don’t provide much control over that data, and they don’t allow you to share data between destinations.
Observability is more operational than analytics based. So instead of running point-to-point solutions, observability means taking all of the event data you have – logs, metrics and trace data – and running it through a centralised point. You can decide how you want to route that data, how you want to filter it, how you might want to redact data such as PII, so you can govern all of your data at one point. This way you can also reduce the amount of data that you are sending to downstream systems, and therefore puts the decision-makingaround processes and storage back into the hands of your organisation.
An end-to-end observability pipeline can in most instances lower infrastructure costs by 30%. Organisations can also reduce the cost of achieving observability by reducing the time and effort required to set up and manage data pipelines. No customisation effort is needed on the part of a DevOps team to take data from any source and send it to any destination, in any format that it needs to be in when it gets there.
The time to act is now
In a world dominated by the digital landscape, businesses need to ensure the reliability of their applications – especially as they increase in complexity. Observability provides them with the huge advantage of understanding what’s happening in this enormouslycomplex environment, enabling them to provide a better service to customers and make managing these environments more cost-effective. It’s a win-win situation. Observability pipelines can also helpyouincrease revenue by helping you to understand your data betterand reduce spending, which is exactly what’s needed in the current economic state.
Ultimately, everyone is going to have to start massively cutting costs. Whether that’s as an individual or as an enterprise. Somewhere of vital importance for the enterprise however is data storage, recall and monitoring. By deployingan observability pipeline, organisations can not only lower their costs by improving internal operations and saving time but can also streamline operations and ensure added security.
