 |
 |
 |
| ARTICLES | May 2007 |
Discovering the secrets of managing data growth
Coping with ever-increasing database storage requirements needs more than a quick fix. An enterprise data storage (EDM) strategy is one of the alternatives
EXPONENTIAL data growth rates are threatening the operational efficiency of many large enterprises, potentially slowing application performance, straining financial and technical resources and jeopardising the timely completion of business-critical processes. Organisations are finding it increasingly difficult to manage the impact of this growth on systems, applications and budgets.
In a recent survey conducted on behalf of the Independent Oracle Users Group (IOUG), more than 92 per cent of respondents said they expected their database storage needs to increase over the coming year.
Reasons for growth
One of the key factors behind this growth is a steep increase in numbers of electronic transactions. This is primarily due to a rise in ebusiness and increasing reliance on the Internet. In addition, initiatives like retailers’ card loyalty schemes are fuelling a desire to store more information.
Data duplication also contributes to growth. The challenge of managing large data volumes is compounded by cloning, or copying the production database to support other functions within the organisation.
It is not uncommon for businesses to maintain several backup copies of critical data or to implement mirrored databases that provide assurance against data loss. Disaster recovery plans often require data duplication in order to store critical data in an alternate location. It is also common to clone an entire production database for use in application development and testing initiatives.
Compounding the challenges are data retention regulations that require the management and storage of different types of data for extended periods. This data must remain easily accessible so that companies can respond to audit or discovery requests.
A large proportion of growth, however, is simply down to organic business development. The quantity of data retained in application databases will increase as a natural part of daily business activities.
It is not unrealistic to assume that in only a few years, a company’s eCommerce, CRM and ERP applications will collect millions of new transactions per day! However, as time goes on, the business value of legacy data will decline, creating a backlog of historical information that requires orderly data management and storage policies.
Issues to resolve
The consequences of these exponential data growth rates can affect all areas of the enterprise, making it difficult to address critical business issues, including meeting information governance requirements, controlling IT infrastructure and ensuring business continuity.
Accumulating vast amounts of application data adds complexity, risk and cost to any business. In addition, it is often difficult for organisations to anticipate surges in demand. They can no longer accurately predict what their data growth is likely to be over a given period.
Also, most large corporates are running enterprise applications which require frequent systems upgrades and migrations. Large enterprises often suffer severely from prolonged downtime of these business-critical systems, caused primarily by the need to port huge volumes of data from one application to the next.
A short-term fix
Most organisations tend to react to the problems of rapid data growth by simply aquiring more storage. This common response may resolve constraints temporarily. However, it also has drawbacks. Chief among these is that it fails to address the root cause of the problem.
Buying in more storage does not stop or even slow database growth. Although businesses may receive significant incremental benefit from adding storage to a small database, that benefit reduces as more storage is added.
The other problem is that expensive storage solutions are a finite resource for most businesses. When organisations have too little storage capacity, their IT management costs tend to be high. This is because they will generally be looking to release more resource from within their existing IT infrastructure.
They will typically spend significant amounts of employee time and IT resource optimising system efficiency. It is a thankless task. As more data is added, the enterprise’s applications steadily degrade. 60 per cent of respondents to the IOUG survey reported that lack of available storage impacted on the performance of their databases and applications.
Many businesses have even delayed the roll-out of applications because they have too little storage resource. These companies realise that any application upgrade is likely to result in the need to take associated databases offline. Any downtime is likely to be unacceptable particularly if the chosen application is business-critical.
The benefits of enterprise data management
So what options, other than traditional data storage are available to businesses? One important alternative is enterprise data management (EDM). This is based on the principle of database archiving. Its key benefit is in enabling businesses to manage and maintain the size of their database over a given period.
It enables organisations to select data within their overall production database based on a set of business rules, compress it and place the compressed data within a separate archive file. Data within this file is still accessible to the application but users save money by removing it from expensive tier-one storage and significantly reducing the size of their data footprint.
Using enterprise data management, a business could, for example, choose to archive off any historical financial records that are more than two years old. It can then automate the process so that as data exceeds the time limit, it is instantly archived.
Reducing the amount of information in the production database means that less disk space is required for application data, cutting storage costs. And because there is less information to assimilate, processes run faster, operations run more efficiently and organisations derive more business value from mission-critical applications.
This type of enterprise data management strategy can be applied across an organisation independent of application, database or infrastructure. And typically, large businesses will achieve a payback on their investment within six months. At the same time, employee, supply chain efficiency and customer service levels can be dramatically improved.
There are also significant benefits from a compliance perspective. Data within a live production environment can still be accessed modified and changed. However, once it has been archived, it cannot be altered or modifed. In compliance terms, it is audit-ready.
Bright future ahead
Today, more businesses, particularly large financial services companies, major telcos, utilities, government and public sector organisations are appreciating the benefits of enterprise data management.
While adding extra storage is a reactive approach to data growth and merely serves to alleviate the worst symptoms of data growth, EDM is a proactive strategy, which allows enterprises to wrest back control of their data and to manage it efficiently on an ongoing basis.
Including routine archiving as part of scheduled application and database maintenance procedures enables companies to meet performance targets consistently while at the same time providing superior customer service.
The author Warren Daniels is marketing manager, Northern Europe and South Africa for Princeton Softech, a major supplier of enterprise data management (EDM). He has held this role since October 2006. Prior to his appointment he spent three years as marketing manager for Cardiff Software, a leading provider of software for intelligent document solutions. As Northern Territory Marketing Manager at Princeton Softech with ten years of technology marketing experience across various industries, Warren Daniels has established himself as a thought leader in the area of database archiving and data privacy. He has been published in a number of industry publications and is an enthusiastic speaker at various events. He holds a 1st Class BA Honours in Business Studies with a concentration in marketing.
