We are living in a digital civilisation that belongs to the web and internet. Our entire user experience as a consumer, as a citizen and perhaps as a being is transforming into an "information experience". While as a consumer our entire customer experience from ignition of desire to actual purchase from the organization point of view is a series of electronic data which the company can piece together, twist and turn to predict future consumer patterns, predict demand, innovation and more. From the citizen point of view as e-governance gets centrestage and the lives, times and assets of the country's citizens are electronically documented, some by legislation and other by initiative, the challenge is electronically generated and stored information and its retrieval at the right moment.
Electronic Vault of Information
As a consumer or a citizen how do you store your most valued electronic documents? Perhaps in a little corner in your computer and a backup in another storage device. Now what would you do if the stored documents became over a billion records with complicated multi categorization, to be maintained in a certain standard so that they can also be retrieved within x amount of time and also be read 10, 20 or maybe even 50 years hence. Complicated right? Or how about this. As an organization there are 1000's of people working and information both internal and external i.e. email systems, filer server environments, instant messaging platforms, content management and collaboration systems and more are the lifeblood of the enterprise.
Storing, managing and retrieving all this information is a highly complicated process. According to Anil Chakravarthy from Symantec Corp "The rapid growth in, and use of, electronically generated and stored information is inundating most organisations. Even the types of data used by organizations continue to expand. The use of systems that contain structured, semi-structured and unstructured data types continues to grow and they contain more business-critical information than ever before. This creates problems beyond just storage, since this data must be indexed and easily retrievable at a moment's notice for a wide variety of applications. Any system deployed within an organization for data retention, extraction and destruction must support all data types."
Information seen from an asset point of view and that's where solutions like Enterprise Vault solutions come into play. While simple storage from the consumer angle is a booming market with giants like IBM, Seagate, Samsung, Hitachi and others are offering innovative solutions.
On the enterprise side the solution is highly complicated where it's not just about storage but also timely retrieval and in many cases also about legal compliance, there are players like Symantec with offerings like Electronic Vault that offer a software-based intelligent archiving platform that stores, manages, and enables discovery of corporate data from email systems, filer server environments, instant messaging platforms, and content management and collaboration systems.
It simply doesn't end there. Storing data is just one challenge. Statutory policies and requirements is another combined with e-governance as it takes off the entire information storage and retrieval is a challenging task. According to Anil Chakravarthy "Complicating the issue still further are ever more stringent statutory requirements for data retention and increasing requirements for capabilities that can extract data for the purpose of supporting legal discovery efforts." For instance take the case of Indian hospitals that have subsidiaries overseas have to be HIPAA and/or SOX compliant in order to do business in countries abroad. A regulatory requirement for storing, with the capability to retrieve, patient records for twenty years can be addressed only through high end technology. These technological advances are critical for hospitals to stay in the business.
It does end just there. Next comes a complicated task of standards. Noteworthy is standards play a very crucial role in terms of retrieval of information. As an example consider the case of intelligent archiving and retrieval of voice files which is an important area for compliance. Says Anil "Addressing the challenges we are currently working on a project called XML Ingestion for Enterprise Vault (XIVE). This project is aimed at enabling Enterprise Vault to archive and index a variety of popular file formats."
The case of digital heritage is also gaining ground and worldwide efforts are being made towards this. Take the case of The Digital Preservation Coalition ( www.dpconline.com ) amongst others are working towards this. In April this year they are holding a conference on Significant Properties of Digital Objects. According to them "Significant properties are essential characteristics of a digital object which must be preserved over time for the digital object to remain accessible and meaningful."
From my archives
Six years ago I had written a story on "Web Archiving" having mentioned the BBC as a case. Cathy Smith of the BBC had spoken about the huge logistical and legal problems this can involve and how with a million pages on its web site, and each regularly being updated, what a challenge archiving was. I had also mentioned about a beauty called Waybackmachine (www.archive.org ) which archives websites for ordinary mortals. That was 2002. How far has technology empowered that in 2008?
Waybackmachine remains a beauty and information on it is simply becoming more abundant with more than 2 billion pages in its kitty. For enterprises hopefully solutions like Electronic Vault and their tribe solve the problem.
Puneet Mehrotra writes on technology www.thebusinessedition.com