PSI - Aspen Extended Multivalue (eMV) - Information Storage Without Limits

Aspen Extended Multivalue (eMV) - Information Storage Without Limits

Published in the Open Systems Database Association Newsletter, Fall 2000

Has anything really changed in the last 30 years?

In the early 1970's, the TV show “All in the Family” was popular. The Jackson Five and The Carpenters were on every radio in America. Personal computers hadn't been invented. As if that's not enough, get this - Bill Gates wasn't rich.

What is also significant about the early 70's is that first strides were being taken toward standardized information storage. While accounts vary, at some point between 1969 and 1971 an IBM mathematician named Dr. E.F. Codd penned a series of rules for how information should be stored. These rules, also known as “the normal forms”, have been a mainstay of traditional data storage for decades.

Fast forward to today. Television has literally hundreds of channels. If you want to hear the Jackson Five or Carpenters, you'll need to find an oldies station and wait for a good long while. Personal computers have broken the 1Ghz speed envelope and slightly less powerful units fit into your shirt pocket. And Bill Gates - well, a one point shift in the stock market causes him to either gain or lose more money than the gross national product of half the world's countries.

Perhaps the most striking difference between then and now is in the definition of “data”. In the 70's, commercial databases were hardly feasible, as the physical space required to house the necessary disk drives was largely prohibitive for companies any smaller than, say, IBM. Those times when information was committed to persistent storage, the content was predominantly numbers with a smattering of text now and again. Today, it's a whole different story; Data is no longer just simple text and numbers; it's all that plus streaming audio, video, graphics and documents in dozens of different formats, not to mention international text. Is a Word™ document data? Of course it is! Can a photograph or family video be considered data? Absolutely! The fact is that data is all around us in various shapes, sizes, character sets, and content.

Despite all the changes in technology and exponential expansion in the types of information to be stored, prevailing database technology remains largely as defined thirty years ago. Certainly, enhancements have been added to different products over the years, but to what end? Our world is not columns and rows, yet traditional databases continue to define data in this format. Our world is also much more than text and numbers, yet extended information such as multimedia, multidimensional structures, and international character sets have been largely ignored throughout the database industry.

Multivalue technology itself is twenty-odd years old now, and while it does provide a certain degree of multidimensionality, the technology itself does have limitations. Ever tried to store a photograph in an attribute? Or what about a list of audio clips? Or what about Japanese text inside of an English record (without expanding the entire record to a wide-character format)? Size considerations notwithstanding, some things are just not practical even with a Multivalue system.

So if Multivalue doesn't inherently support multimedia and international character sets, and traditional database technology doesn't support multidimensionality (et al), what are our options? Must we accept the limitations of the prevailing technology and force our data to fit those limitations?

Despite the contemporary relevance of these questions, they were actually first posed in 1997 when one of our customers went searching for a database for a non-US (wide character) installation. When the search yielded no feasible options, we began to question: Is an internationally aware database simply a technological impossibility? Or are there other factors standing in the way of this functionality?

In response to these questions, we (Precision Solutions) set out to create a storage technology that supports multidimensional structures AND international character sets. En route to this goal, we found that the underlying technology would also support multimedia and other data types, thus making this technology the first ever to support n-dimensional multivalued lists of domestic text, international (wide) text, audio, video, graphics, and any other types of information imaginable. At this point you may be inclined to ask “How is this possible?” or “Why haven't the ‘big boys' come up with such a technology?” The answer to these questions is really quite simple: There's a lot to be said for learning from the rules of the past without actually following them.

And this is exactly what we did. By combining lessons learned from relational technology with some of the more innovative ideas from the Multivalue world, we found that an international and multidimensional database is not only possible, but can be achieved with better performance, more flexibility, and more scalability than has previously been available. This technology - known as Phoenix - forms the basis for our upcoming Aspen Extended Multivalue (eMV) database on Windows, Linux, and Unix.

Here are a few highlights:

From its Multivalue heritage, the basic unit of data is a dynamic array. However, unlike its Multivalue ancestry, a dynamic array can hold any structure or content, including domestic text, international text, multimedia, and any other type of data while maintaining the dimensional structure.
Multidimensional structures aren't limited to attributes, values, and subvalues. Instead, values can have values that can have values up to the memory limit of the platform.
Regardless of the size of a particular structure, whether it be a record or multivalued list, there is no degradation in extracting from the front or back of the structure. As a result, dimensioned arrays and constructs like the REMOVE in BASIC are completely unnecessary.
A graphic, audio clip, or video could be used as the key to a record in a file. (Granted, the audio or video clip may be a bit of a stretch to make work in a practical sense, but it's possible!) Moreover, a multivalued list of these types of elements could be used as the key to a record just as easily, thus simplifying the task of building multiple-part keys.
Multiple languages can be stored in the same record or multivalued list, and the entire structures need not be converted to a wide format to support one field that needs wide character support. This feature allows the database to host any number of languages without having to be configured for a specific locale.
The file format is 100% compatible across Windows, Linux, and Unix systems. In other words, if you start on one platform and need to upgrade to a different platform, you can copy your files from one system to another and keep working without conversion. (Qualifier: the bit depth of the platforms must match.)
Specific files can be created so as to be encrypted to prevent unauthorized access or disclosure via “back door” approaches.

To summarize, the computing world has changed substantially in the last 30 years, yet data storage technology has remained virtually unchanged. The time has come for a new class of information storage technology designed for the needs of corporate e-commerce - and beyond.

Kevin King is the President and Chief Technologist with Precision Solutions, Inc., a leading technology solutions provider in Longmont, Colorado. He can be reached by email at Kevin@PrecisOnline.com or by voice at 303/651-7050.