Experiment


Experiment setup:
We present some preliminary experimental results from our implementation of the software upgrade in the Thor OODBMS. Our experiment focus on the worst case analysis and from the results of these experiments, we argue that the performance impact of our lazy, transaction-based upgrade propagation scheme is acceptable. We are unaware of any previous emprical results that actually proves this point in a real system.

Our experiment is based on the de facto OO7 Benchmark [1]. In summary, the database consist of a tree of assembly objects made up of 3 composite parts. Each composite part is in turn, made up of several atomic parts. We conducted our experiment on the small database configuration. The interested reader should refer to [1] for more details on the benchmark.

The first set of experiment measures the cost of checking for an upgrade. When an application transaction encounters an object, it needs to determine if the object is in fact outdated. Outdated objects can exist in the FE since we propagate our upgrades lazily. Consequently, each application transaction will incur an additional overhead of checking if the object needs to be upgraded. Our experiments showed that the impact of this cost is minimal when running application transactions that did not require any upgrades. In this experiment, the objects required by the application transaction are already in the FE's cache; otherwise, the cost of shipping objects from the OR to the FE will dominate. There are two scenarios that we consider, namely: In the first set of experiments, we measure the overhead of running application transactions when all the objects are up-to-date. Next, we measure the cost of actually running the upgrades, excluding any application specific costs since we are purely measuring the overhead and not the work done by the upgrade itself. In order to do this, we run a null transform on the objects. A null transform transforms an object from Class_1 to Class_2, where Class_1 and Class_2 are identical i.e., it simply copies objects from one schema to another schema of the same structure. In this experiment, we upgraded the schema of the Atomic Part and run traversals on the database tree. When an atomic object is encountered for the first time, the application transaction will be suspended and the transform function will upgrade the atomic object to the new version before the application transaction is allowed to resume. In this way, the application transaction will see only the latest version of the Atomic Object. The fact that the FE contains outdated versions of Atomic Objects is invisible to the application transaction.

Our experiments showed that application with upgrades and that without upgrades are of the same order of magnitude in terms of performance.

[1] M. J. Carey, D. J. Dewitt, C. Kant, and J. F. Naughton. A status report on the OO7 OODBMS benchmarking effort. In Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), October 1994.