Tuesday, July 26, 2011

some notes about In Memory Database

Accessing data in memory reduces the I/O reading activity when querying the data which provides faster and more predictable performance than disk. In applications where response time is critical, such as telecommunications network equipment, main memory databases are often used.

ACID support
Volatile memory-based MMDBs can, and often do, support the other three ACID properties of atomicity, consistency and isolation. Many MMDBs add durability via the following mechanisms:
- Snapshot files, or, checkpoint images, which record the state of the database at a given moment in time.
typically generated periodically, or, at least when the MMDB does a controlled shut-down. Only offer partial durability.
Full durability need to be supplemented by one of the following:

- Transaction logging,
records changes to the database in a journal file, facilitates automatic recovery
- Non-volatile random access memory (NVRAM),
static RAM backed up with battery power (battery RAM), or an electrically erasable programmable ROM (EEPROM). Recover the data store from its last consistent state upon reboot.
- High availability (reduncy/replication)
rely on database replication, with automatic failover to an identical standby database in the event of primary database failure.

To protect against loss of data in the case of a complete system crash, replication of a MMDB is normally used in conjunction with one or more of the mechanisms listed above.

--------------

Oracle In-Memory Database Cache Architecture
- shared libraries
It's in contrast to a more conventional RDBMS, which is implemented as a collection of executable programs to which applications connect, typically over a client/server network.

- memory-resident data structures
It is maintained in shared memory segments in the operating system and contains all user data, indexes, system catalogs, log buffers, lock tables and temp space
.
- database processes
a separate process to each database to perform operations including the following tasks:
> Loading the database into memory from a checkpoint file on disk
> Recovering the database if it needs to be recovered after loading it into memory
> Performing periodic checkpoints in the background against the active database
> Detecting and handling deadlocks
> Performing data aging
> Writing log records to files

- administrative programs
Utility programs are explicitly invoked by users, scripts, or applications to perform services such as interactive SQL, bulk copy, backup and restore, database migration and system monitoring.

- IMDB Cache
A cache group is created to hold the cached data. It is a collection of one or more tables arranged in a logical hierarchy by using primary key and foreign key relationships. Each table in a cache group is related to a database table. A cache table can contain all rows and columns or a subset in the related table. Cache groups support these features:
> Applications can read from and write to cache groups.
> Cache groups can be refreshed from Oracle data automatically or manually.
> Updates to cache groups can be propagated to Oracle tables automatically or manually.
> Changes to either Oracle tables or the cache group can be tracked automatically.

When rows in a cache group are updated by applications, the corresponding rows in tables can be updated synchronously as part of the same transaction, or asynchronously immediately afterward. The asynchronous configuration produces significantly higher throughput and much faster application response times.

Changes that originate in the tables are refreshed into the cache by the cache agent.

Each cache group has a root table that contains the primary key for the cache group.
Rows in the root table may have one-to-many relationships with rows in child tables,
each of which may have one-to-many relationships with rows in other child tables.

Each primary key value in the root table specifies a cache instance. Cache instances form the unit of cache loading and cache aging.

The most commonly used cache group types are:
> Read-only cache group: committed updates to tables are automatically refreshed to the corresponding cache tables in the IMDB Cache database.
> Asynchronous writethrough (AWT) cache group: committed updates to cache tables in the IMDB Cache database are automatically propagated to the corresponding tables asynchronously.
> Synchronous writethrough (SWT) cache group: committed updates to cache tables in the IMDB Cache database are automatically propagated to the corresponding tables synchronously.
> User managed cache group

Cache groups can be either dynamically loaded or explicitly loaded.
> explicitly loaded cache groups: the application preloads data into the cache tables from the database using a load cache group operation. .
> dynamic cache groups: cache instances are automatically loaded into the IMDB Cache from the database when the application references cache instances that are not already in the IMDB Cache. The use of dynamic cache groups is typically coupled with least recently used (LRU) aging.

Keep a cache group synchronized with the corresponding data in the Oracle tables:
> Autorefresh
* incremental autorefresh: updates only records that have been modified since the last refresh.
best when updated often, but only a few rows are changed with each update.
* full autorefresh operation, refreshes the entire cache group at specified time intervals.
best if table is updated only once a day and many rows are changed.
> Manual refresh
best if the application logic knows when the refresh should happen.

Aging:
Records can be automatically aged out, usage-based or time-based.

Passthrough feature
checks whether the SQL statement can be handled locally by the cached tables in the IMDB Cache or if it must be redirected to the database.

Labels: ,