Oracle Coherence does not only works as a key-value paired distributed cache. It does also support distributed data processing (not exactly like map-reduce but it works actually like parallel processing). Also, it can read-through and write-behind Oracle database. It sounds like a perfect solution as our primary domain object storage.

We have about 240GB memory for storage (20 server x 3 JVM x 4GB heap) storage nodes. Due to the number of servers, we decide to have zero backup count. In this scenario, when a node fails, we will lots the data that stored in the node. We assume as long as we have the Oracle database as the permanent storage (thru write behind), we can always read through the missing data.

During our performance testing in the staging environment, Coherence does not performance if we try to random access (add, query, and delete at the same time). To be worse, for each new search index will consume 10-15% more storage. It is really expensive to use RAM for storage!!!

Write behind does not working at all due to our data throughput. Since we have 60 nodes, we have 60 x N JDBC connections. Oracle database basically cannot keep handle all the insert, update, merge, and delete at the same time. The database latency causes the storage nodes evicted from the data grid due to timeout.

After 10 months of development, we decided to get rid of Coherence and use Oracle database directly. Oracle 10g performs pretty good comparing to our legacy system. However, it still cannot provide the performance that we need. We are now looking into other open sourced NOSQL solution that could provide us memory/disk based storage plus Map-Reduced. The current candidates are Cassandra+Hadoop or MongoDB. Cassandra doesn’t support multiple keys indexing but version 0.6 works very well with Hadoop integration (although I have learned from forums that Cassandra has performance issue during high traffic of data loading). MongoDB is very fast and support multiple key indexing. However, the M-R solution comes from MongoDB only supports single thread.

SQl or NOSQL? Still have no answer after 12+ months of development.

According to Wikipedia,

Cloud computing is a style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet . Users need not have knowledge of, expertise in, or control over the technology infrastructure in the “cloud” that supports them.

According to Berkeley RAD Lab,

The services themselves have long been referred to as Software as a Service (SaaS). The datacenter hardware and software is what we will call a Cloud. When a Cloud is made available in a pay-as-you-go manner to the general public, we call it a Public Cloud; the service being sold is Utility Computing. We use the term Private Cloud to refer to internal datacenters of a business or other organization, not made available to the general public. Thus, Cloud Computing is the sum of SaaS and Utility Computing, but does not include Private Clouds.

Ok, so what extractly is cloud computing?  I’m sure you can figure it out by searching from the web so I’m not going to pretend as an expert.  In general, popular cloud computing services are services from Google (such as Gmail, Google Reader, Google Documents), Dropbox, Moobee, Photoshop.com, etc. (I can easily name over 50 popular cloud computing services)

When I start Firefox, I have 3 default pages need to open: Gmail, Google Reader, Facebook.  However, I have Remember the Milk displaying the todo list on the right hand side of the Gmail page.  Other than Firefox, I have DropBox, Spanning Sync, and MobileMe running in the background to help me pushing my documents, calendars, and contacts back to the clouds.  I also backup important files to MobileMe iDisk and Amazon S3.  When I receive documents within emails, I use Google Document to open them.  I also share documents with friends and co-workers using Google Document as well.

The more I wrote and the more I figure out I don’t use thick client as much as before (i.e. Microsoft Office).  95% of the thick clients are replaced by cloud computing services.  Of course, that’s why I can only carry a 8.9″ Acer Aspire One with me most of the time (instead of the 15″ Macbook Pro).

How about you?  How tie you and your cloud computing services?