Architectural review and refinement

This page includes the details of the post-experiment refined architecture.

Deployment View

Source UMLet diagram

Thunderforce lives inside Mozilla Thunderbird as an extension that communicates with the remote Salesforce.com servers using the simple object access protocol (SOAP) and caches information locally using SQLite. A SOAP implementation and SQLite are both built into the Mozilla platform.

Note that this approach will limit the number of Thunderforce clients that can run simultaneously due to API call limits. Thunderforce will include features that limit the number of API calls made in addition to basic API call statistics, such as expected versus actual API calls per hour and the total number of API calls over the past 24 hours. Those statistics can be divided into your organization's daily API call limits to determine how many Thunderforce installations you can have active during the day.

A future extension to the deployment view can involve an optional network server that receives workflow outbound messages from Salesforce.com and broadcasts those changes to local clients to reduce the amount of polling that Thunderforce has to do. Such a server can also substantially increase Thunderforce's scalability as it can be deployed to more computers without drastically increasing the number of API calls performed in a day. Perhaps a generic, client-agnostic solution similar to this might get built in the future (note that this is not a promise of future functionality).

Runtime View

Source UMLet diagram

Thunderforce is largely structured using the model-view-controller pattern to present Salesforce.com data to the end-user using bidirectional interconnects. With Salesforce.com as the master record, changes from any part of the system are pushed down to the lowest layer, and notifications regarding that change propagate from the lowest layer to the top, which is the view.

Salesforce.com's AJAX Toolkit and Mozilla Thunderbird's interfaces differ in various ways. To resolve this impedance mismatch, the Thunderbird interface implementation layer performs the necessary translations, implementing Salesforce.com as regular Thunderbird objects in the process.

Above the model layer, Thunderforce implements a minimal set of XML user interface language (XUL) overlays to extend the view and JavaScript-based user interface behavior override code to extend the controller. With this strategy, most of Thunderforce's functionality lives within the model and leverages Thunderbird's built-in functionality as much as possible.

Module View

Salesforce.com data layer (model)

Source UMLet diagram

The diagram above depicts how the Salesforce.com data layer components relate to each other.

In general, the methods in the API interface are those that are in the Salesforce.com AJAX toolkit. The AJAX toolkit's implementation will, indeed, exist mostly unchanged as the thunderforce.sforce.Salesforce object.

To turn Salesforce.com into a bidirectional API with event notification, the BidirectionalAPIPoller will wrap the Salesforce object to implement the BidirectionalAPI interface. This permits future extensibility with both non-polling solutions such as workflow outbound messages and aggregators such as local Salesforce.com cache servers.

To promote performance, the Cache object will store the returned values of method calls into memory and, with the offline functionality, a SQLite database. When notifications of changes from the backing store arrive, the cached return values that might be affected by those changes are invalidated. To balance performance with memory, the cache will manage its use of memory based on call and time statistics.

• As an example of cache invalidation, a change to an account object invalidates that object, that object's ancestor objects, queries that involve that entity, and searches that involve that entity
• Queries and searches are special functions that might be handled intelligently in specific update cases to further improve cache performance

In the cache, a write-back flag is included to effectively switch between online mode and offline mode. When write-back caching is enabled, the only the cache is updated. The backing store is not notified of the cached changes until the write-back flag is turned off. When the write-back flag is off, the caching becomes write-through, meaning that the backing store is notified synchronously. Note that event generation is performed by the cache when write-back is enabled. When write-back is disabled, the backing store generates the events that the cache propagates to its listeners. The cache will not, however, propagate events that do not alter the cached state, regardless of whether or not write-back is enabled.

One point of current architectural ambiguity is how Thunderbird's undo/redo manager will interoperate with the cache. More research through an experiment will clarify this.

More investigation into how to leverage Mozilla's built-in cache manager is required. Ideally, the Thunderforce cache will not need to implement most of what it provides.

This architectural pattern permits the upper layers to primarily interact with the BidirectionalAPI interface. As a result, the Cache class can be implemented in the future to get an alpha version of Thunderforce ready as soon as possible.

The BidirectionalAPIPoller presents a unidirectional API as a BidirectionalAPI through periodic polling for changes. To reduce the API call impact, only those entities that the user subscribed to within Thunderbird will be polled (question: will summary fields also cause the last-modified timestamp to change and trigger replication (WFOM and getUpdated)?). The set of subscribed entities is also what the Thunderforce searches are limited to, so we don't have to care about those entities that we are not subscribed to.

• Each entity will have its own polling frequency that can be set in setEntityRefreshIntervals(). For now, these will likely be set to default values internally. Eventually, this can be dynamically adjusted based on how frequently that entity is changing. An exponential back-off can be used when that entity has not changed since the last check.
• For subscribed entities that have the replicateable attribute, the replication methods getUpdated() and getDeleted() will be used to reduce the number of API calls to query different entities. If the user does not have permissions to use those methods, the poller will fall back to using entity queries
• If a large amount of data changed, the poller will notify listeners that possibly the entire state of the Salesforce.com data changed and that they should re-query anything that they are interested in. The Cache object will invalidate its entire cache upon receiving that event

A class that collects API usage statistics is needed. It can probably extend APIDelegator and include methods for getting the average number of API calls per hour, the total number of API calls performed in the previous 24 hours, and other useful numbers.

In the implementation, the classes will likely be invoked with the following chain of delegates:

• Cache -> BidirectionalAPIPoller -> APIMetricsCollector -> Salesforce

Thunderbird interface implementation layer (model)

This section describes the classes and interfaces used in the Thunderbird interface implementation layer from a high level. Further experimentation and refinement is required to learn more about the Thunderbird interfaces and which ones are the right ones to use. Thunderbird includes a substantial amount of functionality in its base classes, so the experiments will also dig into which functionality to reuse and which functionality to implement ourselves.

• Account
  • class nsThunderforceProtocolInfo implements nsIMsgProtocolInfo
  • class nsThunderforceServer implements nsIMsgIncomingServer
• Address Book
• Extension
• File I/O
• Filters
• Folders
  • class nsSalesforceMsgFolder implements nsIMsgFolder, nsIMsgFolderNotificationService, nsIDBChangeListener, nsISupportsWeakReference, nsRDFResource, nsIUrlListener
• Offline