Synchronization with Rhodes === As we've shown in the [Rhom section](/rhodes/rhom), adding synchronized data via [RhoConnect](/rhoconnect/introduction) to your Rhodes application is as simple as generating a model and enabling a `:sync` flag. This triggers the internal Rhodes sync system called the **`SyncEngine`** to synchronize data for the model and transparently handle bi-directional updates between the Rhodes application and the RhoConnect server. This section covers in detail how the `SyncEngine` works in Rhodes and how you can use its flexible APIs to build data-rich native applications. ## Sync Workflow The `SyncEngine` interacts with RhoConnect over http(s) using [JSON](http://www.json.org/) as a data exchange format. With the exception of [bulk sync](/rhoconnect/bulk-sync), pages of synchronized data, or "sync pages" as we will refer to them here, are sent as JSON from RhoConnect to the `SyncEngine`. Below is a simplified diagram of the `SyncEngine` workflow: This workflow consists of the following steps: * `SyncEngine` sends authentication request to RhoConnect via [`SyncEngine.login`](#syncengine-api). RhoConnect calls [`Application.authenticate`](/rhoconnect/authentication) with supplied credentials and returns `true` or `false`. * If this is a new client (i.e. fresh install or reset), the `SyncEngine` will initialize with RhoConnect: * It requests a new unique id (client id) from RhoConnect. This id will be referenced throughout the sync process. * It will register platform information with RhoConnect. If this is a [push-enabled application](/rhodes/device-caps#push-notifications) application, the `SyncEngine` will send additional information like device push pin. * `SyncEngine` requests sync pages from RhoConnect, one model(or [Rhom](/rhodes/rhom) model) at a time. The order the models are synchronized is determined by the model's [`:sync_priority`](/rhodes/rhom#property-bag), or determined automatically by the `SyncEngine`. ## Sync Authentication When you generate a Rhodes application, you'll notice there is an included directory called `app/Settings`. This contains a default `settings_controller.rb` and some views to manage authentication with [RhoConnect](/rhoconnect/introduction). ### `login` In `settings_controller.rb#do_login`, the `SyncEngine.login` method is called: :::ruby SyncEngine.login( @params['login'], @params['password'], url_for(:action => :login_callback) ) Here login is called with the `login` and `password` provided by the `login.erb` form. A `:login_callback` action is declared to handle the asynchronous result of the `SyncEngine.login` request. ### `login_callback` When `SyncEngine.login` completes, the callback declared is executed and receives parameters including success or failure and error messages (if any). :::ruby def login_callback error_code = @params['error_code'].to_i if error_code == 0 # run sync if we were successful WebView.navigate Rho::RhoConfig.options_path SyncEngine.dosync else if error_code == Rho::RhoError::ERR_CUSTOMSYNCSERVER @msg = @params['error_message'] end if not @msg or @msg.length == 0 @msg = Rho::RhoError.new(error_code).message end WebView.navigate( url_for(:action => :login, :query => {:msg => @msg}) ) end end This sample checks the login `error_code`, if it is `0`, perform a full sync and render the settings page. Otherwise, it sets up an error message and re-displays the login page with an error. ### `application.rb#on_sync_user_changed` If the `SyncEngine` already knows about a logged-in user and a new user logs in, then the `on_sync_user_changed` hook is called (if it exists) before the `login_callback`. This is useful, for example, if you want to re-initialize personalized settings for a new user. :::ruby require 'rho/rhoapplication' class AppApplication < Rho::RhoApplication def initialize super end def on_sync_user_changed super MyCoolApp.reset_user_preferences! end end **NOTE: If `on_sync_user_changed`, data for all sync-enabled models will be removed. To remove data for all local models as well:** :::ruby def on_sync_user_changed super Rhom::Rhom.database_local_reset end Other auth-related methods are described in the [`SyncEngine` API section](/rhodes/synchronization#syncengine-api). ## Notifications The `SyncEngine` system uses notifications to provide information about the sync process to a Rhodes application. Notifications can be setup once for the duration of runtime or each time a sync is triggered. One a sync is processing for a model, notifications are called with parameters containing sync process state. Your application can use this information to display different wait pages, progress bars, etc. To set a notification for a model, you can use the following method: :::ruby SyncEngine.set_notification( Account.get_source_id, url_for(:action => :sync_notify), "sync_complete=true" ) Which is the same as: :::ruby Account.set_notification( url_for(:action => :sync_notify), "sync_complete=true" ) In this example, once the sync process for the `Account` model is complete, the view will be directed to the `sync_notify` action (with params 'sync_complete=true') if user is on the same page. **NOTE: In these examples, after the sync is complete the notifications are removed.** You can also set a notification for all models: :::ruby SyncEngine.set_notification( -1, url_for(:action => :sync_notify), "sync_complete=true" ) **NOTE: This notification will not be removed automatically.** ### Notification Parameters When the notification is called, it will receive a variable called `@params`, just like a normal Rhodes controller action. #### Common Parameters These parameters are included in all notifications. * `@params["source_id"]` - The id of the current model that is synchronizing. * `@params["source_name"]` - Name of the model (i.e. "Product") * `@params["sync_type"]` - Type of sync used for this model: "incremental" or "bulk" * `@params["status"]` - Status of the current sync process: "in_progress", "error", "ok", "complete", "schema-changed" #### "in_progress" - incremental sync * `@params["total_count"]` - Total number of records that exist for this RhoConnect source. * `@params["processed_count"]` - Number of records included in the sync page. * `@params["cumulative_count"]` - Number of records the `SyncEngine` has processed so far for this source. #### "in_progress" - bulk sync * `@params["bulk_status"]` - The state of the bulk sync process: "start": when bulk sync start and when specific partition is start syncing "download": when client start downloading database from server "change_db": when client start applying new database "blobs": when client start downloading remote blob files "ok": when sync of partition finished without error "complete": when bulk sync finished for all partitions without errors * `@params["partition"]` - Current bulk sync partition. #### "error" * `@params["error_code"]` - HTTP response code of the RhoConnect server error: 401, 500, 404, etc. * `@params["error_message"]` - Response body (if any) * `@params["server_errors"]` - Hash of Type objects of RhoConnect adapter error (if exists): "login-error", "query-error", "create-error", "update-error", "delete-error", "logoff-error" For "login-error", "query-error", "logoff-error": Type object is hash contains 'message' from server: @params["server_errors"]["query-error"]['message'] For "create-error", "update-error", "delete-error": Type object is hash each containing an "object" as a key (that failed to create) and a corresponding "message" and "attributes": @params["server_errors"]["create-error"][object]['message'], @params["server_errors"]["create-error"][object]['attributes'] **NOTE: "create-error" has to be handled in sync callback. Otherwise sync will stop on this model. To fix create errors you should call Model.on_sync_create_error or SyncEngine.on_sync_create_error** #### "ok" * `@params["total_count"]` - Total number of records that exist for this RhoConnect source. * `@params["processed_count"]` - Number of records included in the last sync page. * `@params["cumulative_count"]` - Number of records the `SyncEngine` has processed so far for this source. #### "complete" This status returns only when the `SyncEngine` process is complete. #### "schema-changed" This status returns for bulk-sync models that use [`FixedSchema`](/rhom#fixed-schema) when the schema has changed in the RhoConnect server. **NOTE: In this scenario the sync callback should notify the user with a wait screen and start the bulk sync process.** ### Server error processing on client #### create-error has to be handled in sync callback. Otherwise sync will stop on this model. To fix create errors you should call Model.on_sync_create_error or SyncEngine.on_sync_create_error: :::ruby SyncEngine.on_sync_create_error( src_name, objects, action ) Model.on_sync_create_error( objects, action ) * objects - One or more error objects * action - May be :delete or :recreate. :delete just remove object from client, :recreate will push this object to server again at next sync. #### update-error If not handled, local modifications, which were failing on server, will never sync to server again. So sync will work fine, but nobody will know about these changes. :::ruby SyncEngine.on_sync_update_error( src_name, objects, action, rollback_objects = nil ) Model.on_sync_update_error( objects, action, rollback_objects = nil) * objects - One or more error objects * action - May be :retry or :rollback. :retry will push update object operation to server again at next sync, :rollback will write rollback_objects to client database. * rollback_objects - contains objects attributes before failed update and sends by server. should be specified for :rollback action. #### delete-error If not handled, local modifications, which were failing on server, will never sync to server again. So sync will work fine, but nobody will know about these changes. :::ruby SyncEngine.on_sync_delete_error( src_name, objects, action ) Model.on_sync_delete_error( objects, action ) * objects - One or more error objects * action - May be :retry - will push delete object operation to server again at next sync. For example: :::ruby SyncEngine.on_sync_create_error( @params['source_name'], @params['server_errors']['create-error'], :delete) SyncEngine.on_sync_update_error( @params['source_name'], @params['server_errors']['update-error'], :retry) SyncEngine.on_sync_update_error( @params['source_name'], @params['server_errors']['update-error'], :rollback, @params['server_errors']['update-rollback'] ) SyncEngine.on_sync_delete_error( @params['source_name'], @params['server_errors']['delete-error'], :retry) #### unknown-client error Unknown client error return by server after resetting server database, removing particular client id from database or any other cases when server cannot find client id(sync server unique id of device). Note that login session may still exist on server, so in this case client does not have to login again, just create new client id. Processing of this error contain 2 steps: * When unknown client error is come from server, client should call database_client_reset and start new sync, to register new client id: rho_error = Rho::RhoError.new(err_code) if err_code == Rho::RhoError::ERR_CUSTOMSYNCSERVER @msg = @params['error_message'] end @msg = rho_error.message unless @msg and @msg.length > 0 if rho_error.unknown_client?(@params['error_message']) Rhom::Rhom.database_client_reset SyncEngine.dosync end * If login session also deleted or expired on the server, then customer has to login again: rho_error = Rho::RhoError.new(err_code) if err_code == Rho::RhoError::ERR_CUSTOMSYNCSERVER @msg = @params['error_message'] end @msg = rho_error.message unless @msg and @msg.length > 0 if err_code == Rho::RhoError::ERR_UNATHORIZED WebView.navigate( url_for( :action => :login, :query => { :msg => "Server credentials expired!" } ) ) end ### Notification Example Here is a simple example of a sync notification method that uses some of the parameters described above: :::ruby def sync_notify status = @params['status'] ? @params['status'] : "" bulk_sync? = @params['sync_type'] == 'bulk' if status == "in_progress" # do nothing elsif status == "complete" or status == "ok" WebView.navigate Rho::RhoConfig.start_path elsif status == "error" if @params['server_errors'] && @params['server_errors']['create-error'] SyncEngine.on_sync_create_error( @params['source_name'], @params['server_errors']['create-error'], :delete) end err_code = @params['error_code'].to_i rho_error = Rho::RhoError.new(err_code) if err_code == Rho::RhoError::ERR_CUSTOMSYNCSERVER @msg = @params['error_message'] end @msg = rho_error.message unless @msg and @msg.length > 0 if rho_error.unknown_client?(@params['error_message']) Rhom::Rhom.database_client_reset SyncEngine.dosync elsif err_code == Rho::RhoError::ERR_UNATHORIZED WebView.navigate( url_for( :action => :login, :query => { :msg => "Server credentials expired!" } ) ) else WebView.navigate( url_for( :action => :err_sync, :query => { :msg => @msg } ) ) end end end **NOTE: If the view was updated using AJAX calls, this mechanism may not work correctly as the view location will not change from one AJAX call to another. Therefore, you might need to specify the `:controller` option in WebView.navigate.** ### Sync Object Notifications The `SyncEngine` can also send a notification when a specific object on the current page has been modified. This is useful if you have frequently-changing data like feeds or timelines in your application and want them to update without the user taking any action. To use object notifications, first set the notification callback in `application.rb#initialize`: :::ruby class AppApplication < Rho::RhoApplication def initialize super SyncEngine.set_objectnotify_url( url_for( :controller => "Product", :action => :sync_object_notify ) ) end end Next, in your controller action that displays the object(s), add the object notification by passing in a record or collection of records: :::ruby class ProductController < Rho::RhoController # GET /Product def index @products = Product.find(:all) add_objectnotify(@products) render end # ... def sync_object_notify #... do something with notification data ... # refresh the current page WebView.refresh # or call System.execute_js to call javascript function which will update list end end #### Object Notification Parameters The object notification callback receives three arrays of hashes: "deleted", "updated" and "created". Each hash contains values for the keys "object" and "source_id" so you can display which records were changed. ## Binary Data and Blob Sync Synchronizing images or binary objects between RhoConnect and the `SyncEngine` is declared by having a 'blob attribute' on the [Rhom model](/rhodes/rhom). Please see the [blob sync section](/rhoconnect/blob-sync) for more information. ## Filtering Datasets with Search If you have a large dataset in your backend service, you don't have to synchronize everything with the `SyncEngine`. Instead you can filter the synchronized dataset using the `SyncEngine`'s `search` function. Like everything else with the `SyncEngine`, `search` requires a defined callback which is executed when the `search` results are retrieved from RhoConnect. ### Using Search First, call `search` from your controller action: :::ruby def search page = @params['page'] || 0 page_size = @params['page_size'] || 10 Contact.search( :from => 'search', :search_params => { :FirstName => @params['FirstName'], :LastName => @params['LastName'], :Company => @params['Company'] }, :offset => page * page_size, :max_results => page_size, :callback => url_for(:action => :search_callback), :callback_param => "" ) render :action => :search_wait end Your callback might look like: :::ruby def search_callback status = @params["status"] if (status and status == "ok") WebView.navigate( url_for( :action => :show_page, :query => @params['search_params'] ) ) else render :action => :search_error end end **NOTE: Typically you want to forward the original search query `@params['search_params']` to your view that displays the results so you can perform the same query locally.** Next, the resulting action `:show_page` will be called. Here we demonstrate using Rhom's [advanced find query syntax](/rhodes/rhom#advanced-queries) since we are filtering a very large dataset: :::ruby def show_page @contacts = Contact.find( :all, :conditions => { { :func => 'LOWER', :name => 'FirstName', :op => 'LIKE' } => @params[:FirstName], { :func => 'LOWER', :name=>'LastName', :op=>'LIKE' } => @params[:LastName], { :func=>'LOWER', :name=>'Company', :op=>'LIKE' } => @params[:Company], }, :op => 'OR', :select => ['FirstName','LastName', 'Company'], :per_page => page_size, :offset => page * page_size ) render :action => :show_page end If you want to stop or cancel the search, return "stop" in your callback: :::ruby def search_callback if(status and status == 'ok') WebView.navigate( url_for :action => :show_page ) else 'stop' end end Finally, you will need to implement the `search` method in your source adapter. See the [RhoConnect search method](/rhoconnect/source-adapters#source-adapter-api) for more details. ## SyncEngine API Below is the full list of methods available on the `SyncEngine`: ### `login(login, password, callback)` Authenticates the user with RhoConnect. The callback will be executed when it is finished. See the [authentication section](/rhodes/synchronization#sync-authentication) for details. :::ruby SyncEngine.login( @params['login'], @params['password'], url_for(:action => :login_callback) ) ### `logout` Logout the user from the RhoConnect server. This removes the local user session. See the [authentication section](/rhodes/synchronization#sync-authentication) for details. :::ruby SyncEngine.logout ### `logged_in` Returns 1 if the `SyncEngine` currently has a user session, 0 if not. :::ruby if SyncEngine::logged_in == 1 render :action => :index else render :action => :login end ### `dosync(show_sync_status = true, query_params = "", sync_only_sources_with_local_changes = false )` Start the `SyncEngine` process and display an optional status popup (defaults to true). query_params will pass to sync server. sync_only_sources_with_local_changes indicates that only sources that have local changes will be synced. :::ruby SyncEngine.dosync(false) #=> no status popups are displayed SyncEngine.dosync(false, "param1=12¶m2=abc") #=> no status popups are displayed and parameters will pass to sync server SyncEngine.dosync(false, "", true) #=> no status popups are displayed and synchronization will be performed only for sources with local changes. ### `dosync_source(source_id_or_name, show_sync_status = true, query_params = "")` Star the `SyncEngine` process for a given source id or source name and display an optional status popup (defaults to true). query_params will pass to sync server :::ruby SyncEngine.dosync_source(Product.get_source_id.to_i, false) #sync by source id SyncEngine.dosync_source(Product.get_source_name, false) #sync by source name ### `lock_sync_mutex` Blocking call to wait for `SyncEngine` lock (useful for performing batch operations). :::ruby SyncEngine.lock_sync_mutex #... perform blocking tasks... SyncEngine.unlock_sync_mutex ### `unlock_sync_mutex` Release the acquired `SyncEngine` lock (make sure you do this if you call `lock_sync_mutex`!). ### `stop_sync` Stops any sync operations currently in progress. :::ruby SyncEngine.stop_sync #=> no callback is called ### `set_notification(source_id, callback_url, params = nil)` See the [sync notification section](/rhodes/synchronization#notifications). ### `set_notification(-1, callback_url, params = nil)` Set notification callback for all models. This callback is not removed after the sync process completes. See the [sync notification section](/rhodes/synchronization#notifications). ### `clear_notification(source_id)` Clears the sync notification for a given source id. :::ruby SyncEngine.clear_notification(Product.get_source_id) ### `on_sync_create_error( src_name, objects, action )` "create-error" has to be handled in sync callback. Otherwise sync will stop on this model. To fix create errors you should call Model.on_sync_create_error or SyncEngine.on_sync_create_error. :::ruby SyncEngine.on_sync_create_error( @params['source_name'], @params['server_errors']['create-error'], :delete) * objects - One or more error objects * action - May be :delete or :recreate. :delete just remove object from client, :recreate will push this object to server again at next sync. ### `on_sync_update_error( src_name, objects, action, rollback_objects = nil )` :::ruby SyncEngine.on_sync_update_error( @params['source_name'], @params['server_errors']['update-error'], :retry) SyncEngine.on_sync_update_error( @params['source_name'], @params['server_errors']['update-error'], :rollback, @params['server_errors']['update-rollback'] ) * objects - One or more error objects * action - May be :retry or :rollback. :retry will push update object operation to server again at next sync, :rollback will write rollback_objects to client database. * rollback_objects - contains objects attributes before failed update and sends by server. should be specified for :rollback action. ### `on_sync_delete_error( src_name, objects, action )` :::ruby SyncEngine.on_sync_delete_error( @params['source_name'], @params['server_errors']['delete-error'], :retry) * objects - One or more error objects * action - May be :retry - will push delete object operation to server again at next sync. ### `set_pollinterval(interval)` Update the `SyncEngine` poll interval. Setting this to 0 will disable polling-based sync. However, you may still use [push-based-sync](/rhoconnect/push). :::ruby SyncEngine.set_pollinterval(20)' #=> now polls every 20 seconds ### `set_syncserver(server_url)` Sets the RhoConnect server address and stores it in [`rhoconfig.txt`](/rhodes/configuration). :::ruby SyncEngine.set_syncserver("http://myapp.com/application") #=> don't forget the '/application' path ### `set_objectnotify_url(url)` See the [sync notification section](/rhodes/synchronization#notifications). ### `set_pagesize(size)` Set the sync page size for the `SyncEngine`. Default size is 2000. See [the `SyncEngine` workflow](/rhodes/synchronization#syncengine-workflow) for how this is used. :::ruby SyncEngine.set_pagesize(5000) ### `get_pagesize` Get the current sync page size for the `SyncEngine`. See [the `SyncEngine` workflow](/rhodes/synchronization#syncengine-workflow) for how this is used. :::ruby SyncEngine.get_pagesize #=> 2000 SyncEngine.set_pagesize(5000) SyncEngine.get_pagesize #=> 5000 ### `enable_status_popup(false)` Enable or disable show status popup. True by default for Blackberry, false for other platforms. :::ruby SyncEngine.enable_status_popup(true) ### `set_ssl_verify_peer(true)` Enable or disable verification of RhoConnect ssl certificates, true by default. :::ruby # using a self-signed cert SyncEngine.set_ssl_verify_peer(false) ### `get_user_name` Returns current username of the `SyncEngine` session if `logged_in` is true, otherwise returns the last logged in username. :::ruby SyncEngine.get_user_name #=> "testuser" ### `search(*args)` Call search on the RhoConnect application with given parameters. See the [search section](#filtering-datasets-with-search) for more details. :::ruby # :from Sets the RhoConnect path that records # will be fetched with (optional). # Default is 'search'. # # :search_params Hash containing key/value search items. # # :offset Starting record to be returned. # # :max_results Max number of records to be returned. # # :callback Callback to be executed after search # is completed. # # :callback_param (optional) Parameters passed to callback. # # :progress_step (optional) Define how often search callback # will be executed with 'in_progress' state. # :sync_changes (optional) - true or false(default). Define should client changes send to server before search. Contact.search( :from => 'search', :search_params => { :FirstName => @params['FirstName'], :LastName => @params['LastName'], :Company => @params['Company'] }, :offset => page * page_size, :max_results => page_size, :callback => url_for(:action => :search_callback), :callback_param => "", :sync_changes => false ) ### `search(*args) (multiple sources)` Call search on the RhoConnect application with multiple source names. This is useful if your `search` spans across multiple models. For example: :::ruby SyncEngine.search( :source_names => ['Product', 'Customer'], :from => 'search', :search_params => { :FirstName => @params['FirstName'], :LastName => @params['LastName'], :Company => @params['Company'] }, :offset => page * page_size, :max_results => page_size, :callback => url_for(:action => :search_callback), :callback_param => "", :sync_changes => false ) Parameters are the same as for ModelName.search with an additional parameter: * `:source_names` - Sends a list of source adapter names to RhoConnect to search across. ## SyncEngine AJAX API [Sync engine AJAX API](syncengine-ajax-api) has been implemented to provide access to low-level control on synchronization process right from plain HTML/javascript UI pages. It isn't intended to be used in every application. It requires deep knowledge of SyncEngine functionality and operations. It may broke your application if used improperly so use it with care please. ## Backround synchronization on iOS On iOS, if application is put to background, it will be suspended. To allow application finish sync after application goes to background, you can use 'finish_sync_in_background' parameter in [`rhoconfig.txt`](/rhodes/configuration). When this parameter is set to '1', if sync is active in the time of background transition ( e.g. started from app_deactivate handler ), application will not be suspended until sync is finished.