Configure hierarchical multi-document management

This guide describes how Fonto can work with multiple documents at once.Fonto allows authors to edit multiple documents simultaneously. Some schema's allow documents that refer to other documents in a manner that describes a hierarchy. To support working with such documents, Fonto may be configured to automatically build and maintain the corresponding hierarchy when these documents are loaded and/or edited.

An example of such documents are DITA maps. Support for DITA maps is included in the fontoxml-dita add-on and enabled automatically if the add-on is used.

Loading, unloading, reloading and retrying - the loading strategy

The first part of implementing your own multi-document management implementation is creating a loading strategy. This consists of a number of callbacks that Fonto will use when loading, unloading and reloading documents. Configure this behaviour using the setLoadingStrategy method on InitialDocumentsManager.

Older releases of Fonto did not support the full set of callbacks, and allowed callbacks to be provided as separate arguments. To avoid breaking such implementations, these forms continue to work in the current version. However, omitting the optional callbacks causes related functionality to be disabled. For instance, if the retryLoadingDocumentForHierarchyNode callback is omitted, Fonto will remove the "Reload" options that would otherwise be shown for documents that failed to load.

We recommend that existing implementations that use custom hierarchical multi-document management implement any new callbacks added when upgrading to a new release.

Loading initial documents

The loadInitialDocuments handler is used to scan a newly loaded document, load additional documents and build a hierarchy. The default implementation is a good starting point for writing your own:

loadInitialDocuments.js
import documentCapabilitiesManager from 'fontoxml-documents/src/documentCapabilitiesManager.js';
import documentLoader from 'fontoxml-remote-documents/src/documentLoader.js';
import DocumentReference from 'fontoxml-documents/src/DocumentReference.js';
import documentsHierarchy from 'fontoxml-documents/src/documentsHierarchy.js';
import DocumentsHierarchyNode from 'fontoxml-documents/src/DocumentsHierarchyNode.js';

export default function loadInitialDocuments(remoteDocumentIds) {
	return Promise.all(
		remoteDocumentIds.map(function(remoteDocumentId) {
			// Make a document hierarchy node
			var documentReference = new DocumentReference();
			var newHierarchyNode = new DocumentsHierarchyNode(documentReference);

			// If the 'map' document is purely used for hierarchy purposes,
			// the isVisible property may be set to false. This prevents it from being shown in Fonto
			documentReference.setTarget(remoteDocumentId);

			// Append it to the root hierachy
			documentsHierarchy.addChild(newHierarchyNode);

			// Actually load the document
			return documentLoader
				.loadDocument(remoteDocumentId)
				.then(function(documentId) {
					// It is loaded, finish the hierarchy node
					documentReference.setInstance(documentId);
					// Trigger the notifier to cause updates in the view.
					// This will cause the new document to be visible in the content view
					documentsHierarchy.hierarchyChangedNotifier.executeCallbacks();

					// Remember to set the capability for every loaded document. We only have to set it to operable,
					//   locking may cause it to turn editable if we have acquired a lock.
					// The document should be operable before we can start processing it because schema validation
					// is only performed for documents at or above this level.
					return documentCapabilitiesManager
						.ensureCapabilityLevel(documentId, 'operable')
						.then(function() {
							return documentId;
						});
				})
				.then(function() {
					// As we now have a document(Id), this is an ideal place to scan the document
					//   and load additional documents.
					// Per found document, perform the following steps:
					// 1. Make a new DocumentReference for the new child document. Use the sourceDocumentId and the
					//    sourceNodeId parameters of the constructor to link it to the node referring this new child document.
					//    This will cause the contextual operations of said sourceNode to be included in the structure view.
					// 2. Make a new DocumentsHierarchyNode, passing the documentReference as the constructor parameter
					// 3. Set the target of the DocumentReference to the newly found remote document id,
					//    using either the 'setTarget' or 'setRelatedDocumentTarget' methods on DocumentReference,
					//    depending on whether the CMS requires a referrer document id
					// 4. Use the addChild method of the parent hierarchy node to include the new node into the hierarchy
					// 5. Trigger the hierarchyChangeNotifier by calling the executeCallbacks method,
					//    this triggers a rerender of the document in a loading state
					// 6. Load the document by calling DocumentLoader#loadDocument or DocumentLoader#loadRelatedDocument,
					//    depending on whether a referrer id is required by the CMS.
					//    Make sure this corresponds to the choice made in #3.
					// 7. When that is done, use the setInstance of the new child hierarchy node to finish the loading
					// 7.a   If the loading fails, use the setDocumentError method on DocumentsManager to set a loading error
					//       for this document. This influences the visualization of this document.
					// 8. Trigger the hierarchyChangeNotifier by calling the executeCallbacks method again,
					//    this triggers a rerender of the document in the loading state
					// 9. Bundle and return all promises of all children. This causes the spinner to disappear after
					//     all documents are loaded and made operable, instead of when the first one is done loading.
				})
				.then(function() {
					return null;
				})
				.catch(function(error) {
					// Collect errors rather than failing the entire promise.all
					return error;
				});
		})
	).then(function(errors) {
		// Only fail if every document failed to load.
		// Failing here causes a permanent error which will prevent the editor from loading
		//   and show an author the error screen instead.
		if (
			errors.length > 0 &&
			errors.every(function(error) {
				return !!error;
			})
		) {
			// This is not required. Fonto allows loading no documents, or loading only errored documents.
			// This causes Fonto to display an error placeholder for the root documents.
			console.error('Unable to load any initial document', errors);
			throw errors[0];
		}
	});
}

Refer to the API documentation of DocumentsHierarchy, DocumentsHierarchyNodeDocumentReferenceDocumentLoader and DocumentsManager for more information on how to use these types.

Unloading all documents

Since it is the responsibility of the application to construct a document hierarchy, it is also the responsibility of the application to deconstruct it. The default implementation of the unloadAllDocuments handler is a simple call to DocumentsManager#removeAllDocuments, applications which build a hierarchy should also at least call DocumentsHierarchy#clear. This callback will not be executed if the author decides to close the browser in which Fonto is running.

Unloading a single document

The unloadDocument handler was introduced in Fonto 7.6 in order to support the unload-document operation as well as automatic unloading for just-in-time loaded documents (see below). It should unload the given document using DocumentsManager#removeDocument and update any references in the hierarchy accordingly. The easiest way to accomplish this is simply rebuilding the hierarchy, taking care to not load unloadable documents automatically. More information on such lazy loading strategies and just-in-time loading is given below.

Reloading a document

The reloadDocument handler is called when a document is reloaded. This works similar to the initial load, but the hierarchy related to the reloaded document must be reconstructed too.

The default implementation is a good starting point for writing your own. For more complicated scenarios, simply re-building the hierarchy after reloading the specific document can be a valid approach as well.

reloadDocument.js
import documentCapabilitiesManager from 'fontoxml-documents/src/documentCapabilitiesManager.js';
import documentLoader from 'fontoxml-remote-documents/src/documentLoader.js';
import documentsHierarchy from 'fontoxml-documents/src/documentsHierarchy.js';
import documentsManager from 'fontoxml-documents/src/documentsManager.js';

export default function reloadDocument(documentId) {
	var remoteDocumentId = documentsManager.getRemoteDocumentId(documentId);

	var hierarchyNodesForDocument = documentsHierarchy.findAll(function(hierarchyNode) {
		return (
			hierarchyNode.documentReference &&
			hierarchyNode.documentReference.documentId === documentId
		);
	});
	hierarchyNodesForDocument.forEach(function(hierarchyNode) {
		// Unload the hierarchy node
		hierarchyNode.documentReference.setInstance(null);

		// The hierarchy should be unloaded recursively by looping over the children,
		//   removing them using documentsManager.removeDocument and calling removeAllChildren if the document contained references.
		// If the application holds some extra state for these documents, that must be destroyed as well.
	});

	// Remove the root document
	documentsManager.removeDocument(documentId);

	// Signal the update to the view.
	// This causes the content view to show loading spinners for all documents which have been unloaded
	documentsHierarchy.hierarchyChangedNotifier.executeCallbacks();

	return documentLoader
		.loadDocument(remoteDocumentId)
		.catch(function(error) {
			// This causes the content view to show loading errors for all documents which have an error state
			documentsHierarchy.hierarchyChangedNotifier.executeCallbacks();

			throw error;
		})
		.then(function(newDocumentId) {
			// Ensure this document is at least operable. Again, locking may immediately set it to editable if locks allow it.
			return documentCapabilitiesManager
				.ensureCapabilityLevel(newDocumentId, 'operable')
				.then(function() {
					// Restore instances, assuming no traversal root node IDs are used
					// (which matches the loadInitialDocuments implementation).
					hierarchyNodesForDocument.forEach(function(hierarchyNode) {
						hierarchyNode.documentReference.setInstance(newDocumentId);
					});

					// Trigger a rerender
					documentsHierarchy.hierarchyChangedNotifier.executeCallbacks();

					// TODO: Because this new document is different from the old one, the references may be changed.
					// This is the place where new documents should be loaded if this document references other new documents, and old documents may be unloaded.
					// Use the same steps as described in the initial loading part, but try to unload only documents that have been removed.
					// Use the removeDocument method on DocumentsManager to unload these documents and update the hierarchy accordingly.

					return newDocumentId;
				});
		});
}

Retrying to load a document that encountered an error

Documents may fail to load for various reasons, ranging from validity issues to temporary connectivity issues. Fonto can offer the option to users to retry loading such documents, as a more efficient alternative to restarting the entire editor session. To enable this for custom implementations, supply the retryLoadingDocumentForHierarchyNode handler to InitialDocumentsManager#setLoadingStrategy. This callback will be called to retry loading, and is given a DocumentsHierarchyNode representing the reference to the document. Loading the document should proceed similar to the initial load, extending (but not replacing) the existing hierarchy if the document loads successfully.

Again, the default implementation is a good starting point for writing your own:

retryLoadingDocumentForHierarchyNode.js
import documentCapabilitiesManager from 'fontoxml-documents/src/documentCapabilitiesManager.js';
import documentLoader from 'fontoxml-remote-documents/src/documentLoader.js';
import documentsHierarchy from 'fontoxml-documents/src/documentsHierarchy.js';

export default function retryLoadingDocumentForHierarchyNode(hierarchyNode) {
	var documentReference = hierarchyNode.documentReference;
	if (!documentReference || documentReference.documentId) {
		return Promise.resolve();
	}

	// A DocumentReference can either be absolute (remoteDocumentId) or relative (relativeUrl and referrerId),
	// use the appropriate loading method.
	var loadingPromise = documentReference.remoteDocumentId
		? documentLoader.loadDocument(documentReference.remoteDocumentId)
		: documentLoader.loadRelatedDocument(
				documentReference.relativeUrl,
				documentReference.referrerId
		  );

	// Ensure the loading state is shown in the editor
	documentsHierarchy.hierarchyChangedNotifier.executeCallbacks();

	return loadingPromise
		.then(function(documentId) {
			// Ensure this document is at least operable. Again, locking may immediately set it to editable if locks allow it.
			return documentCapabilitiesManager.ensureCapabilityLevel(documentId, 'operable');
		})
		.then(
			function(documentId) {
				// Link the hierarchy node to the newly loaded document and signal a change to update the UI
				documentReference.setInstance(documentId);

				// TODO: If the newly loaded document contributes nodes to the hierarchy, this is a good point to create and add those.
				// Alternatively, implementations could simply rebuild the entire hierarchy by calling a function such as refreshHierarchy defined below.

				documentsHierarchy.hierarchyChangedNotifier.executeCallbacks();
			},
			function(_error) {
				// The error will be shown in the hierarchy automatically if we trigger an update
				documentsHierarchy.hierarchyChangedNotifier.executeCallbacks();
			}
		);
}

Changing the hierarchy

The DocumentsHierarchy must be kept up-to-date when adding and removing documents or when moving them around. It's best to implement these as normal operations that modify the XML defining the hierarchy structure, followed by a step that refreshes the hierarchy to match the changes.

Refreshing the hierarchy

The hierarchy should be refreshed manually after an update has happened. An action should be used for this. This action can rebuild the hierarchy by walking over the DocumentsHierarchy and scanning the documents for references to other documents.

While it is possible to perform a minimal edit on the hierarchy, it will usually not impact performance for the better. Even when the operation recreates the entire hierarchy, Fonto will attempt to update the UI minimally and re-assign HierarchyNodeIds according to their source node and targets referenced by the hierarchy's DocumentReference properties as well as their position in the previous hierarchy.

Your implementation should take care to unload documents that are no longer referenced by the new hierarchy. A good starting point is given below:

refreshHierarchy.js
function getDocumentIdsReferencedInHierarchy (documentsHierarchy) {
	// Build a list of unique document instances currently referenced by the hierarchy
	var referencedDocumentIdsSet = Object.create(null);
	(function collectReferencedDocumentsIds (hierarchyNodes) {
		for (var i = 0, l = hierarchyNodes.length; i < l; ++i) {
			var hierarchyNode = hierarchyNodes[i],
				documentId = hierarchyNode.documentReference &&
					hierarchyNode.documentReference.documentId;
			if (documentId) {
				referencedDocumentIdsSet[documentId] = true;
			}
			collectReferencedDocumentsIds(hierarchyNode.children);
		}
	})(documentsHierarchy.children);


	return Object.keys(referencedDocumentIdsSet);
}


function refreshHierarchy () {
	// Save the old document ids, so that we can clean up which documents have been removed.
	var documentIdsInOldHierarchy = getDocumentIdsReferencedInHierarchy(documentsHierarchy);

	// Starting from the top-level hierarchy nodes, reload the entire hierarchy
	var oldHierarchyChildren = documentsHierarchy.children.concat();
	documentsHierarchy.clear();
	var documentsLoading = oldHierarchyChildren.map(function (hierarchyNode) {
		// Create a new hierarchy node to replace the old one
		var newHierarchyNode = new DocumentsHierarchyNode(hierarchyNode.documentReference);
		documentsHierarchy.addChild(newHierarchyNode);

		// Reload the document - the loader will just return the instance IDs for any documents still present in
		// the DocumentsManager, so this will only cause CMS requests for new documents
		return documentLoader.loadDocument(newHierarchyNode.documentReference.remoteDocumentId)
			.then(function (documentId) {
				// The document can now be processed and scanned for new document references. 
				// This processing should be similar to how documents are loaded initially.
				return processDitaDocument(newHierarchyNode, documentId);
			});
	});

	return Promise.all(documentsLoading)
		.then(function () {
			// Remove documents which were referenced by the previous hierarchy but not by the new one
			var documentIdsInNewHierarchy = getDocumentIdsReferencedInHierarchy(documentsHierarchy);
			for (var i = 0, l = documentIdsInOldHierarchy.length; i < l; ++i) {
				var documentId = documentIdsInOldHierarchy[i];
				if (!documentIdsInNewHierarchy.includes(documentId)) {
					// Document is no longer referenced, unload it
					documentsManager.removeDocument(documentId);
				}
			}

			// We are done, signal an update to the UI
			documentsHierarchy.hierarchyChangedNotifier.executeCallbacks();

			// Resolve to nothing
		});
}

Mutating the XML defining the hierarchy

The easiest mutations are those that are confined to a single document. Moving a document in the hierarchy can usually be translated directly to moving its referencing element. Because of this, the full toolbox of operations may be used. Useful operations are:

The indent and outdent operations are usually written as custom mutations.  Use unsafeMoveNodes to move elements which may contain positions (such as the selection, comments from fontoxml-annotations or change markers from fontoxml-track-changes). If nodes are removed from the document using Blueprint#removeChild, or moved directly using the appendChild or insertBefore methods, any positions contained within them will collapse to where the element was located, which may cause unintended effects for tracked changes, comments or other functionality.

When using the undo operation to revert a hierarchy affecting operation, the DOM will be reverted but the hierarchy will not. When performing a mutation which affects the hierarchy, remember to clear the undo stack. This can be done with the clearUndoStackForDocument action.

In some configurations - for example when using DITA maps - the same document may be included multiple times in the hierarchy. To disambiguate these cases, any operation invoked from the structure view is automatically passed a hierarchyNodeId step data property with the ID of the hierarchy node from which the operation was invoked. When invoking operations based on the cursor position, for example using a button in the toolbar, use SelectionManager#getFocusedHierarchyNode(Id) to obtain the (ID of the) corresponding hierarchy node.

Cross-document mutations

In some cases, such as in hierarchies consisting of multiple DITA map documents, indent and outdent may need to move elements across document boundaries. The recommended approach for these is the following:

  1. use a transform to determine the affected nodes and documents,
  2. use the acquire-document-locks operation to ensure locks have been acquired for all affected documents
  3. use a custom mutation to move the nodes to their new positions,
  4. Clear the undo stack using the clearUndoStackForDocument action.

Previous versions of this guide defined an approach that involved serializing and parsing nodes moved between documents. Starting with release 7.5, this is no longer required nor recommended as nodes may now simply be moved directly between documents.

Don't render the references

In Fonto, the elements representing references to other documents are usually of the configureAsRemoved family. When these nodes are rendered elsewhere, for instance in DITA maps where the map is itself rendered as a separate sheet frame, the visibleChildSelectorOrNodeSpec property of the configureAsSheetFrame family should be used to prevent hierarchy children being rendered in their parent's sheet frame. 

The referencing elements and attributes that define the hierarchy should be edited using special, dedicated operations that include refreshing the hierarchy. Use of the fontoxml-attributes-editor add-on or cursor-based editing to manipulate the hierarchy is not supported.

Lazy hierarchies and just-in-time loading

When dealing with hierarchies consisting of large amounts of documents, loading all documents referenced in the hierarchy may not be feasible. To support such hierarchies, the loading strategy implementation can be made lazy. In a lazy implementation, the documents hierarchy may reference documents that have not yet been loaded.

For content stored in very large, single documents, we recommend a server-side chunking approach in which such content is presented to Fonto as if it were stored in a number of smaller "chunk" documents. This not only allows better performance via lazy and just-in-time loading, but can also potentially improve collaboration by allowing authors to simultaneously edit different chunks.

Just-in-time loading

Starting with Fonto 7.6, Fonto will automatically load any document that is referenced by the hierarchy but is not yet loading or loaded as soon as they are scrolled into view. In order to conserve memory and improve performance, Fonto may also automatically unload any such just-in-time loaded document again when it leaves and remains outside the viewport. In order to support navigating back and forth inside a limited set of documents without constant loading / unloading, Fonto will automatically keep a number of the most recently viewed documents loaded even when they are outside of the viewport.

Just-in-time loading will never automatically unload documents after they have been modified, nor will it attempt to unload a document that has been loaded manually. This includes documents loaded initially by the current loading strategy, as well as those loaded by any operations.

Just-in-time loading is enabled automatically if the loading strategy is lazy, and currently has no configuration.

If you're using a custom loading strategy, make sure to implement the optional unloadDocument loading strategy callback. Without this callback documents can never be unloaded automatically.

Considerations

A lazy loading strategy obviously implies that not all documents in the hierarchy will be loaded at all times. This has some implications for certain features:

  • Find & Replace only considers loaded content. When using lazy loading, we recommend disabling the fontoxml-find-and-replace add-on and implementing a custom CMS-based search.
  • Comments, tracked changes and any other cross-document lists will only show results within loaded content. Again, the recommendation is to disable such features or implement similar functionality using custom CMS support.
  • Fonto can not show content from documents that have not been loaded. This includes the titles shown in the outline view for any unloaded documents referenced from the hierarchy. We recommend including document titles in the referencing element, and configuring the titleQuery and icon properties using configureStructureViewItemProperties.
  • As Fonto can only build the documents hierarchy from loaded documents, chunking approaches that mix hierarchy and content do not work as well as those that use dedicated documents to express only the hierarchy. In the latter case, it is easy to limit loading to those hierarchy-defining documents and have the content load in lazily.
  • The JIT loading feature does not work with reference resolving for documents. For DITA this means that `map-manager-use-permanent-references` must be set to `false`.

These considerations can be solved, but require careful design since this depends on precise interaction between Fonto Editor and the CMS.

Lazy loading and DITA

For applications using the fontoxml-dita add-on, lazy loading can be enabled by setting the map-manager-automatically-load-topicrefs configuration value to false. If more control is required, applications may subscribe to the  initialDocumentsLoadedNotifier and use the load-document-for-hierarchy-node to manually load additional documents referenced in the current hierarchy.

The <navtitle> element may be used to provide title content for a <topicref> for which the target document is unloaded. The recommended approach is to have the CMS include this information when the map document is loaded. Consider using configuration similar to the following:

configureProperties(
    sxModule,
    'self::*[fonto:dita-class(., "map/topicref") and topicmeta/navtitle]',
    { titleQuery: 'topicmeta/navtitle' }
);

configureStructureViewItemProperties(
    sxModule,
    'self::*[fonto:dita-class(., "map/topicref")]',
    { icon: 'file-text-o' }
);

configureStructureViewItemProperties(
    sxModule,
    'self::*[fonto:dita-class(., "map/mapref")]',
    { icon: 'folder-open-o' }
);

Lazy loading and custom loading strategies

The key to implementing lazy loading for a custom loading strategy is to pass the onlyFromCache option to DocumentLoader#loadDocument for any document referenced from the root document that should be loaded lazily. Passing this flag will cause the resulting promise to resolve to undefined if the document had not already been loaded. This allows the same hierarchy-building code to be used both for initial construction as well as for any rebuilds of the hierarchy, without needing to keep track of which documents are loaded at all times. For documents that should be loaded automatically, do not pass onlyFromCache (or set it to false). For example, you may want to automatically load all documents (e.g., DITA maps) that determine the hierarchy structure, but not leaf documents (e.g., DITA topics).

Make sure to add code to handle the possible undefined value returned from loadDocument if onlyFromCache is set. Passing this to functions that expect an actual DocumentId will cause errors.

Do create a DocumentsHierarchyNode with a DocumentReference and set its target (and source node, where applicable). Only call its setInstance method if the loadDocument promise resolves with an actual DocumentId. Any other document references will remain idle until a load is triggered. Just-in-time loading is enabled automatically.