Software Architecture for large-scale Meteor applications

March 18, 2018 0 Comments

Software Architecture for large-scale Meteor applications

 

 

This article consider the large-scale Javascript NodeJS/Meteor applications architecture and treat the case study of the framework KeplerJs.

I will look for understand how to structure scalable applications designed to greatly increase their complexity, facilitating contributions and organizing the source code in a sustainable way from today until the end of the universe.

arrows are the dependencies, green are the base packages and the plugin packages are yellow.

Kepler is a open source geosocial framework that allow to aggregate different geospatial data sources in a single application, the data sources are many and varied then the structuring of the code requires a care in the organization of the logic that implements each of them to give space to personalization and special cases of work. It is also an open source project then is need this logic is simple to understand, any developer who wants to contribute to the project should be able to do it in a few minutes without having to study long implementation and reference manuals.
One of the problems with using new frameworks (especially in javascript) lies in the fact that their learning curve is often too steep for the new contributors. In Kepler we tried to avoid this problem by avoiding adding too many new concepts specific of the framework, but working with some simple concepts already present in the minds of javascript programmers, both novices and experts.

Basically it use a package-based architecture meaning that the entirety of its codebase resides in /packages path, then each complex module or a group of modules in Kepler are organized in Meteor packages which can be added or removed at runtime.

In addition to this standard design adopted by many famouse Meteor applications Kepler implements a plugins mechanism that allows you to split different pluggable features into different packages, this option facilitates the work of developers that will be able to connect their services or include their customizations writing a minimum amount of code and using the common API and environment from base packages and plugins.

These packages are mandatory for a basic presentation of data and behavior of platform:

  • Lib includes 3rd party libraries and external meteor packages
  • i18n contains all languages for base packages
  • Core implement the base business logic
  • UI define the structure of User Interface
  • API implement a restful API (optional for some applications)

All this set of base packages can be included in a single hit by installing the meta-package Base probably your starting case study will need only this package.

In addition to the Base Packages these plugin packages includeed in the Kepler Application show content with a better look, allow the data entry by users and a centralized application management by the admin users.

  • Theme include custom CSS styles icons and images
  • Edit plugin to edit/remove places’s data
  • Admin plugin to administer the platform’s data/users

The Kepler plugins packages provide useful pluggable features for your Kepler application. A plugin package only need to make your own package depend on keplerjs:core or others plugins if needed, it is simply a standard Meteor package that contains a file plugin.js that defines the UI structure, custom settings and others configurations for the Kepler environment.
The dependencies between plugins are defined in the classic package.js file, as is the case for Meteor packages.

In Kepler each plugin should implement not redundant features without replicating it in other plugins, but favoring dependencies.

some of the main Kepler plugins and dependency relationships between them

In the diagram above we can observe some dependency relationships:

  • (Edit→Admin) the plugin Admin use Edit to editing and removing places by admins users
  • (Edit→OSM) the plugin OSM use Edit for creation of new places (imports) from OpenStreetmap
  • (OSM→Pois) the plugin Pois(Point Of Interests) use OSM to imports and filter OpenStreetmap and transform these into new Pois
  • (Routing →Pois) the plugin Pois use Routing to create the best paths that connect places to its neighboring Pois
  • (OSM→Tracks) the plugin Tracks use OSM to imports and filter OpenStreetmap Highways data
  • (Geoinfo→Tracks) the plugin Tracks use Geoinfo to imports and update tracks meta-data with geospatial info(Elevations from services)

A plugin.js file efines the UI structure, the templates and their placement rendered inside the User Interface, custom settings and others Configurations in the environment, similar in concept to package.js file of Meteor. The following is an example of a plugin definition:

K.Plugin({
name: 'pluginName',
templates: { /* where render the plugin templates /
navSidebar: 'navSidebar_pluginName',
panelProfile: 'panelProfile_pluginName',
panelUser: 'panelUser_pluginName',
popupPlace: 'popupPlace_pluginName'
},
schemas: {
place: { / extend base model place with additional fields /
fieldName: []
},
pluginModel: { / define new model of data /
updatedAt: '',
fieldId: ''
}
},
filters: {
placePanel: { / extend a default filter /
fields: {
fieldName: 1
}
},
pluginFilter: { / define new filter /
fields: {
fieldName: 1
}
}
},
settings: {
public: {
pluginName: {
/ here any custom settings /
}
}
}
});

Any basic configuration can be extended by plugins inside their plugin.js file. There is a list of them and their meaning:

  • Schemas defines structures for documents in the collections, can be extended by *Kepler plugins to host the plugin fields
  • Filters defines the fields exposed in the queries for pubblications and methods, the structure of this file is deliberately aligned to enhance the different levels of data privacy
  • Templates defines the templates in the User Interface where the plugins can extend the content with others templates/views
  • Settings contains the main default settings extended from Meteor.settings

Kepler implements a convenient mechanism to give plugins the ability to extend the platform’s basic UI structure. 
Using the dynamic template pluginsTemplate and register the plugin’s templates inside the plugin.js in the section templates.

Here an example of templates defined for the plugin keplerjs:share

K.Plugin({
name: 'share',
templates: {
panelPlace: ['panelPlaceshare'],
popupCursor: 'popupCursor
share'
}
});

Any plugin can be define one or many templates to includes inside default template placeholders: panelPlace, popupCursor.
The templates positioned inside the templates:
panelPlace(keplerjs-ui/client/views/panels/place.html)
popupCursor(keplerjs-ui/client/views/popups.html)
Through a helper called pluginsTemplate which takes care of rendering in a single position all the templates registered in all plugins, related to a certain placeholder, in this case panelPlace

<template name="panelPlace">
...
{{> pluginsTemplate name="panelPlace"}}
...
</template>

This is the complete list of default templates placeholders is defined in the Kepler module K.templates:
navSidebar, tabLocation, pageHome, panelSettings, panelProfile, panelPlace, panelUser, popupPlace, popupUser, popupCursor, popupGeojson, itemPlace, itemUser, footer, attribution

This article is work in progress…
UPDATES:
19 March 2018 (Plugins dependecies)


Tag cloud