Maintenance and Migration
Last updated
Was this helpful?
Last updated
Was this helpful?
Once deployed into the customer environment, the Form.io Platform needs to be upgraded to the latest versions of the software at regular intervals. This does not happen automatically and requires a manual procedure. Please become familiar with the process.
Many cloud providers, such as AWS and Azure, offer their own methods for upgrading container deployments. Refer to the cloud provider's documentation for the container upgrade procedure. This section will describe how to upgrade a local deployment of the Form.io Platform.
Please note before upgrading that the Form.io Platform is a "stateless" system. Each of the containers provided by Form.io does not maintain any files, sessions, or any other data that would be lost between upgrades. It is a best practice that all non-development environments contain multiple container instances behind a load balancer. When the Form.io Platform is deployed in multiple instances behind a load balancer, each deployment is an ephemeral system that can be removed without any effect on the stability of the overall deployment. This allows updates to be performed in a "rolling" fashion where each container is removed from the load-balancer, updated, and then added back into the load balancer once the update is complete.
At times, servers can get in a bad state when multiple instances are behind a load balancer and software is upgraded synchronously. Perform software upgrades one instance at a time to avoid any potential issues.
When performing an upgrade, please be aware of the distinction between "major" and "minor" upgrades.
A major version upgrade is anytime the version number where the leftmost, "whole" number in the version number has been incremented. For example, updating a 6.10.0 server to version 7.1.0 is considered a major upgrade. Upgrading from 6.10.0 to 6.11.0 is considered a minor version upgrade. These semantic versions are positioned in a way to indicate the "risk" associated with the upgrade and should be taken into consideration. Here are the numbers and what they mean.
6.10.2 - The 2 here is considered the "patch" version. Increments of this version are considered patch releases and are the least risky versions. They only include minor bug fixes against the current minor release.
6.10.2 - The 10 here is considered the "minor" version. Increments of this version are considered minor releases, which include new minor versions of the renderer which could introduce new features, etc.
6.10.2 - The 6 here is considered the "major" version. Increments of this number are considered major releases. They only occur rarely (usually once per year) and include major refactoring and improvements to the platform. There may be some reverse compatibility breaks in these versions so migrations to new major versions must be done with great care.
For every release issued, maintaining reverse compatibility is the highest priority. It is VERY rare that any release will break reverse compatibility, especially within the Form JSON schema that is used to render forms. Because of this, it is generally safe to upgrade the software without concern that forms will "break". If any refactoring does occur, they will usually be paired with an update hook that will automatically update the database to the correct Schema version so that the behavior of the server is maintained. Please see for more explanation on the updated schema.
Before upgrading a server, it is important to create a database backup of the deployment so that it can be restored if any data is lost. While this is never expected or anticipated, it is a best practice to minimize downtime in the event of any errors or interruptions during the upgrade process.
The Form.io Platform only depends on MongoDB for any "state" that is stored within the server, so as long as it is backed up, the upgrade can continue. There are two ways to perform a backup:
Once the database has been backed up, update the Form.io API server.
To perform an update with a Docker container system, stop the currently running container, remove it, and then re-register the new version utilizing all of the same environment variables that were used when originally deploying. The following commands illustrate how this can be done on a per-instance basis to perform a manual upgrade within a single environment.
If the values that were used when originally deploying the container were not saved, use the following command to determine the values of these environment variables:
Once the values of these environment variables are recorded, upgrading is easily achieved with the following command:
After running this command, inspect the logs by typing the following:
For updates that require a database update, the output should appear similar the the following:
Ensure that the updates complete as expected. If any update fails, proceed with the following section to remediate the issue.
If any of the updates fail to execute, a series of steps that can be taken to help resolve the problem. To understand how to resolve the update failures, it is important to understand how the DB schema works and how to get the server operational once again.
When an update is started, the server first companes the code schema version with the version present in the database. The code schema version can be found by looking at the package.json file within the deployment codebase and noting the "schema" property of that file. This indicates the "DB schema" of the codebase. Compare this value with the value that is found within the database by running the following command:
Running this command within the database connection should return a result similar to the following:
This indicates that the database is currently on the 3.3.9 version and is not locked. If the "code" schema version were a larger version number than this, this would indicate that an update needs to be run.
If an update fails, the "isLocked" property is set to a timestamp, and the "version" is set to the last successful update.
In the event of a problem, first reset the "isLocked" flag to "null", to retry the update by doing the following:
After this, retry the update by restarting our docker container:
Once it has restarted, inspect the logs to see if it has moved past the "stuck" update. If it has not, then bump the version of the update by one patch version and then retry as follows. The following example command would be used if th4 server was stuck on version 3.3.7:
Then restart the server as follows:
In most cases, this will not have any ill effects. However, please reach out to Form.io Support so that we can provide you with the "manual" update script you had to skip so that are able to ensure all updates are applied cleanly to your deployment.
Upgrading a PDF Server can be done by using the following command.
The following steps describe how to clone a database to different servers. This document refers to the existing database as the "source" database/container and the database on the new server as the "destination."
In the destination Mongo database, determine the "mount" folder by running the following command:
Find the "mounts" for the database. They should look like the following:
Note the "Source" and "Destination" folders. Record them for future use.
SSH into the source MongoDB container using the following command:
Perform a mongodump of the database, and place it in the "Destination" folder noted above. The sample command below assumes the database is named form.io:
Exit the container (type "exit", then press enter) and then verify that the "archive" in the "Source" folder looks like the following:
Use the "scp" command to move this file from the Source machine to the Destination machine as follows:
SSH into the destination machine and ensure the Form.io Docker Containers are stopped.
Ensure that the Environment variables for these containers are the same as the source container environment variables.
On the destination machine, find the "Source" and "Destination" folders of the running Docker MongoDB container, similar to step #4.
Move the archive file into the "Source" folder of the destination Mongo container folder like so:
Using Bash to connect to the destination container:
Finally, restore the backup to the destination database.
When upgrading the version of MongoDB deployed to a container by pulling the latest
version, note the minimum supported version required by the featureCompatibilityVersion
parameter. If an attempt to deploy MongoDB returns a Feature Compatibility Version error, it is usually because the currently installed version is incompatible with the latest
's specified featureCompatibilityVersion
. In these cases, it may be necessary to upgrade in stages to meet the next supported version until reaching the desired version.
The following documentation describes how to perform different kinds of migrations within the Form.io platform. This documentation will refer to both Source and Destination projects, where the source is the project where to migrate FROM while the Destination is the project to migrate into.
Migrating from one project to another can easily be achieved with a combination of the Form.io Staging system as well as the Form.io CLI tool. The Staging system is used to migrate the Forms, Resources, and all Project level configurations. It is used to migrate everything EXCEPT Submissions and Settings.
To start, export the project of the Source Project using the staging interface. This is found by clicking on the Source Project in the Developer portal, and then clicking on Settings > Stage Versions.
For a complete migration, export the whole template. Click on the Export Template button. This will download a JSON file onto the local machine, to migrate to a new project.
The export JSON file on the local machine can be used to create a new project, or update an existing project:
To create a new Destination project:
Click on the Create Project button from the Developer portal homepage.
Click Upload A Project Template, and select the JSON file.
To update an existing Destination project:
Open the Destination project from the Developer Portal.
Navigate to Settings > Staging.
Click on the Import Template section.
Click Choose File and then select the template exported from the Source project. Then, click the Import Project Template button to complete the import.
This will clone the Forms, Resources, Roles, and Actions into this Destination Project.
Migrating submissions is accomplished using the Form.io CLI tool.
With the CLI tool now on the local machine, make sure that an API key has been configured for both the Source and Destination Projects. To do so, from the Developer Portal, go to the Settings > API Settings section and create an API key.
Once both the Source and Destination project have API keys, issue the following command from the local computer:
If you are migrating submissions from open source then https://SOURCE_PROJECT.form.io would instead be the domain of your open source server e.g. localhost:3001
This will copy all submissions from the Source project into the Destination project.
To migrate the project settings, use the Project API. The API calls that will be used and the general process are as follows:
Copy the Destination project API Keys. Record them for later use.
Copy the Source project settings, and set them as the Destination settings.
Copy the Destination project keys back into the Destination settings.
The following shell command performs all of the steps above, and can be used to migrate project settings from one project into another while maintaining the same API Keys for future migrations:
The Form.io platform can use the hosted Developer Portal (https://portal.form.io) to connect to remote deployed environments through the On-Premise Environments section within the project. This is useful to allow a remote environment to serve as an API-only interface for an application, while at the same time, managing that deployment through a hosted portal interface. In some cases, though, it may be neccessary to enable the Developer Portal directly within a remote environment. To do so, configured the following environment variables:
After the Developer Portal has been enabled, login to the Developer Portal by navigating to the root URL of the deployed API. Once logged in, any existing projects will not be visible. Do not worry, they are still available. The "owner" of these projects must be established in order to view them while logged in as the root user account. To do this, first connect to the MongoDB database, and then run the following command on that environment:
This command will set all of the projects to have the "owner" of the account that was created within the Portal Base project. Now the root user will see all of the existing projects within that environment, so that they can be managed accordingly.
Once this tool is installed, you follow these steps general steps:
Create a new project within Form.io.
Create an API Key within this project by going to the Project Settings > Stage Settings > API Settings.
Execute the following command to deploy the local project into Hosted Form.io:
Make sure you replace {PROJECTNAME}
, {srcAPIKEY}
, and {dstAPIKEY}
with your new Hosted Form.io project name (found in the API url), as well as the API key that was created in the second step above.
While there are several major changes with the 9.0.0 release, one of the primary goals for this release is to maintain an easy upgrade path from 8.x versions. There are several important points that have been implemented with the 9.0.0 release to ensure that the migration from 8.0.0 is a quick and easy transition.
No database upgrade scripts or schema changes With the 9.0.0 release, there are no schema changes or upgrade scripts that will be performed on deployments during an upgrade from 8.x to 9.0.0
No major Developer Portal, Formio.js Renderer, or Form Builder changes We encourage the upgrade to 9.0.0 to take advantage of the security enhancements and CVE resolutions as soon as possible. To enable an easy upgrade path, the first 9.0.0 version contains only the necessary upgrades to dependencies and libraries and is without any major changes to the Developer Portal Application, Formio.js Renderer, or Form Builder since 8.x.
Most of the changes for 9.0.0 pertain to security updates, performance improvements, major library dependency and runtime upgrades. The following is a detailed list of all major changes that have been made for the 9.0.0 release:
Form Component: Calculated Values w/ “Calculate on Server” enabled
Form Component: Custom Default Values
Form Component: Advanced Logic w/ Custom triggers or actions
Form Component: Custom Conditionals
Form Component: Custom Validations
Form Component: Select Available Items Validation
Form Actions: Email Action template rendering
Form Actions: Save Submission Transform
Form Actions: Custom Action Conditions
Form Actions: Webhook Action Transforms
Project Settings: Token Parse
This library replacement also required a refactor of the Form.io Server Side data processing system. Previously, the Javascript renderer, Formio.js, was leveraged as the mechanism to perform this validation within VM2, but this was no longer viable considering the level of protection surrounding evaluation contexts within Isolated VM.
get root
get component
get currentForm
get data
get parent
get dataValue (getter)
set dataValue (setter)
getValue()
setValue()
isEmpty()
get schema
get options
on()
off()
render()
redraw()
ready()
init()
destroy()
teardown()
attach()
detach()
build()
t(text)
sanitize(dirty)
renderString(template)
getComponent(path)
get submission
set submission
set form
get root
instance.root fields available:
data (submission data)
There are a few breaking changes with the 9.x upgrade you should be aware of.
Buttons are now included within the submission object
There are a few breaking changes with this upgrade you should be aware of:
Developer Portal Users
Admin accounts work as they do in 7.x where Admins can Create new Projects and Teams.
There is an automatic update hook upon upgrade to API 8.0.0 that affects the original user created with the environment variables ADMIN_EMAIL and ADMIN_PASS. The update hook migrates this original user from the User Resource to the Admin Resource in the Portal Base Project.
Any external scripts that assume the User resource to contain this user would need to be updated to point to the Admin resource.
Webhook Improvements
Payload Transformations
Modify the data payload being sent by the Webhook before it is sent using JavaScript. This allows you to alter the payload to map to any external service interfaces.
Proper Error Handling and Logging
The webhook action now features improved handling of errors and logging to ensure that there is visibility into how the webhook is operating when pointed to the 3rd party services.
Action Deprecation: Because of the Webhook Improvements, it is now possible to integrate with a wide variety of 3rd party services using nothing more than the Webhook action, resulting in several existing integration actions now being redundant. With this in mind, the following action items are being deprecated:
Deprecated actions have been removed from the Portal UI but will still function on forms where the action was previously configured
Actions: Office 365, Jira, Twillio, Hubspot
These actions will be removed in favor of the Webhook Action
Action: Reset Password
Dropbox integration
Our webhooks allow customers the flexibility to use whichever services they wish to integrate
Signature Relocation
There are also many new features included in this release designed to provide Form.io users with reliability, platform extensibility, and (more) ease of use. Please check out these new capabilities below:
Portal Admin Permissions
Portal Admins now have full visibility and access to all projects within the Form.io Developer Portal, regardless of who created them. This includes viewing all Environment Project Stages for the deployment, even if the parent Project does not reside within that specific environment.
Previously, any user with access to the Form.io Developer Portal could create their own projects, but these projects were only visible to their creators. This lack of visibility extended to Portal Admins, making it difficult to manage and control the number of active projects under a license.
To resolve this, we have introduced two distinct roles within the Form.io Developer Portal: Admins and Authenticated Users, each with specific project creation permissions and capabilities.
Form Building and Form Management:
Use Google Drive as storage for the file component.
Google Sheet action to create and update fields within your own Google Sheets.
Migration of Form Revisions
Form Revisions now migrate to destination stages during Stage Deployments. All references to revisions within nested form definitions are updated to include the correct revisions that are being transferred.
Two Factor Authentication
2FA can now be enabled for logging into the Developer portal as well as on a per-project basis to enable 2FA within your application.
Populate Resources and Forms from CSV
Form.io now enables an easy-to-use CSV Upload Feature within the Developer Portal to pre-populate Submissions against any Form or Resource with a single click of a button!
New Conditional Show & Hide
Form.io Security & Compliance Module:
Internal Process Improvements:
Form.io has automated over 90% of our testing suite to increase our agility and performance for our customers
Migrating your deployment from a 6.x version into a 7.x version is not complicated, but there are a few things that you should be aware of to ensure that your transition goes smoothly. The 7.x version of the Enterprise Server introduces a number of new features that you will certainly want to take advantage of, which are as follows.
Feature
Description
New Licensing System
Group Permission Levels
User Session Management
Audit Logging
Simplified PDF Setup
In addition to the 7.x Enterprise Server, this release also includes the 3.x Enterprise PDF server, which introduces a number of improvements to the management and configuration of how the PDF server is connected to the API Server.
Isomorphic Validation
The new Isomorphic validation utilizes the Form.io core renderer found @ https://github.com/formio/formio.js as the mechanism for validating submissions within the server logic. This ensures that any validation that occurs on the front end form, is the exact same validation that occurs within the server validation system.
Before you begin your migration, it is recommended that you first create complete backups of your existing environment. Once this is completed, you can now spin up a replica environment and point this environment to the SAME database as your current 6.x environment. This should not cause any problems because the 7.x upgrade does not introduce any changes to database schemas as well as does not perform any update hooks (as of version 7.0.0). Before this environment is launched, however, you will need to ensure that you change the following environment variables for this cloned deployment.
6.x variable
7.x variable
Action
PRIMARY
PORTAL_ENABLED
Change name. Same value.
LICENSE
Delete this environment variable
LICENSE_KEY
Set value as new license key
FORMIO_FILES_SERVER
PDF_SERVER
Change name. Same value.
2.x variable
3.x variable
Action
FORMIO_PROJECT
Delete this environment variable
FORMIO_PROJECT_TOKEN
Delete this environment variable
FORMIO_PDF_APIKEY
Delete this environment variable
FORMIO_PDF_PROJECT
Delete this environment variable
MONGO
Add same value as provided to enterprise server.
LICENSE_KEY
Set value as the new license key. Same as API Server
After your two environments are up and running, you can now perform necessary tests against your replica environment. Once it is determined that all features are working to your expectation, you can then flip a DNS switch over to your replica environment, making it become the new production, while at the same time keeping the old 6.x environment running as a fail safe if anything should happen (where you can then DNS switch back to the old environment).
One other thing that should be noted is that, in some cases, the License will erroneously register your "stage" projects within your deployment as "licensed" projects. This occurs because your projects within your deployed environment do not contain the correct "project" property associating them with the correct licensed project. To fix this, you must do the following.
Determine the Project ID of the "main" project that is licensed. This can be found within the License Manager or by clicking on the main project in your hosted portal and taking note of the Project ID within the Url of the browser.
Once you have the Project ID, you will need to connect your terminal to your MongoDB instance and perform the following command to set the correct "project" property values on your remote projects.
Of course, you will replace the LICENSED_PROJECT_ID with the ID of your "main" project. Once you do this, any projects within your new remote environment will register as "stages" within the main project and will not erroneously count against your license.
If you are using PDF Server APIs directly you need to change the way your application authorizes the requests:
- This should be used when keeping the same MongoDB database between upgrades. This maintains all indexes and internal metadata about the database.
- This should be used when switching databases during the export process. This is more of a JSON export that will store all records within JSON files that can be re-imported into a new database. This is useful for importing the database into a new database where new indexes will be created.
Please refer for both of these options. For most upgrade processes, mongodump is preferred. If situations arise in which it cannot be used, then a mongoexport could be used as a backup plan. In either case, it does not hurt to perform both operations before an upgrade process.
Follow the same process as described in the section to ensure that the update was processed cleanly. Follow the same steps provided in the section to ensure that any errors in the deployment are fixed.
To migrate submissions, use the Form.io CLI tool, which can be found on the Form.io Github repository . Download this tool to the local machine using the following command:
GET source project using .
GET destination project using .
Perform a to save the settings into the Destination settings.
Once this is done, restart the server so that the initialization process will install the as well as create the initial admin account for this project.
To deploy all of the forms and resources from the Community Edition into the Form.io Hosted (Saas) Platform at or your own Self-Hosted platform, use the Form.io CLI command line tool:
Next, all submissions can be migrated by following the documentation.
Be sure to review the before migrating major versions of the API and PDF servers. Before commencing the migration process, create a complete backup of any existing environments.
Please visit the for a full list of changes and fixes.
VM2 replaced with Isolated-VM One of the instigating motivations for releasing a new major version was the recent deprecation of the heavily depended on library VM2. Before 9.0.0, this library was heavily used to ensure proper sandbox execution of any server-side JavaScript evaluations that would occur within a number of Form features. The following server-side evaluations were previously executed within the VM2 runtime:
Each of these systems relies on a secure JavaScript evaluation context to securely execute JavaScript within a sandboxed environment. Due to VM2 being deprecated, Isolated VM was selected to replace this library: .
Therefore a new Submission Data Processing system was developed, which was released under the @formio/core library. The code behind this new system is Open Source and can be found @ .
New Server Validation Runtime Along with the new data processing system is a new validation runtime for every submission that is processed on the server. This system has been refactored to no longer use the full “formiojs” renderer on the server, but instead use a more dedicated data processing system provided by our core validation engine found @ . This change will improve performance as well as memory allocation when new submissions are sent to the server.
Upgrade to Node v20 As part of the upgrade for dependencies and data processing, we are also moving to use the Node v20 runtime within the Docker containers that run our Enterprise deployments. Node v20 includes several performance and security improvements, which can be found in their release notes found @ .
Migrate your 7.x API Server to the latest 8.x API Server with ease by following our .
Customers are encouraged to utilize our workflow solution.
With the addition of , the signature component has been relocated to the Premium components tab.
Additionally, Authenticated Users can now be configured to restrict their ability to create Projects and Teams by adding the following for the Enterprise API Server. ONLY_PRIMARY_WRITE_ACCESS
.
As of server release 8.1.0, Form.io has released a code-free Conditional Show/Hide UI using simple dropdown fields. This update relieves the need to write custom Javascript when the Show/Hide workflow is dependent on multiple fields or values.
Submission Revisions, Custom Collections, Localization of the Developer Portal, and more!
If you are using any features that are within the Security Module, please ensure you have this Module as part of your subscription in order to use these features. Please send us an email if you have any questions, or to have this module added to your subscription here: .
This allows you to manage all of your licenses with the Form.io platform in one single location. It also streamlines how licenses are applied to both the Enterprise Server as well as the PDF Server. In addition to this, both PDF and Enterprise Servers now manage their licenses and configure their licenses in the same way. For more information, see the section of our help docs.
With the 7.x Server release, you can now configure the Group permissions to be categorized into different levels. For example, you can now configure users to be Admins of a group, or members of a group and then assign permissions separately based on their role within the group. For more information, please see the section in our user guide.
The 7.x Server release adds additional levels of security around user session management and ensures that any outstanding JWT token can no longer be utilized once a user has logged out of their current "session". This ensures that each JWT token can only be associated with a "current" session and JWT tokens associated with invalid sessions (through logout) can no longer be used. For more information, see our User user guide.
The audit logging system allows you to add additional logging capabilities that logs ever action taken by all users within the Form.io system. See for more information.
For the PDF Server (3.x version), you will also need to ensure that this container has access to the same database as the API Server. In some cases, this may require you to add a --link
to the mongo container. Please read the for detailed instructions on how to accomplish this.
In some cases, you may have been using the Hosted Portal (found at https://portal.form.io) to connect to your remote environment, which was running 6.x. Once you upgrade your deployments to the version 7.0.0 or greater it must be known that you can not use any portal version less than version 7.1.0. This version can be found at the footer of the hosted portal. If the hosted portal is not yet on that version, then one option is to enable the deployed portal within your environment. This can be accomplished by following the .
The migration process between 5.x server to the 6.x server should be a very seamless process. You will simply need to follow the instructions provided in the sections. There are some cases where you may run into DB update failures. For these cases, you will want to make sure that you follow the instructions provided in the section to mitigate these issues.
If you are doing client-to-server requests (your application is working on browser) then you should send requests to API Server's /pdf-proxy
endpoint using x-jwt-token
header, see of API docs.
If you are doing server-to-server requests (your application is working on the server and request readers cannot be exposed) then you can send requests directly to PDF Server using admin key authorization. To do this you should add the FORMIO_PDF_ADMINKEY
environment variable to your PDF Server run config. Set it to some secret key. After that, you can authorize your requests using x-admin-key
header, which should be the same as the environment variable value. See of API docs.