AWS Deployment

Amazon AWS is perfect for Enterprise customers who wish to have a robust platform with high availability. It is used and trusted by some of the largest companies in the world to serve as their cloud infrastructure. Below is a diagram showing the architecture of an AWS Deployment.

Prerequisites

This procedure assumes the user has an account with Amazon AWS.

DB Requirements

Form.io requires MongoDB or a MongoDB-compatible DB. Before deployment, create a database to store all the Forms and Submissions within the Form.io deployment. This document describes what to configure on each respective service:

• MongoDB Atlas

• AWS DocumentDB

MongoDB Atlas

Before deployment, create an account at MongoDB.com/atlas. Follow the steps provided by MongoDB to create the cluster while observing the guidelines described below.

Performance

When configuring the performance of the cluster, it may be necessary to review the advanced settings. The particular settings selected may depend on the planned deployment. The following baseline standard may be used when determining the application's needs:

1. For production environments, the dedicated tier may be required.

2. Consider enabling multi-region replication to create additional instances for greater availability.

3. Choose a sufficient cluster tier based on the expected usage. M10 is typically sufficient for development. M20 is the recommended baseline for production.

Security

When configuring the security settings in MongDB Atlas after clicking Create Cluster, refer to the following guidelines:

1. Record the username and password for use in the connection string later.

2. Configure VPC peering with the appropriate details.

   1. The configuration window will require the AWS ID and VPC ID from the deployment's AWS Account.

   2. Enter the VPC CIDR as found in Amazon's AWS Dashboard at Networking > VPC.

3. Record the Peering Connection ID provided by MongoDB Atlas.

4. From the AWS Dashboard, complete the peering configuration from the Networking > VPC > Peering Connections menu.

If the deployment requires additional security that would mandate a private endpoint, refer to the MongoDB Documentation regarding connecting cloud services to a private endpoint.

Connection String

From the MongoDB Atlas Project Overview, open the Connect Menu and generate a legacy (standard) connection string. It will appear similar to the following:

mongodb://formio:<db_password>@<node1>,<node2>,<node3>/?replicaSet=<replicaSetName>&ssl=true&authSource=admin&retryWrites=true&w=majority&appName=<appName>

Note the following placeholders:

• <node1>... : These will correspond to the individual instances of the DB.

• <replicaSetName>: The atlas-generated replica set name.

• <appName>: Typically formio.

Supplying the required info creates a connection string similar to the following, using the example password badPassword:

mongodb://formio:[email protected]:27017,formio-shard-00-01.qa5hm.mongodb.net:27017,formio-shard-00-02.qa5hm.mongodb.net:27017/?replicaSet=atlas-6fxuzs-shard-0&ssl=true&authSource=admin&retryWrites=true&w=majority&appName=formio

Observe that?ssl=true indicates a database connection over SSL, &retryWrites=trueindicates retryWrites is supported, and uses the admindatabase rather than IAM to authenticate.

Record this connection string for later.

DocumentDB

DocumentDB is an AWS managed NoSQL database that can store projects, forms, and submissions.

From the AWS Console, navigate to the Amazon DocumentDB service. Launch a new DocumentDB service, following the AWS process.

Performance

Configure the instance according to the following parameters:

1. Cluster Identifier: Keep default or type a unique name.

2. Engine Version: 5.x and below.

3. Instance Class:

   • Production environment: At least db.r5.large

   • Development environment: At least db.t3.medium

4. Number of Instances:

   • Production Environment: At least 2

   • Development Environment: At least 1

Authentication

Provide a Master username and a secure password for the cluster. These will be required later.

Connection String

Now that the DocumentDB cluster is created, click on the cluster link, and copy the application connection string. It should look like the following:

mongodb://formio:<insertYourPassword>@docdb-2024-08-23-12-49-44.cluster-chnyabcd6feh.us-east-1.docdb.amazonaws.com:27017/?tls=true&tlsCAFile=global-bundle.pem&replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false

This connection string must be converted to a standard connection string. To do so:

1. Remove everything after the “:27017/”, and then append the database name to the end, for example, "formio". It should look like:

 mongodb://formio:<insertYourPassword>@docdb-2019-02-05-03-07-17.cluster-chnyabcd6feh.us-east-1.docdb.amazonaws.com:27017/formio

1. Replace <insertYourPassword> with the Master Password previously entered in the Authentication field. This example uses "badpassword":

mongodb://formio:badpassword@docdb-2019-02-05-03-07-17.cluster-otsyrtio9xoe.us-east-1.docdb.amazonaws.com:27017/formio

1. Add ?tls=true&retryWrites=false&authSource=admin to the end of the URL. This indicates a database connection over SSL, disables retryWrites since DocumentDB does not support it, and uses the admindatabase rather than IAM to authenticate (see here for more information on IAM authentication):

mongodb://formio:badpassword@docdb-2019-02-05-03-07-17.cluster-otsyrtio9xoe.us-east-1.docdb.amazonaws.com:27017/formio?tls=true&retryWrites=false&authSource=admin

Record this connection string for use later.

Navigate to the newly created cluster's dashboard. At the bottom of the page, take note of the Security Group Name that the cluster was added to. It will be used later to ensure that the Form.io Enterprise Server can communicate with the database.

Cloud File Storage Using S3

Before setting up Form.io API and PDF servers, set up an S3 Bucket which will contain the uploaded PDFs used by the PDF server.

Create a bucket

1. From the AWS Console, navigate to the AWS S3 service.

2. Create a new S3 Bucket, following the AWS process.

3. Give the bucket a unique name and record it for later use.

Create a User

Create an IAM role with admin rights to this S3 bucket. From the AWS dashboard, navigate to the IAM service and use the Create User wizard. Create a descriptive named use (such as pdf-server), noting the following essential settings:

1. In the Set Permissions page for the new user, select Attach existing policies directly, and then select Create Policy.

2. Ensure that within the Service menu's Specify Permissions page, the Select a Service option is set for S3.

3. In the Actions section, select All S3 actions to give this IAM role access to all of the read, write, and update actions available via the S3 service.

1. In the Resources dropdown, click the Add ARN link next to the bucket section. Enter the bucket name specified in the previous section. Click Add ARNs.

2. In the object resource section, click Add ARN and then provide the bucket name. Next, click on the Any checkbox for the object section. Then click the Add ARNs button.

1. For the remaining Resources, click on each one and enable Any in this account for all of them. It should look like the following:

1. Click the Next button to view the Review and Create page. Name the policy "pdf-server-s3" and click on the Create policy button.

Attach the policy

Now that a policy has been created, it must be attached to the IAM role. From AWS dashboard, clicking the Refresh icon, and then searching for the newly created policy (pdf-server-s3) in the search bar.

1. Click Next in the IAM wizard to view the Review and Create page. Select Create User to return to the list of Users.

2. From the newly created user, navigate to Security Credentials.

3. Click Create Access Key, then. select Third-party Service. Enter a description for its use.

4. Write down the access key ID and the secret key in a secure location, both will be used to configure environment variables later.

Following these steps, the S3 bucket should be configured to allow file storage for the deployment. Next, proceed to the Elastic Beanstalk deployment!

Generate Deployment Package

Once the infrastructure is in place, generate the deployment packages. Refer to the process described in the Readme in the Form.io Github Deploy repo.

Following the procedure will result in running a command similar to the following:

formio-deploy package compose/aws/multicontainer.zip --license=YOURLICENSE

This will generate a new ZIP file within the deployments folder for compose/aws/multicontainer.zip as well as place the deployment in the deployments/current folder. This .zip file will be used next to deploy to AWS Elastic Beanstalk.

To configure the deployment package for use with an alternative DB:

1. Extract docker-compose.yml from the multicontainer.zip.

2. In a text editor, locate the MongoDB connection string. It defaults to

   Copy

   MONGO: mongodb://mongo:27017/formio

3. Change the parameter to the connection string configured previously. It may look like the following:

mongodb://formio:[email protected]:27017,formio-shard-00-01.qa5hm.mongodb.net:27017,formio-shard-00-02.qa5hm.mongodb.net:27017/?replicaSet=atlas-6fxuzs-shard-0&ssl=true&authSource=admin&retryWrites=true&w=majority&appName=formio

1. Save the file.

Elastic Beanstalk

Now that the database and S3 have been configured, this deployment will use AWS Elastic Beanstalk to manage the Docker deployments.

Create Role

Before proceeding, configure an EC2 instance profile so the underlying instances can interact with Elastic Beanstalk. Return to the IAM Dashboard and select Roles from the menu, then click Create Role. Use the following steps to create the appropriate role:

1. Set the Trusted Entity Type to AWS Service and EC2 for the Use Case, then click Next.

2. Filter the Permissions Policies by the string "ElasticBeanstalk" and select each of the following:

• AWSElasticBeanstalkWebTier

• AWSElasticBeanstalkWorkerTier

• AWSElasticBeanstalkMulticontainerDocker

1. Click Next, then give the role a meaningful name (e.g aws-elasticbeanstalk-ec2-role). This role will be the EC2 Instance Profile role for the Elastic Beanstalk deployment.

2. Click Create Role to complete the process.

Elastic Beanstalk Deployment

Next, proceed with the Elastic Beanstalk Deployment.

Environment

To begin, open the Elastic Beanstalk Dashboard and click either Create Application or Create New Environment.

1. Select Web Server Environment and provide a meaningful Application Name.

2. Select Docker as the managed platform:

1. Select Upload your code and upload the multicontainer.zip created previously. Give the version a meaningful label.

2. Under Presets, select High Availability, and proceed.

1. Choose to Use an existing service role. Under Existing service roles, select the aws-elasticbeanstalk-service-role.

2. Optionally, associate a key pair with the EC2 instances associated with your environment. Refer to the AWS Document "Amazon EC2 key pairs and Amazon EC2 instances" for more information.

3. In the EC2 instance profile, select the role previously created in the IAM dashboard, then proceed.

Network, Database, and Tag Configuration

Proceed with the process on AWS.

1. Select the same VPC and Instance Subnets configured previously. A typical deployment will run the instances in private subnets, and the load balancer in public subnets.

2. Click Next to proceed to instance traffic and scaling.

3. Set the Root Volume Type to General Purpose (SSD).

4. Increase the size of the storage for each instance from the default of 8GB to 12GB. The PDF server container stores a large volume of fonts. Theis can be fine tuned later based on the deployment requirements.

5. Under EC2 Security Groups, select the security group previously created in the DocumentDB cluster above.

6. Select the desired capacity settings. In general:

• Development Environments: t3.medium, or higher

• Production Envrionments: t3.large, or higher

1. Use the default Load Balancer, Listeners, Processes, and Rules settings. TLS listeners are configured later in this document.

2. Click Next to navigate to updates, monitoring, and logging configuration.

3. Configure the desired Health Reporting metrics, Platform Updates settings, and Updates and Deployments settings. Generally, a Rolling deployment policy ensures the Form.io deployment doesn't experience downtime when updating the platform.

4. Provide the following environment variables within the Environment Properties section. These are the environment variables which will govern the Form.io platform in important ways.

To complete the setup, click Next to review the configuration, then click Submit to begin the environment creation process.

After a few minutes, the environment will be available and ready for a new Form.io Project.

Form.io Project

Now that the deployment is complete, verify it is operational by logging in.

On the first page, use the ADMIN_EMAIL and ADMIN_PASS values (established in the Environment Variables in a previous step) to authenticate into the developer portal.

Once logged into the Developer Portal, click Create Project.

In the popup modal, give the project a title, then click Create Project.

Domain Routing (Route 53)

Now that the environment is operational and a project has been created, the next task is to attach a domain to the Elastic Beanstalk deployment.

If Elastic Beanstalk was configured to use High Availability, it created Elastic Load Balancers in front of the deployment, which can be linked against the DNS records.

To get begin:

1. Navigate to the AWS homepage and then search for Route 53.

1. From the left pane, select Hosted zones and click Create hosted zone.

1. Provide the domain name and click Create hosted zone.

2. Next, create a new Record Set Provide the following record:

• *.yourdomain.com

• Type - A Record

• Alias - Yes

• Select the Elastic Load Balancer as the target

• Press Create.

When configured, the routes should look something like the following:

1. Set the domain nameservers to point to the ones provided by Route 53. Once the domains evaluate, verify the deployed API is visible within that domain.

2. Return to the Formi.io Portal, and configure the PDF Server URL that was configured in a previous step with the new DNS name. It should resemble "https://pdf.yourdomain.com".

Configure SSL certificate for Application Load Balancer

Create basic records in Route 53. The following example shows a record for both the root path and for the "www" path:

The route record after the update should look like the following:

Ensure the IAM use with appropriate permissions is logged in to the AWS Certificate Manager and click the Request button.

Add the SSL certificate to the Route 53 domain, then validate the cert. Choose the recommended option for validating the certificate.

If a "CNAME" record type is created in the Route 53 domain, verify that it is the issued certificate from the previous step.

Refresh the certificate screen. It should look similar to the following configuration when the certificate has been issued:

The next steps will add a listener to the load balancer for port 443 (HTTPS).

1. Navigate to the EC2 Dashboard, then open the Load Balancers tab from the left pane.

2. On the main screen open the Listener tab, then click Add Listener.

1. Set the listener to HTTPS and port to 443, then choose a Security Policy.

2. Click the Select a Certificate button.

1. Navigate back to the Listeners tab of the Load Balancer pane:

If a small orange warning icon appears next to the port 443 listener, add an Inbound rule for port 443 to the security group.

• Hover over the icon for details on what security group needs to be updated.

• Navigate to the Security Groups (EC2 Dashboard > Security Groups). Expand the column to determine which security group it is then click Edit Inbound Rules.

• Select HTTPS, then choose a source. Click Save Rules.

This should remove the orange caution mark on the listener's tab shown in the previous step:

Navigate to the domain to see if the SSL Certificate has been configured.

Troubleshooting

If the deployment fails, the best first step is to download the logs for the container. These can be downloaded in the Elastic Beanstalk console by first navigating to your Elastic Beanstalk deployment, and then click on Logs.

Next, you will select Full from the Request Logs dropdown.

Once the logs have downloaded, unzip the logs provided into the local machine. Use a text editor to open the following log file:

/log/eb-docker/containers/eb-current-app/eb-stdouterr.log

This file will display the logs from the Docker containers, and explains what happened to the server during the deployment process.

One common issue is that it cannot connect to the DocumentDB database, as seen in the following logs:

api-server-1   | Uncaught exception:
api-server-1   | Error: Could not connect to the given Database for server updates: mongodb://formio:<PASSWORD>@abcde.cluster-abcdefg6feh.us-east-1.docdb.amazonaws.com:27017/formio?ssl=true&replicaSet=rs0&retryWrites=false.
api-server-1   |     at /src/node_modules/formio/src/db/index.js:255:17
api-server-1   |     at unlock (/src/node_modules/formio/src/db/index.js:77:14)
api-server-1   |     at /src/node_modules/formio/src/db/index.js:254:9
api-server-1   |     at /src/node_modules/formio/node_modules/mongodb/lib/utils.js:695:9
api-server-1   |     at /src/node_modules/formio/node_modules/mongodb/lib/mongo_client.js:285:23
api-server-1   |     at connectCallback (/src/node_modules/formio/node_modules/mongodb/lib/operations/connect.js:367:5)
api-server-1   |     at /src/node_modules/formio/node_modules/mongodb/lib/operations/connect.js:554:14
api-server-1   |     at connectHandler (/src/node_modules/formio/node_modules/mongodb/lib/core/sdam/topology.js:286:11)
api-server-1   |     at Object.callback (/src/node_modules/formio/node_modules/mongodb/lib/core/sdam/topology.js:672:9)
api-server-1   |     at Timeout._onTimeout (/src/node_modules/formio/node_modules/mongodb/lib/core/sdam/topology.js:443:25)
api-server-1   |     at listOnTimeout (internal/timers.js:557:17)
api-server-1   |     at processTimers (internal/timers.js:500:7)
api-server-1   | Error: Could not connect to the given Database for server updates: mongodb://formio:<PASSWORD>@abcde.cluster-abcdefg6feh.us-east-1.docdb.amazonaws.com:27017/formio?ssl=true&replicaSet=rs0&retryWrites=false.
api-server-1   |     at /src/node_modules/formio/src/db/index.js:255:17
api-server-1   |     at unlock (/src/node_modules/formio/src/db/index.js:77:14)
api-server-1   |     at /src/node_modules/formio/src/db/index.js:254:9
api-server-1   |     at /src/node_modules/formio/node_modules/mongodb/lib/utils.js:695:9
api-server-1   |     at /src/node_modules/formio/node_modules/mongodb/lib/mongo_client.js:285:23
api-server-1   |     at connectCallback (/src/node_modules/formio/node_modules/mongodb/lib/operations/connect.js:367:5)
api-server-1   |     at /src/node_modules/formio/node_modules/mongodb/lib/operations/connect.js:554:14
api-server-1   |     at connectHandler (/src/node_modules/formio/node_modules/mongodb/lib/core/sdam/topology.js:286:11)
api-server-1   |     at Object.callback (/src/node_modules/formio/node_modules/mongodb/lib/core/sdam/topology.js:672:9)
api-server-1   |     at Timeout._onTimeout (/src/node_modules/formio/node_modules/mongodb/lib/core/sdam/topology.js:443:25)
api-server-1   |     at listOnTimeout (internal/timers.js:557:17)
api-server-1   |     at processTimers (internal/timers.js:500:7)

When this occurs, consider the following processes to resolve the issue:

Unable to connect to Database

When a deployment is unable to connect to the database, the following troubleshooting steps may help:

• Verify the database and Elastic Beanstalk are on the same VPC. Do this by navigating to the Elastic Beanstalk deployment and verify the "vpc-" ID is the same as the database.

• Check the Security Group for DocumentDB. Verify that the Elastic Beanstalk environments have Security Groups that can connect to the DocumentDB. The security group for the DocumentDB should have Incoming network requests that allow TCP communication on the 27017 port as follows:

• Check your Security Group for Elastic Beanstalk. After verifying that the security group for DocumentDB, check the Security Group for Elastic Beanstalk. Verify that one security group is configured to allow Outbound rules to allow for TCP communication over the 27017 port.

• Check the Connection string. Verify that it matches the format as documented in the DocumentDB section of this walkthrough tutorial.

• Check for expired Certificates being used on the DocumentDB containers. In order to determine if this is the problem. please refer to the next section, "Fixing Certificate Expiration Issues."

Fixing Certificate Expiration issues

One common problem for database connection issues is that the Certificate Authority of the DocumentDB has expired and now refuses to connect.

The following procedure will explain how to diagnose and fix the problem:

1. Navigate to the DocumentDB cluster, then click on each instance of the cluster.

1. Select on Configurations and click Modify.

1. Select the rds-ca-rsa2048-g1 certificate.

1. Save the settings for this instance.

Repeat the steps above for all instances in the DocumentDB cluster.

Next, navigate to the Elastic Beanstalk deployment. Click on Application Versions, then download the version that is used by the server to the local machine.

Extract the contents of this ZIP on the local machine. Use a text editor to open the following file:

certs/rds-combined-ca-bundle.pem

Download the ˆglobal-bunch.pem" file found at the URL: https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem

Copy the contents of the “global-bundle.pem” file to the clipboard.

Replace ALL the contents of the “rds-combined-ca-bundle.pem” file with the contents of the “global-bundle.pem” file. Make sure to save the file.

Create a NEW .zip file by selecting the “docker-compose.xml” file, along with the other folders in that directory, like so:

Give the file a label with the form “9.3.0-NEW-CERT.zip”, using the same version as the deployed version.

Upload this new version to the Application Versions.

Select the version that was just uploaded, and deploy to all environments. It should be able to re-connect to the database after a re-deploy.

If deployment issues persist, SSH into the deployment container to diagnose the issues on the actual container instance.

SSH into your deployments

There are many reasons why Docker containers may fail to start. When this happens, troubleshoot the reason by inspecting the logs from those containers.

This can usually be done by downloading the logs from the Elastic Beanstalk. On some rare occasions (such as when a License has been disabled) the containers will not create the logs.

When this occurs, the best course of action is to SSH into the EC2 containers to diagnose the problem by completing the following steps:

1. Create a KeyPair to enable the SSH. Do this by navigating to the EC2 section of AWS, and then clicking on Key Pairs. Then click Create key pair.

1. Follow the instructions to create a new key pair. When completed, download the private key onto the local machine. Ensure that this downloaded key has the correct permissions by executing the following command:

chmod 0400 my-key.pem

1. Navigate to the Elastic Beanstalk deployment, and then edit the Configurations for the deployment. Click Edit on the Security section.

1. Select the new key pair and click Save.

2. Once the deployment is finished making these updates, go to the EC2 section of AWS, then click on Instances.

3. Find the instance that is associated with the deployment. Copy the Public DNS name of that instance.

You can now ssh into the instance by with the following command:

ssh -i ./my-key.pem [email protected]

After logging into the instance, perform the following to see the Docker images:

sudo su
docker ps -a

This should display the failed container, as seen here:

Copy the Container ID of one of the failed containers. Use the following command to SSH into the failed container:

docker commit [CONTAINER_ID] formio-debug
docker run -it \
    -e "LICENSE_KEY=---LICENSE KEY---" \
    -e "MONGO=--- MONGO CONNECTION STRING ---" \
    -e "PORTAL_ENABLED=true" \
    -e "DEBUG=true" \
    -e "[email protected]" \
    -e "ADMIN_PASS=CHANGEME" \
    -e "DB_SECRET=CHANGEME" \
    -e "JWT_SECRET=CHANGEME" \
    -e "FORMIO_S3_BUCKET=--- PDF BUCKET ---" \
    -e "FORMIO_S3_KEY=--- S3 KEY ---" \
    -e "FORMIO_S3_REGION=us-west-2" \
    -e "FORMIO_S3_SECRET=---- S3 SECRET ---" \
    --rm --entrypoint sh formio-debug

Try to manually run the software with the following commands:

For pdf-server:

node pdf.js

For formio-enterprise:

node formio.js

This will output the reason it failed to start. For example, here is an output from a PDF server that could not start:

This indicates the license for this server has been disabled.