# Audit Logging Produce audit logs of all user activity on the system. This will generate an auditable trail of user session events and access modification of all entities in the system. Audit logs will output to the standard out for the docker container and can be routed into a log aggregation system if needed. {% hint style="info" %} This feature is included in the [**Security and Compliance**](https://help.form.io/developer-tool-ecosystem/the-security-module) package. Follow the link or contact for more details. {% endhint %} The Audit Log data is presented in the following format: ``` date EVENT uuid projectId sessionId userId [additional information] ``` You can turn off audit logging by setting the `NOAUDITLOG` flag to true in the `.env` file or in docker secrets. ### Audit Logging Events There are a number of events that are tracked when audit logging is enabled. The following outlines these events that are logged. ### Successful/Unsuccessful account login events When a user is trying to authenticate into the portal, four different types of events can occur based on whether the login was successful, unsuccessful (username), unsuccessful (password), and a log in count. These four events are... #### EAUTH\_NOUSER (Unsuccessful username login) The `EAUTH_NOUSER` event occurs when a user tries to log in with a username/email that does not exist. Example Log | date | event | uuid | projectId | sessionId | userId | Attempted username/email | | ------------------------ | ------------- | ------------------------------------ | ------------------------ | --------- | ------ | ------------------------ | | 2024-04-25T14:50:03.172Z | EAUTH\_NOUSER | 27759aa2-085b-4719-9193-5bad3fad74a2 | 659d83632824949f20a882e1 | NoSession | NoUser | | #### EAUTH\_PASSWORD (Unsuccessful password login) The `EAUTH_PASSWORD` event occurs when a user tries to log in with a username/email that does exist but the password is incorrect. Example Log | date | event | uuid | projectId | sessionId | userId | submissionId of the resource | Attempted username/email | | ------------------------ | --------------- | ------------------------------------ | ------------------------ | --------- | ------ | ---------------------------------------- | ------------------------ | | 2024-04-25T17:10:21.472Z | EAUTH\_PASSWORD | a3245767-dcc5-4ed2-b998-cd36cdbeb9c6 | 659d83632824949f20a882e1 | NoSession | NoUser | new ObjectId("659d83632824949f20a88389") | | #### AUTH\_LOGIN (Successful login) The `AUTH_LOGIN` event occurs when a user logs in successfully. Example Log | date | event | uuid | projectId | sessionId | userId | | ------------------------ | ----------- | ------------------------------------ | ------------------------ | ---------------------------------------- | ---------------------------------------- | | 2024-04-25T17:10:21.472Z | AUTH\_LOGIN | ac245617-fd22-479d-9f8e-4846b0ac8177 | 659d83632824949f20a882e1 | new ObjectId("662a9137cf86fe3fba034396") | new ObjectId("659d83632824949f20a88389") | #### EAUTH\_LOGINCOUNT (Unsuccessful login count) The `EAUTH_LOGINCOUNT` event occurs when a user tries to log in with a username/email that does exist but the password is incorrect. A log count is created for the number of times a user unsuccessfully tries to log in. Example Log | date | event | uuid | projectId | sessionId | userId | Attempted username/email | | ------------------------ | ----------------- | ------------------------------------ | ------------------------ | --------- | ------ | ------------------------ | | 2024-04-25T14:50:03.172Z | EAUTH\_LOGINCOUNT | 27759aa2-085b-4719-9193-5bad3fad74a2 | 659d83632824949f20a882e1 | NoSession | NoUser | | ### Process tracking These events occur to track the process of other events. They can be used to group processes together. #### REQUEST\_START (Start of any API request) The `REQUEST_START` event occurs when any API request is made to the API server. It denotes the start of the request. Note: All request proceeding the `REQUEST_START` will have the same `uuid` until the `REQUEST_END`. This allows for many events to be grouped by the API request that was made. Example Log | date | event | uuid | projectId | sessionId | userId | method | path | parameters | | ------------------------ | -------------- | ------------------------------------ | ------------------------ | --------- | ------- | ------ | ------------------------------------------------------- | ------------ | | 2024-04-25T17:43:17.654Z | REQUEST\_START | 5f46bed7-2a55-4ad7-b21f-cfccd724912b | 659d83632824949f20a882e1 | NoSession | No User | POST | /project/659d83632824949f20a882e1/user/login/submission | {"live":"1"} | #### REQUEST\_END (End of REQUEST\_START event) The `REQUEST_END` event occurs when the API request has finished. It denotes the end of the request made by a REQUEST\_START event. Note: All request preceding the `REQUEST_END` will have the same `uuid` until the `REQUEST_START`. This allows for many events to be grouped by the API request that was made. Example Log | date | event | uuid | projectId | sessionId | userId | status code | rtt | | ------------------------ | ------------ | ------------------------------------ | ------------------------ | --------- | ------- | ----------- | ----- | | 2024-04-25T17:43:17.654Z | REQUEST\_END | 5f46bed7-2a55-4ad7-b21f-cfccd724912b | 659d83632824949f20a882e1 | NoSession | No User | 200 | 167ms | ### Authentication checks These events occur for requests that are related to authentication #### AUTH\_ANONYMOUS (Request without auth token) The `AUTH_ANONYMOUS` event occurs when a request with no authorization token (x-jwt-token) is made. Example Log | date | event | uuid | projectId | sessionId | userId | | ------------------------ | --------------- | ------------------------------------ | ------------------------ | --------- | ------- | | 2024-04-25T17:43:17.654Z | AUTH\_ANONYMOUS | 5f46bed7-2a55-4ad7-b21f-cfccd724912b | 659d83632824949f20a882e1 | NoSession | No User | #### AUTH\_LOGOUT (Logout of account) The `AUTH_LOGOUT` event occurs when a user logs out. Example Log | date | event | uuid | projectId | sessionId | userId | | ------------------------ | ------------ | ------------------------------------ | --------- | ---------------------------------------- | ---------------------------------------- | | 2024-04-25T14:50:03.172Z | AUTH\_LOGOUT | 27759aa2-085b-4719-9193-5bad3fad74a2 | NoProject | new ObjectId("662fab333d908589b8931543") | new ObjectId("659d83632824949f20a88389") | ### Data Access When a resource is accessed/viewed in the portal, a data access event will occur. These events have a naming scheme of *RESOURCE*\_*ACCESS* #### SUBMISSION\_READ (Submission is viewed) The `SUBMISSION_READ` event occurs when a submission is viewed. This can occur multiple times in a request if a list of submissions are viewed. Example Log | date | event | uuid | projectId | sessionId | userId | submissionId | formId | | ------------------------ | ---------------- | ------------------------------------ | ------------------------ | ---------------------------------------- | ---------------------------------------- | ------------------------ | ------------------------ | | 2024-04-25T17:43:17.654Z | SUBMISSION\_READ | 4d7649c7-d24c-4b61-bc7e-6e672186eca6 | 6627c5f740a5d95c7edc30ab | new ObjectId("662aa339cf86fe3fba0343fe") | new ObjectId("659d83632824949f20a88389") | 662aa865cf86fe3fba034d96 | 662aa7bccf86fe3fba034afa | #### FORM\_READ (Form is viewed) The `FORM_READ` event occurs when a form is viewed. This can occur multiple times in a request if a list of forms are viewed. Example Log | date | event | uuid | projectId | sessionId | userId | formId | | ------------------------ | ---------- | ------------------------------------ | ------------------------ | --------- | ------- | ------------------------ | | 2024-04-25T17:43:17.654Z | FORM\_READ | 4d7649c7-d24c-4b61-bc7e-6e672186eca6 | 6627c5f740a5d95c7edc30ab | NoSession | No User | 662aa7bccf86fe3fba034afa | #### ACTION\_READ (Action is viewed) The `ACTION_READ` event occurs when an action is viewed. This can occur multiple times in a request if a list of actions are viewed. Example Log | date | event | uuid | projectId | sessionId | userId | actionId | formId | | ------------------------ | ------------ | ------------------------------------ | ------------------------ | ---------------------------------------- | ---------------------------------------- | ------------------------ | ------------------------ | | 2024-04-25T17:43:17.654Z | ACTION\_READ | 4d7649c7-d24c-4b61-bc7e-6e672186eca6 | 6627c5f740a5d95c7edc30ab | new ObjectId("662aa339cf86fe3fba0343fe") | new ObjectId("659d83632824949f20a88389") | 662aa85bcf86fe3fba034d08 | 662aa7bccf86fe3fba034afa | #### PROJECT\_READ (Project is viewed) The `PROJECT_READ` event occurs when a project is viewed. This can occur multiple times in a request if a list of projects are viewed. If you see a log with a projectId of `NoProject` then the ending Id represents stages within that project. Example Log | date | event | uuid | projectId | sessionId | userId | projectId / stageId | | ------------------------ | ------------- | ------------------------------------ | ------------------------------------ | ---------------------------------------- | ---------------------------------------- | ------------------------ | | 2024-04-26T16:27:58.279Z | PROJECT\_READ | 23b2c367-6fb6-41f2-a36a-0ba9a42f8bc9 | 6627c5f740a5d95c7edc30ab / NoProject | new ObjectId("662aa339cf86fe3fba0343fe") | new ObjectId("659d83632824949f20a88389") | 6627c5f740a5d95c7edc30ab | #### ROLE\_READ (Role is viewed) The `ROLE_READ` event occurs when a role is viewed. This can occur multiple times in a request if a list of roles are viewed. Example Log | date | event | uuid | projectId | sessionId | userId | roleId | | ------------------------ | ---------- | ------------------------------------ | ------------------------ | ---------------------------------------- | ---------------------------------------- | ------------------------ | | 2024-04-26T16:27:58.279Z | ROLE\_READ | a780acd0-2691-4cf4-9934-44f32e5415a3 | 6627c5f740a5d95c7edc30ab | new ObjectId("662aa339cf86fe3fba0343fe") | new ObjectId("659d83632824949f20a88389") | 6627c5f840a5d95c7edc30b2 | #### TEAM\_READ (Team is viewed) The `TEAM_READ` event occurs when a team is viewed. This can occur multiple times in a request if a list of roles are viewed. Example Log | date | event | uuid | projectId | sessionId | userId | teamId | teamResourceId | | ------------------------ | ---------- | ------------------------------------ | --------- | ---------------------------------------- | ---------------------------------------- | ------------------------ | ------------------------ | | 2024-04-26T16:27:58.279Z | TEAM\_READ | a780acd0-2691-4cf4-9934-44f32e5415a3 | NoProject | new ObjectId("662aa339cf86fe3fba0343fe") | new ObjectId("659d83632824949f20a88389") | 662967ad49e0b4f5b46904e8 | 659d83632824949f20a88316 | #### V\_READ (Form revision is viewed) The `V_READ` event occurs when a form revision is viewed. This can occur multiple times in a request if a list of revisions are viewed. Example Log | date | event | uuid | projectId | sessionId | userId | formRevisionId | | ------------------------ | ------- | ------------------------------------ | ------------------------ | ---------------------------------------- | ---------------------------------------- | ------------------------ | | 2024-04-29T14:39:44.708Z | V\_READ | e1627c41-1d39-4326-961b-681e5821bc99 | 6627c5f740a5d95c7edc30ab | new ObjectId("662faecc3d908589b893161a") | new ObjectId("659d83632824949f20a88389") | 662c17f09f39954d132d4525 | #### PROJECT\_SETTINGS (Project settings is viewed) The `PROJECT_SETTINGS` event occurs when project settings is viewed. This event is usually preceded by a GET /project/:projectId request. Example Log | date | event | uuid | projectId | sessionId | userId | | ------------------------ | ----------------- | ------------------------------------ | ------------------------ | ---------------------------------------- | ---------------------------------------- | | 2024-04-26T16:27:58.279Z | PROJECT\_SETTINGS | a780acd0-2691-4cf4-9934-44f32e5415a3 | 6627c5f740a5d95c7edc30ab | new ObjectId("662aa339cf86fe3fba0343fe") | new ObjectId("659d83632824949f20a88389") | ### Data changes When data is created or updated, a data change event will occur. These events have a naming scheme of *RESOURCE*\_*ACCESS* #### SUBMISSION\_CREATE (Submission is created) The `SUBMISSION_CREATE` event occurs when a new submission is created. Example Log | date | event | uuid | projectId | sessionId | userId | submissionId | formId | | ------------------------ | ------------------ | ------------------------------------ | ------------------------ | --------- | ------- | ------------------------ | ------------------------ | | 2024-04-25T17:43:17.654Z | SUBMISSION\_CREATE | 4d7649c7-d24c-4b61-bc7e-6e672186eca6 | 6627c5f740a5d95c7edc30ab | NoSession | No User | 662aa865cf86fe3fba034d96 | 662aa7bccf86fe3fba034afa | #### SUBMISSION\_UPDATE (Submission is updated) The `SUBMISSION_UPDATE` event occurs when a submission is updated. Example Log | date | event | uuid | projectId | sessionId | userId | submissionId | formId | | ------------------------ | ------------------ | ------------------------------------ | ------------------------ | --------- | ------- | ------------------------ | ------------------------ | | 2024-04-25T17:43:17.654Z | SUBMISSION\_UPDATE | 4d7649c7-d24c-4b61-bc7e-6e672186eca6 | 6627c5f740a5d95c7edc30ab | NoSession | No User | 662aa865cf86fe3fba034d96 | 662aa7bccf86fe3fba034afa | #### FORM\_CREATE (Form is created) The `FORM_CREATE` event occurs when a new form is created. Example Log | date | event | uuid | projectId | sessionId | userId | formId | | ------------------------ | ------------ | ------------------------------------ | ------------------------ | ---------------------------------------- | ---------------------------------------- | ------------------------ | | 2024-04-25T17:43:17.654Z | FORM\_CREATE | 4d7649c7-d24c-4b61-bc7e-6e672186eca6 | 6627c5f740a5d95c7edc30ab | new ObjectId("662faecc3d908589b893161a") | new ObjectId("659d83632824949f20a88389") | 662faf703d908589b89316c7 | #### FORM\_UPDATE (Form is updated) The `FORM_UPDATE` event occurs when a form is updated. Example Log | date | event | uuid | projectId | sessionId | userId | formId | | ------------------------ | ------------ | ------------------------------------ | ------------------------ | --------- | ------- | ------------------------ | | 2024-04-25T17:43:17.654Z | FORM\_UPDATE | 4d7649c7-d24c-4b61-bc7e-6e672186eca6 | 6627c5f740a5d95c7edc30ab | NoSession | No User | 662aa7bccf86fe3fba034afa | #### PROJECT\_CREATE (Project is created) The `PROJECT_CREATE` event occurs when a project is created. Example Log | date | event | uuid | projectId | sessionId | userId | projectId | | ------------------------ | --------------- | ------------------------------------ | --------- | ---------------------------------------- | ---------------------------------------- | ------------------------ | | 2024-04-26T18:12:46.382Z | PROJECT\_CREATE | 23f11fd3-a36e-4ce7-afae-58ab18bc988a | NoProject | new ObjectId("662bcba30dd222f70702f99e") | new ObjectId("659d83632824949f20a88389") | 662bf0840dd222f707030342 | #### PROJECT\_UPDATE (Project is updated) The `PROJECT_UPDATE` event occurs when a project is updated. Example Log | date | event | uuid | projectId | sessionId | userId | projectId | | ------------------------ | --------------- | ------------------------------------ | ------------------------ | ---------------------------------------- | ---------------------------------------- | ------------------------ | | 2024-04-26T18:12:46.382Z | PROJECT\_UPDATE | b3672267-494f-4d3d-838d-05e7637c23ef | 6627c5f740a5d95c7edc30ab | new ObjectId("662bcba30dd222f70702f99e") | new ObjectId("659d83632824949f20a88389") | 6627c5f740a5d95c7edc30ab | #### ROLE\_CREATE (Role is created) The `ROLE_CREATE` event occurs when a role is created. Example Log | date | event | uuid | projectId | sessionId | userId | roleId | | ------------------------ | ------------ | ------------------------------------ | ------------------------ | ---------------------------------------- | ---------------------------------------- | ------------------------ | | 2024-04-26T18:12:46.382Z | ROLE\_CREATE | 4d489fcc-e6a5-4a26-b747-50bd06f46405 | 6627c5f740a5d95c7edc30ab | new ObjectId("662bcba30dd222f70702f99e") | new ObjectId("659d83632824949f20a88389") | 662bf2c10dd222f7070305e0 | #### ROLE\_UPDATE (Role is updated) The `ROLE_UPDATE` event occurs when a role is updated. Example Log | date | event | uuid | projectId | sessionId | userId | roleId | | ------------------------ | ------------ | ------------------------------------ | ------------------------ | ---------------------------------------- | ---------------------------------------- | ------------------------ | | 2024-04-26T18:12:46.382Z | ROLE\_UPDATE | 4d489fcc-e6a5-4a26-b747-50bd06f46405 | 6627c5f740a5d95c7edc30ab | new ObjectId("662bcba30dd222f70702f99e") | new ObjectId("659d83632824949f20a88389") | 662bf2c10dd222f7070305e0 | #### TEAM\_CREATE (Team is created) The `TEAM_CREATE` event occurs when a team is created. Example Log | date | event | uuid | projectId | sessionId | userId | teamId | teamResourceId | | ------------------------ | ------------ | ------------------------------------ | --------- | ---------------------------------------- | ---------------------------------------- | ------------------------ | ------------------------ | | 2024-04-26T18:12:46.382Z | TEAM\_CREATE | 4d489fcc-e6a5-4a26-b747-50bd06f46405 | NoProject | new ObjectId("662bcba30dd222f70702f99e") | new ObjectId("659d83632824949f20a88389") | 662bf84c0dd222f7070306c1 | 659d83632824949f20a88316 | #### TEAM\_UPDATE (Team is updated) The `TEAM_UPDATE` event occurs when a team is updated. Example Log | date | event | uuid | projectId | sessionId | userId | teamId | teamResourceId | | ------------------------ | ------------ | ------------------------------------ | --------- | ---------------------------------------- | ---------------------------------------- | ------------------------ | ------------------------ | | 2024-04-26T18:12:46.382Z | TEAM\_UPDATE | 4d489fcc-e6a5-4a26-b747-50bd06f46405 | NoProject | new ObjectId("662bcba30dd222f70702f99e") | new ObjectId("659d83632824949f20a88389") | 662bf84c0dd222f7070306c1 | 659d83632824949f20a88316 | ### Aggregation System Deployment There are special aggregation systems that help to aggregate and manage your Docker logs. In this article, we will consider connection using the open-source **Elastic Stack** log analysis software as an example. The Elastic Stack consists of two different tools: **Elasticsearch** and **Kibana**. Elasticsearch is needed to store log data and Kibana to visualize it. To transfer the logs into ElasticSearch we will use the open source data shipper **FileBeat**. There are many ways to set up the Elastic Stack, the easiest is using docker-compose. We will use the official docker images and there will be a single ElasticSearch node. **Step 1:** Add the following rows to the existing `docker-compose.yml`: ``` services: elasticsearch: image: "docker.elastic.co/elasticsearch/elasticsearch:7.2.0" environment: - "ES_JAVA_OPTS=-Xms1g -Xmx1g" - "discovery.type=single-node" ports: - "9200:9200" volumes: - elasticsearch_data:/usr/share/elasticsearch/data kibana: image: "docker.elastic.co/kibana/kibana:7.2.0" ports: - "5601:5601" filebeat: image: "docker.elastic.co/beats/filebeat:7.2.0" user: root volumes: - /MY_WORKDIR/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro - /var/lib/docker:/var/lib/docker:ro - /var/run/docker.sock:/var/run/docker.sock volumes: mdb-data: elasticsearch_data: ``` *Note that you need to replace `MY_WORKDIR` with the path of your working directory.* For ElasticSearch and Kibana basic docker configuration is enough. They are respectively available on ports 9200 and 5601. ElasticSearch has a volume to keep its data. Kibana doesn't need a volume as it uses ElasticSearch to persist its configuration. FileBeat, on the other hand, needs to be configured. To share this config file with the container, we need a read-only volume `/usr/share/filebeat/filebeat.yml:ro`. FileBeat also needs to have access to docker log files. They can usually be found in `/var/lib/docker/containers` but that might depend on your docker installation. The docker socket `/var/run/docker.sock` is also shared with the container. That allows FileBeat to use the docker daemon to retrieve information and enrich the logs with things that are not directly in the log files, such as the name of the image or the name of the container. **Step 2**: Create the filebeat configuration file in `/MY_WORKDIR/filebeat.yml` ``` filebeat.inputs: - paths: - /var/lib/docker/containers/*/*.log type: container logging.json: true logging.metrics.enabled: false output.elasticsearch: hosts: - "elasticsearch:9200" indices: - index: "filebeat-%{[agent.version]}-%{+yyyy.MM.dd}" processors: - add_docker_metadata: host: "unix:///var/run/docker.sock" - decode_json_fields: fields: - message overwrite_keys: true target: json ``` * The FileBeat input type `container` is needed to import logs from docker. `/var/lib/docker/containers/*/*.log` is the location of the log files inside the FileBeat container. * The output `elasticsearch` setting allows you to configure the ElasticSearch address as well as the indexes where the logs are imported. The template index `filebeat-%{[agent.version]}-%{+yyyy.MM.dd}` includes a date. This means that docker logs are imported into the index corresponding to the date they appeared. * `add_docker_metadata` is needed in order to add useful information to the logs, such as the name of the image or the name of the container. Only IDs are displayed by default. * `decode_json_fields` lets to parse logs encoded in JSON. The logs in FileBeat, ElasticSearch, and Kibana consist of multiple fields. The `message` field is what the application (running inside a docker container) writes to the standard output. **Step 3:** Launch docker containers with `docker-compose` Kibana is now available on [http://localhost:5601](http://localhost:5601/). Click on the left on **Discover** and on **Create Index Pattern**. Create a pattern using `filebeat-*` (to include all logs from FileBeat) and `@timestamp`. ![](https://3305536326-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPHoF2HwOA0s5HV_AIB%2F-MPpG8EsVd8fKxvceF63%2F-MPpjaxUFHqzvniaLq5j%2FScreen%20Shot%202020-12-31%20at%2001.18.27.png?alt=media\&token=d327fb57-ccb2-4f18-9805-b42cb1c3c37b) ![](https://3305536326-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPHoF2HwOA0s5HV_AIB%2F-MPpG8EsVd8fKxvceF63%2F-MPpjh1wJgFKVpzf875c%2FScreen%20Shot%202020-12-31%20at%2001.18.43.png?alt=media\&token=7ad8f7ed-d9b5-40d5-81ad-7a5ba7822b9e) Done! Click on Discover again and you will see the logs. Note that logs of all containers are displayed with the default configuration. The formio logs described at the beginning of this article come from the formio container. ![](https://3305536326-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPHoF2HwOA0s5HV_AIB%2F-MPpG8EsVd8fKxvceF63%2F-MPplCP2VNYt-MoKK1yT%2FScreen%20Shot%202020-12-31%20at%2001.20.31.png?alt=media\&token=1015f612-3931-4c23-a0da-f36f43f1137b)