Skip to main content

Data Setup

Introduction​

With Netuno, application development begins with building the database, through the construction of forms and fields.

And when working in a team or when it's necessary to move application development to another environment, it's necessary to create a backup of the database along with the code.

Thus, at the destination, it will be necessary to reconfigure the database so that the application can be implemented as it was developed in its origin.

The problem with this process is the effort of always remembering to create a new backup every time there is a change in the database. And the more people involved in development and/or the more environments there are to publish the application, the more complicated this process becomes.

So Netuno, to solve this problem, provides an application setup system, which automatically generates a database structure (schema), and additionally allows the manual programming of the loading of initial data through additional scripts.

It also allows you to change the database server type, ensuring that the setup runs normally and the application runs without any problems. This means that an application can be developed in H2DataBase and, with the generated schemas, the application's database connection can be automatically changed to PostgreSQL or MariaDB. On the first execution of the application on a new database, everything will be rebuilt, regardless of any change in the database server type.

Therefore, it allows the database to be completely changed without impacting the development already carried out.

The setup programming files are located within the application folder at:

  • server/setup

Setup Configuration​

In the application environment settings located at:

  • /apps/MY_APP/config/_development.json
  • /apps/MY_APP/config/_production.json

You can manipulate the behavior of the 'setup' through the settings:

    ...
    "setup": {
        "enabled": true,
        "schema": {
            "auto_create": true,
            "execution": true
        },
        "scripts": {
            "execution": true
        }
    },
    ...
enabled: boolean | default true​

It defines whether the setup is active or not, and thus activates or deactivates, the first time the application is run, the execution of data setup operations.

schema.auto_create: boolean | default true​

It allows Neptune to know whether or not to generate database schema files whenever a form or field is created or modified.

schema.execution: boolean | default true​

Controls whether the schema should be executed or not during setup operations performed on the first execution of the application.

scripts.execution: boolean | default true​

Enables or disables the execution of additional scripts during setup operations performed on the first run of the application.

Schema​

During the creation or modification of forms and fields in the Netuno web interface, schema files will be generated within the setup folder as follows:

  • server/setup/_schema-FORM_NAME.js

There will be a _schema-*** file for each form in the application.

These files should not be manually modified because they are automatically generated by Netuno.

Therefore, if you change the content of these files, in a subsequent generation of schemas they will be automatically replaced, and the changes made directly to the files will be lost.

For the automatic generation of the database schema to be executed, it is necessary to ensure that the configuration schema.auto_create is active, and that the setup folder exists inside the server folder.

Scripts​

Custom-programmed code scripts are files located in the setup folder that do not begin with _.

The main purpose of these scripts is to load the data, ensuring that the database contains the minimum information necessary for the application to function correctly.

Thus, when the application is reconfigured in another environment, it will be automatically loaded with the initial data, becoming ready to start its use.

To perform these data loading operations, you should use the functions of the _db resource.

The scripts are executed after the execution of the schemas in alphanumeric order of the file names that do not begin with _.

Starting filenames with a numeric sequence is ideal for defining the execution order, therefore custom scripts are executed in the order of filenames that do not start with _.

Insert Custom Data​

And, to save the work of performing database queries to check whether or not an insert data can be performed, Netuno provides the insertIfNotExists function.

This function checks if the data exists exactly as it is defined in the database; if it does not exist, then it creates it.

An example of its use would be to load the Code and Name fields from the Territory Type form, where the table name is territory_type and it has the columns code and name. To do this, simply create the file with the following content:

server/setup/001-territory_type.js
_db.insertIfNotExists("territory_type", _val.map()
    .set("code", "1001")
    .set("name", "Continental")
);
_db.insertIfNotExists("territory_type", _val.map()
    .set("code", "1002")
    .set("name", "Archipelago")
);
_db.insertIfNotExists("territory_type", _val.map()
    .set("code", "2003")
    .set("name", "Maritime")
);
_db.insertIfNotExists("territory_type", _val.map()
    .set("code", "3004")
    .set("name", "Aerial")
);

Processing Cycle​

As previously explained, setup is only executed on the first run of the application.

And it's execution order is:

  1. _start - executes the initialization file, which can contain any type of custom programming:
    • MY_APP/server/setup/_start.js
  2. _cleanup - performs the removal of forms, reports, and their respective fields programmatically:
    • MY_APP/server/setup/_cleanup.js
  3. _schema-* - Executes all scripts with the prefix _schema-, containing the commands for creating forms, reports, and their respective fields:
    • MY_APP/server/setup/_schema-autogenerated_structure.js
    • MY_APP/server/setup/_schema-category.js
    • MY_APP/server/setup/_schema-product.js
    • MY_APP/server/setup/_schema-client.js
  4. 001-others_scripts - executes custom-programmed script files; these are all files that do not begin with _ and that will likely contain the necessary initial data inserts or other custom developments, for example:
    • MY_APP/server/setup/001-insert_custom_data.js
    • MY_APP/server/setup/002-category.js
    • MY_APP/server/setup/003-product.js
    • MY_APP/server/setup/004-client.js
  5. _end - executes the finalization file, which can contain any type of custom programming:
    • MY_APP/server/setup/_end.js

That is, it is allowed to create custom code in the start and end scripts to perform some operations that make sense to be executed before the setup or even when it is completed.

It is important to emphasize that this processing cycle is executed every time the application is run for the first time, which is why it is necessary to make sure of the existence of then data before it is created.

Make sure the following environmental settings are active to allow this process to run in the MY_APP/config/_development.json or MY_APP/config/_production.json files depending on the environment:

  • Enables or disables completely the execution of any setup operation:

    setup.enabled = true|false

  • Enables or disables the execution of _schemas-:

    setup.schema.execution = true|false

  • Enables or disables the execution of scripts:

    setup.scripts.execution = true|false

Cleaning​

To schedule the removal of forms and fields, we must use the _cleanup.js script, which ensures that the structure will be removed, if it exists, from all environments that synchronize project files during setup.

Example of the cleanup script:

server/setup/_cleanup.js
// Remove the zone form:
_form.dropIfExists("zone");
// Remove the subarea_id field from the area form:
_form.dropFieldIfExists("area", "subarea_id");

DBML​

Whenever Netuno runs the data setup, the following file is generated:

  • server/setup/_db-schema.dbml

This is the file containing the entire data structure that can be used with Artificial Intelligence.

Based on the DBML content, Artificial Intelligence understands the data structure and can generate precise SQL queries.

Try using this file as an attachment in the command prompt and ask it to generate complex SQL queries; you will be able to obtain good results.

Conclusion​

The setup process of applications in Netuno has as main functions:

  • _schema-: automatically generates the entire structure of forms and reports with their respective fields, which allows you to ensure the entire consistency of the structure of the application database, especially when the application is run with a new empty database.
  • scripts: perform database operations to ensure that the application contains the mandatory data previously loaded each time it is run for the first time.

This way Netuno allows applications to work in new environments, even if you change the type of database, ensuring the correct operation of the application without the need to perform backups and restore of the database.

Additionally allowing all automatically and manually generated configuration code to be versioned with GIT, for example.