Upgrade on Linux
Please read carefully before starting the upgrade process.
Introduction​
Depending on whether there have been changes to the DBmarlin repository schema between the currently installed version and target version, the upgrade may be very fast or could take some time.
- Upgrading between versions where the DBmarlin schema has not changed is a quick process and just requires the new tar.gz file to be extracted over the top of the old installation while all DBmarlin processes are stopped and then running
configure.sh. - Upgrading between versions where the DBmarlin schema has has changed, will take longer and will require a period of downtime while the data is migrated to the new schema.
Please follow the pre-requisites first to find out which case you are dealing with. The upgrade steps will be the same in either case but the time taken to complete will be different.
Pre-requisites​
Follow the pre-requisites below to check the repository schema version for the version you already have and the version you are looking to install.
Backup existing installation​
It is a good idea to have a backup of the current installation to rollback to in the event that the upgrade is unsucessful. A directory copy or filesystem snapshot while the DBmarlin processess have been stopped would give you something to restore back if needed.
Download the latest version​
Download the dbmarlin-Linux-x64-[VERSION].tar.gz file from https://www.dbmarlin.com/download
Check the current schema version​
You can see which version you have installed and the version of the repository schema it is using by looking at the version.txt file in the root of the DBmarlin intallation directory. For example:
cat version.txt
ui-version=1.0.0
schema-version=1.3
Check the new schema version to be installed​
Extract the tar.gz of the newer version into a temporary location and do the same check of the version.txt file to check the schema-version.
If the schema-version is the same as the one you already have, then no data migration is required and the upgrade will be quick. If the schema-version is newer than the one you already have, then data migration is required which could take some time. Additionally, you must check you have enough disk space before proceeding.
Checking disk space​
If the schema-version has changed then you should make sure that you have enough disk space on the volume where DBmarlin is installed to accommodate double the amount of data, plus 10 percent headroom, for a short period of time while migration completes. You can use du to get the disk usage for the DBmarlin directory and df to get the disk space available on the file system which DBmarlin is installed on. For example:
cd /opt/dbmarlin
du -hs .
6.1G .
df -h .
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/centos-root 97G 11G 87G 11% /
In the example above DBmarlin is using 6.1GB so would required an extra 6.71GB (6.1GB +10%) free space to complete the upgrade. The filesystem has 87GB available so we are ok to proceed.
Once the data migration has completed and all data has been copied to the new schema, the old schema will be automatically dropped freeing up the extra disk space.
Upgrade steps​
Stop all DBmarlin server components​
Stop all DBmarlin processeses and make sure they are all stoppped.
./stop.sh
./status.sh
nginx (Not running)
tomcat (Not running)
postgres (Not running)
Extract the tar.gz file​
Extract the newer tar.gz over the top of the current install. For example if DBmarlin is installed in /opt/dbmarlin then run:
tar -xzvf dbmarlin-Linux-x64-[VERSION].tar.gz -C /opt/
Configure the installation​
After extracting the tar.gz file you will find configure.sh which needs to be run first before you can start up the server. This configures ports and other settings.
cd /opt/dbmarlin
./configure.sh
Press Enter then space bar to scroll
...
Do you accept the agreement Y/N ? y
Done.
Set port for Nginx (current 9090): 9090
Set port for Tomcat (current 9080): 9080
Set port for PostgreSQL (current 9070): 9070
See https://docs.dbmarlin.com/docs/Getting-Started/server-installation/hardware-requirements for Profile sizes
1) XSmall
2) Small
3) Medium
4) Large
5) Xlarge
Choose the profile [1-5]: 2
Small profile selected
Recommended RAM is 4GB. You have 4G
Continue Y/N ? y
...
The configure.sh will check if this is a first time installation or if there is an existing schema and whether that schema is already the correct version or if a new schema needs to be created.
Unattended configuration (Optional)​
If you want to automate the running of configure.sh rather than run it interactively, you can pass in command line options. This allows installations and upgrades to be automated through automation tools.
# Example:
./configure.sh -a -n9090 -t9080 -p9070 -sMedium -u
# All parameters must be specified:
-a = Accept license agreement in LICENSE.txt
-n = Nginx port number to use
-t = Tomcat port number to use
-p = PostgreSQL port number to use
-s = Size of profile to use. Valid options are: XSmall, Small, Medium, Large, XLarge
-u = Unattended mode. Script will complete without prompting for Y/N to proceed
For descriptions of profile sizes see Hardware requirements
Scenario 1 - No schema update necessary​
If the schema version is the same you will see a message like this indicating that no schema change required and the upgrade is now complete. You can run start.sh to restart all processes.
Detected the following schema versions:
v1_3
You have the latest schema already. No schema update necessary.
All done!
Next run ./start.sh to startup the DBmarlin services
Scenario 2 - New schema and data migration required​
If a new schema is needed, you will see a message like this. This means a new schema is needed. The data migration will start run as long as you have enough free disk space and could take another from a few mins to a few hours depending on the volume of data you have collected.
Detected the following schema versions:
1_7
Checking for enough disk space. You need double + 10 percent during a schema upgrade.
DBmarlin is using: 299236 K
Filesystem free is: 94096000 K
Space required is: 329159 K
Freespace is OK for schema upgrade.
Current schema 1_7 is less than required version 1_8
Will create the new 1_8 schema now.
...
Data will copy from the old 1_7 schema to the new 1_8 schema now.
Please be patient. It could take over 20 mins depending on your database size. Please let it complete to avoid leaving you database in an inconsistent state.
03-Dec-2021 13:46:16.185 Data-migrator: migrate data from v1_7 to v1_8
03-Dec-2021 13:46:16.191 Data-migrator: stop sensors null
03-Dec-2021 13:46:16.198 Data-migrator: stop host sensors null
03-Dec-2021 13:46:16.207 Data-migrator: restart sequences
03-Dec-2021 13:46:16.213 Data-migrator: restart sequence for licence_id at 2
03-Dec-2021 13:46:16.222 Data-migrator: restart sequence for agent_id at 2
03-Dec-2021 13:46:16.235 Data-migrator: restart sequence for event_type_id at 5
03-Dec-2021 13:46:16.241 Data-migrator: migrate tables
03-Dec-2021 13:46:16.242 Data-migrator: migrate table licence
03-Dec-2021 13:46:16.363 Data-migrator: migrate table integration
03-Dec-2021 13:46:16.372 Data-migrator: migrate table agent
03-Dec-2021 13:46:16.381 Data-migrator: migrate table datasource
03-Dec-2021 13:46:16.394 Data-migrator: migrate table host
03-Dec-2021 13:46:16.405 Data-migrator: migrate table datasource_host
03-Dec-2021 13:46:16.417 Data-migrator: migrate table integration_map
03-Dec-2021 13:46:16.426 Data-migrator: migrate table change
03-Dec-2021 13:46:16.437 Data-migrator: migrate table custom_event_type
03-Dec-2021 13:46:16.447 Data-migrator: migrate table custom_event
03-Dec-2021 13:46:16.456 Data-migrator: migrate table sql
03-Dec-2021 13:46:16.459 Data-migrator: migrate table sql_plan
03-Dec-2021 13:46:16.463 Data-migrator: migrate table activity_1hour
03-Dec-2021 13:46:16.472 Data-migrator: migrate table activity
03-Dec-2021 13:46:16.486 Data-migrator: migrate table sql_statistic_1hour
03-Dec-2021 13:46:16.490 Data-migrator: migrate table sql_statistic
03-Dec-2021 13:46:16.496 Data-migrator: migrate table statistic_1hour
03-Dec-2021 13:46:16.498 Data-migrator: migrate table statistic
03-Dec-2021 13:46:16.505 Data-migrator: migrate table host_statistic_1hour
03-Dec-2021 13:46:16.508 Data-migrator: migrate table host_statistic
03-Dec-2021 13:46:16.518 Data-migrator: drop schema v1_7
03-Dec-2021 13:46:16.596 Data-migrator: start sensors null
03-Dec-2021 13:46:16.596 Data-migrator: start host sensors null
03-Dec-2021 13:46:16.597 Data-migrator: data migration took 633780ms
Stopping postgresql
waiting for server to shut down.... done
server stopped
Listing files which aren't in the manifest.txt
Deleting ...
All done!
Next run ./start.sh to startup the DBmarlin services
Then connect to http://dbmcentos7:9090/ in your browser
Data migration could take several minutes or even several hours for large databases. During this time you will not be able to start any sensors until the migration has completed. Please do not terminate the process or stop postgres during this time or you will end up with an inconsistent database.
Restart all DBmarlin server components​
./start.sh
./status.sh
nginx (Running)
tomcat (Running)
postgres (Running)
If you try to start/stop/edit a sensor which the upgrade is in progress you will be prevented from doing so and the sensor status will report 'Stopped - upgrading'.
Refresh the UI​
It is recommended that following an upgrade that you do a hard refresh in your browser to make sure that you have the latest versions of all files loaded. shift + cmd + R on MacOS or Ctrl + F5 on Windows.
Troubleshooting​
Check the log file​
You can tail the latest Tomcat localhost log file to see where the upgrade has got up to. For example:
cd /opt/dbmarlin/tomcat/logs
vi localhost.2020-11-30.log
You can search the log for Data-migrator to find lines related to the data migration.
Check in PostgreSQL if the migration is still running​
You can query pg_stat_activity to see the active sessions and whether the migration process is still running or is blocked on something such as a lock.
cd /opt/dbmarlin
echo "select * from pg_stat_activity where state <> 'idle';" | ./postgresql/bin/psql
Check the output to see if the migration is still prgressing or if it is blocked on something.
Rolling back to the previous version​
If the data migration got part way through before failing then you may have both the old and new schema versions. You should clean up the new schema that was created so that you can retry the upgrade again. We provide a drop_schema.sql script for this.
List the schema versions in the database
cd /opt/dbmarlin
echo "\dn" | ./postgresql/bin/psql | grep -o 'v[[:digit:]]*_[[:digit:]]' | sort
Run the drop_schema.sql script BEFORE you restore the tar.gz from the previous version. This is so the newer schema is dropped and not the older one you want to roll back to.
cd /opt/dbmarlin
cat ./postgresql/sql/drop_schema.sql | ./postgresql/bin/psql
To go back to the old version you can now run ./stop.sh to stop all processes and then extract the tar.gz file from the old version back over the top of the current installation and then run ./start.sh
Retrying a failed upgrade​
To try an upgrade again you should first rollback to the previous version as descibed above. Then run through the upgrade steps again.
Contact support​
Reach out to support via one of the contact options listed on https://www.dbmarlin.com/support