Skip to main content

Frequently Asked Questions

Find answers to the most commonly asked questions.

Getting log files for support​

  1. cd /opt/dbmarlin/tomcat/logs
  2. tar -czvf logs.tar.gz *

To get them off the server either sftp them or alternatively.

  1. mv logs.tar.gz ../../www/static/
  2. Then download from the browser https://[dbmarlinserver]:[port]/static/logs.tar.gz

Time zone error on starting MySQL sensor​

When adding a MySQL database target you might find the sensor fails to connect with a timezone error like this.

Sensor - The server time zone value 'EDT' is unrecognized or represents more than one time zone. You must configure either the server or JDBC driver (via the 'serverTimezone' configuration property) to use a more specifc time zone value if you want to utilize time zone support.

The reason is that newer versions of the MySQL JDBC driver which we use for the sensor don’t like older style timezone names such as EDT which have been deprecated. The workaround is to use a custom JDBC URL with a parameter to set a new style time zone name.

For example ?serverTimezone=America/New_York would work in this case. See List of tz database time zones for more detail on which timezones are Canonical and which are Deprecated.

You can set by toggling Advanced mode in the Database target setup screen.

/img/setting-custom-timezone.jpg

Monitoring interval​

DBmarlin samples the active sessions to see the statements they are running and the events they are waiting on. The default monitoring interval is 1 second. In most cases this should be ok but in rare cases it might not be possible to sample every second if the database isn't able to respond quickly enough. In this case you might choose to run at a lower sampling rate like every 2 seconds or even every 5 or 10 seconds in extreme cases.

You can change the monitoring interval by editing the sensor and enabling Advanced mode where you will see a setting Monitoring Interval (milliseconds). Decreasing the frequency will mean that timing data will be less accurate over shorter time ranges but will become more accurate over longer periods.

Batch interval​

Several things share the batch interval. The default is every 1 minute. Database statistics and SQL Statistic (if enabled) will use the batch interval. Explain plan capture for the top statements and change detection for parameters and schema changes will also use the batch interval.

Since change detection use the batch interval the actual change time could be anytime within the period before the change was detected. If you wish you can run change detection more frequently by editing the sensor and enabling Advanced mode where you will see a setting Batch Interval. Increasing frequency may increase the overhead of DBmarlin (particularly if you have a very large number of schema objects), so should be done with due caution. Change detection can also be disabled completely for an instance by setting Detect Changes to off.

What is the data retention for DBmarlin?​

DBmarlin collects data and stores it at 1 second granularity giving you maximum visibility into database performance problems even if they are only short spikes. That second-level data is stored for 8 days before being purged which means you can do root cause analysis even long after the event. The 1 second data is rolled up into 1 hour aggregations and the hourly data is stored for 31 days currently for longer term trending.

While the retention period can be extended, it will require more disk storage and could slow down the DBmarlin UI and upgrade process. Therefore we recommend using Snapshots as an alterative way to keep the data for short periods of interest.

caution

These parameters should not be increased by too much otherwise DBmarlin performance may degrade and disk space usage will increase. This setting will affect all monitored instances.

Change retention period for 1s data​

DBmarlin stores data at 1s granularity for maximum visibility. This same data is also used for the 1 minute aggregations. The default retention is 8 days. Should you wish to keep granular data for longer than 8 days a server side parameter raw_data_retention_days in the DBmarlin parameter table which can be updated via SQL. There are scripts psql.sh (Linux) or psql.bat (Windows) in the scripts/dbadmin directory that can be used to connect to the DBmarlin repository database.

select * from parameter p where parameter_name = 'raw_data_retention_days';
insert into parameter (parameter_name, parameter_value) values (‘raw_data_retention_days’, ‘14’);

This will change the retention period for raw data to 14 days.

Change retention period for hourly data​

DBmarlin stores data at hour granularity to allow querying longer time ranges efficiently. The default retention is 31 days. Should you wish to keep hourly data for longer than 31 days a server side parameter hourly_data_retention_days in the DBmarlin parameter table which can be updated via SQL. There are scripts psql.sh (Linux) or psql.bat (Windows) in the scripts/dbadmin directory that can be used to connect to the DBmarlin repository database.

select * from parameter p where parameter_name = 'hourly_data_retention_days';
insert into parameter (parameter_name, parameter_value) values (‘hourly_data_retention_days’, ‘60’);

This will change the retention period for hourly data to 60 days.

Password protecting DBmarlin​

You can password protect the DBmarlin Nginx server using Basic authentication. See section on authentication.

Deleting old DBmarlin log files​

The main log files for DBmarlin are under ./tomcat/logs. The log files will automatically rotate daily but not be deleted.

If you want to delete any files older than 7 days you can run the following command.

find /opt/dbmarlin/tomcat/logs -mtime +7 -exec rm {} \;

If you want this to run on a scheduled then cron can be used to run this command daily.

Change default sampling frequency​

DBmarlin samples the target database every second by default. This allow us to show granular 1s data points for maximum visibility. In some instances, it may not be possible to sample every second if the database is too slow to respond or network conditions between the DBmarlin agent and the target database don't allow it. If you wish you can run sampling less frequently. See Monitoring Interval.

Increase connection timeout​

Anything that takes longer than a second can cause the sampling to lag. You can increase the connect timeout by editing the JDBC connect string. You can edit your connect string by toggling on Advanced mode. For Oracle for example, setting CONNECT_TIMEOUT=10 would increase the timeout to 10 seconds. The full connect string would look like:

jdbc:oracle:thin:@(DESCRIPTION=(CONNECT_TIMEOUT=10)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=<ip address>)(PORT=<port>)))(CONNECT_DATA=(SERVICE_NAME=<service-name)))

Monitoring MySQL 5.1​

MySQL 5.1 is a very old database these days and therefore requires an older version of the JDBC driver than the one that DBmarlin comes with. If you are running DBmarlin v1.5.0 or above it is possible to monitor MySQL 5.1 by copying the older JDBC driver into the tomcat/lib directory, restarting Tomcat and then modifying the Class name that the Sensor uses by editing the DBmarlin database.

  1. Download the ZIP Archive mysql-connector-java-5.1.39.zip from https://downloads.mysql.com/archives/c-j/.
  2. Extract the zip file and find inside it mysql-connector-java-5.1.39-bin.jar which should be copied to the tomcat/lib directory.
  3. cd /opt/dbmarlin to change directory to the base directory where DBmarlin is installed.
  4. ./stop.sh -t to stop the Tomcat process.
  5. For Linux set export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/dbmarlin/lib:/opt/dbmarlin/postgresql/lib
  6. ./postgresql/bin/psql -U dbmarlin -p 9070 to login to the DBmarlin PostgreSQL database. Then run the following 2 SQL statements:
    1. update datasource set driver = 'org.gjt.mm.mysql.Driver' where driver = 'com.mysql.cj.jdbc.Driver'; to change any existing MySQL databases to use the older MySQL 5.1 driver.
    2. update datasource_type set default_driver = 'org.gjt.mm.mysql.Driver' where default_driver = 'com.mysql.cj.jdbc.Driver'; to change the default for any new MySQL databases added.
  7. ./start.sh -t to restart the Tomcat process.

Note that after an upgrade of DBmarlin steps 4-7 above will need to be repeated.

SQL Statements truncated​

PostgreSQL​

There is a limit on the length of SQL text that can be captured for PostgreSQL databases. The default value is 1024 characters. This means that large SQL statements are truncated and also that the statements cannot be explained, causing the Execution Plans tab to be empty. If you have long SQL text then you can increase this limit by setting a parameter in your postgresql.conf file of the target database. For example, to double the limit to 2048 characters you can set the following parameter. Note that this needs a restart of PostgreSQL to take effect.

track_activity_query_size = 2048  # (change requires restart)

MySQL & MariaDB​

If you notice that the SQL text for statements in MySQL or MariaDB are truncated then this is because of a limit in the database and not DBmarlin. The default limit for the SQL text length in the performance schema in MySQL and MariaDB is 1024 characters. You will also not be able to obtain execution plans from such statements, since the complete text is required.

If you wish to increase the length, you will need to change a parameter in the database configuration file and restart the database. Which one you need will depend on the flavour and version of the database. Below are some examples of how to set the limit to 2048 characters. Note that this needs a restart of MySQL/MariaDB to take effect.

MySQL 5.7​

max_digest_length = 2048   # (change requires restart)

MariaDB 10.5.2+​

performance_schema_max_sql_text_length = 2048 # (change requires restart)

As far as is known, there is not a way to increase the text length in MariaDB prior to 10.5.x: max_digest_length and performance_schema_max_sql_text_length do not seem to work.

Common Vulnerabilities​

Apache log4j Vulnerabilities CVE-2021-44228 & CVE-2021-45046​

DBmarlin does NOT use Log4j or contain any Log4j files within it's JAR files and therefore there is no risk from CVE-2021-44228 or CVE-2021-45046 for DBmarlin installations.

  • CVE-2021-44228 - Known as Log4Shell, this is a high severity vulnerability impacting versions 2.0 to 2.14.1 of the Apache Log4j 2 utility. It was disclosed publicly via the project’s GitHub on December 9, 2021.
  • CVE-2021-45046 - It was found that the fix to address CVE-2021-44228 in Apache Log4j 2.15.0 was incomplete in certain non-default configurations and applications could still vulnerable to a denial of service attack. This is fixed in Log4j 2.16.0.

No action is required for DBmarlin installations.

Excluding wait events​

Some wait events are actually idle events such as SLEEP_TASK in the case of SQL Server. We try to update idle events to exclude then from monitoring as soon as we find then. If you find any wait events like this you can report them to us via our knowledge-base suggestion form and we will get them updated in the next possible release. There is also a mechanism whereby customers can exclude wait events themselves by updating a field in the dbmarlin schema. Please contact support if you need this.

Supporting DB connections requiring TLS v1.0 or v1.1​

If you are monitoring a server which requires TLS v1.0 or v1.1 which are now obsolete you will need to edit ./lib/security/java.security to change disabledAlgorithms and restart the DBmarlin Tomcat as follows:

  1. Open [dbmarlin]/java/jdk[version]-jre/lib/security/java.security in a text editor
  2. Search for jdk.tls.disabledAlgorithms
  3. Remove TLSv1, TLSv1.1 from the list of disabledAlgorithms
  4. Save the file
  5. Restart tomcat using either ./stop.sh -t && ./start.sh -t or using the Windows or Linux Services if applicable.

Setting a static Mac address for your DBmarlin server when running in AWS EC2​

If you are running your DBmarlin server in AWS on an EC2 instance, there is a chance that if you ever needed to change the instance type, or if Amazon needed to replace your instance with a new one due to hardware degradation, then it would come back with a new Mac address for the network interface, which would invalidate your DBmarlin license which is tied to the Mac address. In order to avoid this, it is recommended that you assign an Elastic Network Interface (ENI) which is like a virtual network card that can move from one instance to another if required. You can then use the Mac address of the ENI as the one which the license is tied to.

Instana integration not working​

  1. Check that the Instana Host Address or Instana API Key have been entered correctly.
  2. Note that the Instana Host Address should be like this https://abc-xzy.instana.io (IMPORTANT - without a trailing slash). If you add a trailing slash then the API calls to Instana will fail with CORS errors.
  3. Use Browser Developer Tools, Network Panel to see if the XHR calls are being made to the Instana API and what Response they are getting.

Collecting more execution plans​

DBmarlin will try to collect execution plans for the top 20 statements in each Batch interval. For v3.11 and below, the default was to collect plans for the top 10 statements. Starting in v3.12, there is a parameter explain_limit in tomcat/webapps/agent##n.n/META-INF/context.xml which allows you to control the number of plans that DBmarlin tries to collect each batch interval.

caution

Any context.xml parameter changes do not persist through upgrades and will need to be reset after an upgrade.

Error relating to libcrypt.so.1 on starting DBmarlin​

If you get an error relating to libcrypt.so.1 being missing when you startup DBmarlin on a newer Linux such as RHEL/Rocky/Centos 9 then you need to install the libxcrypt-compat.x86_64 package.

yum install libxcrypt-compat.x86_64