Installing the TIBCO® Metadata Agent

Installation and configuration

TIBCO® Metadata agent is a Java application provided as a jar file configurable by a “properties” file. Its main purpose is to establish interactions between the TIBCO® Data Virtualization (TDV) instance installed on a customer’s server and TIBCO Cloud™ Metadata.

One agent can be connected to one instance of TDV. TIBCO Cloud™ Metadata supports connection to multiple agents that manage different data sources.

Prior to agent configuration, you have to create a technical user in Metadata Administration Menu.

For more details, see this page.

Beware, the technical user created is automatically prefixed with ebx-.

Prerequisites

The TIBCO® Metadata Agent requires:

• JRE 11 minimum
• TIBCO® Data Virtualization version 8.3 minimum

Security and Cloud Access

In order to access to TIBCO Cloud Metadata (CIC Cloud), and to allow TIBCO Cloud™ Metadata to access to the TIBCO® Metadata Agent, infrastructure considerations are to be taken into consideration :

• TIBCO® Metadata Agent hosting server must be opened to the Internet.
• It is recommended to define a DNS Name for the TIBCO® Metadata Agent hosting server, in order to ease certificate management.
• TIBCO® Metadata Agent hosting server must be reachable via HTTPs (HTTP over SSL), using a CA-signed certificate (no self-signed certificate).
• TIBCO® Metadata Agent external port must be port 443 (default HTTPs port), using a reverse-proxy (IIS, Apache, Ngninx, …). (ma.http.ws-url has to be set to the reverse-proxied URL to the agent, reachable from Internet).
• TIBCO® Metadata Agent internal port can be configured on any available port on the machine (see ma.http.port in Agent configuration properties), and use in the reverse-proxy configuration to expose the TIBCO® Metadata Agent to Internet.

Installation process

Below are the steps to follow to get the TIBCO® Metadata Agent running on your system:

1. Download TIBCO® Metadata Agent package from the Metadata Administration perspective.
2. Unpack it on your system.
3. Fill in properties template as described below in the ‘Properties configuration’ section. Please, note that for some properties it is required to log into Metadata and/or obtain some information from Metadata administrator.
4. Launch TIBCO® Metadata Agent as described below in the ‘Execution’ section.

Packaging

The agent is available as a standalone Java application (jar file).

It can be downloaded within TIBCO Cloud™ Metadata, in Administration menu.

Properties template

######################################################
######################################################
#####                                            #####
#####               TDV Database                 #####
#####                                            #####
######################################################
######################################################
ma.tdv.datasource.url=jdbc:compositesw:dbapi@[[TDV SERVER HOST]]:9401?domain=composite&dataSource=system
ma.tdv.datasource.username=[[TDV USER]]
ma.tdv.datasource.password=[[TDV PASSWORD]]

#####################################################
######################################################
#####                                            #####
#####               TDV SOAP                     #####
#####                                            #####
######################################################
######################################################
#Base URL for TDV SOAP interfaces
ma.tdv.soap.base-url=http://[[TDV SERVER HOST]]:9400
ma.tdv.soap.username=[[TDV USER]]
ma.tdv.soap.password=[[TDV PASSWORD]]
#A root folder to get private data sources. This folder will be scanned as well as its subfolders
#You can indicate several root comma separated paths
ma.tdv.soap.datasources-root-folder=/shared/examples
#Folder in which Agent will create new private datasources from TCMD. It must exist in your TDV server.
ma.tdv.soap.new-datasources-folder=/shared/examples
#Additional SQL data types not supported for profiling feature
ma.tdv.profiling.not-supported-types=
#Additional SQL data types supported by profiling feature except for the cardinality analyses
ma.tdv.profiling.not-supported-types-for-cardinality=

######################################################
######################################################
#####                                            #####
#####               Metadata connection          #####
#####                                            #####
######################################################
######################################################
ma.metadata.context=[[METADATA BASE URL]]
ma.metadata.username=[[METADATA TECHNICAL USER NAME]]
ma.metadata.password=[[METADATA TECHNICAL USER PASSWORD]]
ma.metadata.public-key=[[METADATA TECHNICAL USER JWT]]
ma.metadata.label=[[LABEL FOR THIS AGENT]]
####
#### Uncomment this line if you want to use encryption for agent passwords. 
#### In this case, you will have to enrypt password using TIBCO Metadata Agent Tools
#### Read the documentation for more informations
####
#ma.metadata.useEncryption=true

######################################################
######################################################
#####                                            #####
#####               Agent properties             #####
#####                                            #####
######################################################
######################################################
ma.http.port=[[AGENT PORT]]	
ma.http.ws-url=[[AGENT CONTEXT URL]]

######################################################
######################################################
#####                                            #####
#####               Agent log properties         #####
#####                                            #####
######################################################
######################################################
#Folder where the logs will be stored
ma.logging.path=./logs
#Log level for technical logs (Spring framework, external libraries).
ma.logging.level.root=INFO
#Log level for the Agent's functional logs.
ma.logging.level.agent=INFO
#Log level for the SOAP messages. Set to TRACE to see full request body.
ma.logging.level.soap.sent=INFO
#Log level for the SOAP messages. Set to TRACE to see full response body.
ma.logging.level.soap.received=INFO

Properties configuration

Prior to launching the TIBCO® Metadata Agent, you need to complete the information in the properties file to check the connection to the TDV and to TIBCO Cloud™ Metadata.

Users are strongly discouraged to modify properties that are not described in the table below but are present in the properties file, especially specific URLs configured for Metadata operation or the H2 database JDBC URL. A good practice is to remove old specific properties from your prior-to-version 3 agent, in order to keep only specific properties in your configuration file.

Once these properties are configured, you can launch the agent as a Java application, using command line or any other suitable tool (see below on this page).

The configuration file should be named application.properties

Upon the first launch, the agent will query TIBCO Cloud™ Metadata REST API for registration and will obtain a GUID to be used in all information exchanges between the two applications.

Please, check that in the agent’s log file after start up you see the message ‘Started MetadataAgentApplication in xx seconds’. This indicates that the start up was correct and the agent is running.

If you don’t see this message, your need to check the log file for error messages explaining the issue preventing agent to start.

Once registered, the agent will be displayed in TIBCO Cloud™ Metadata Administration Menu, as described here

TDV Connection rights

The agent must be authorized to connect to TDV using two connections flows:

1. JDBC connection to the TDV system datasource using JDBC URL defined in property ‘ma.tdv.datasource.url’.
2. SOAP connection using URL defined in property ‘ma.tdv.soap.resources-url’.

Check that this is authorized on your TDV instance prior launching the Agent.

TDV and Metadata Connection Encryption

If ma.metadata.useEncryption is activated, then ma.tdv.datasource.password, ma.tdv.soap.password and ma.metadata.password must be encrypted into application.properties before starting the agent :

• As a prerequisite, ma.metadata.public-key must be defined in application.properties according to the corresponding TIBCO Cloud™ Metadata Technical User.

• Use TIBCO Metadata Agent Tools to encrypt the original password. Launch in a terminal :

  java -jar metadata-agent-tools.jar MY_PLAIN_PASSWORD

The output provided is as follows :

 `**************************`
 `Value to encrypt is MY_PLAIN_PASSWORD`
 `**************************`
 `Encrypted value is : I7UOEPOHtF+J4EYPTPRxR0++QG4hoXFe87uSPluI1lbi`
 `**************************`

• Copy-paste the encrypted values into application.properties for each corresponding password property.

Caution : with this feature enabled, encrypted password must be re-encrypted on each renewal of JWT key.

Agent execution (as a Standalone Java Application)

You have to configure a launcher to execute the agent, adding the configuration (application.properties) file into a config directory under the directory where is placed the agent.

logs and storage directories are created by the agent itself directly under the structure.

Under storage, two files are created, one for agent local database (configDB.mv.db), one for scheduling (quartzDB.mv.db). If you have previously used different names for these files in your previous configuration, please rename these files accordingly in your file system, after stopping the current agent.

To launch the agent, you can create a shell file as follows (in this example the jar is placed in a /metadata-agent directory):

You have to define the default encoding as UTF-8 for the JVM using -Dfile.encoding=UTF-8, if not already set.

Example on Linux :

java -jar metadata-agent.jar >> /logs/metadata-agent.log  2>&1

Example on Windows :

java -Dfile.encoding=UTF-8 -jar metadata-agent.jar > /logs/metadata-agent.log  2>&1

Once started, an application.pid file is created containing the process id of the agent. You can stop the agent by killing this process.

kill $(cat application.pid)

Troubleshooting

TDV connection issues

Metadata connection issues