Software Version - Document Version: v1.8.0-1.4

COPYRIGHT

© 2020 TIXEL GmbH

The information in this document is property of TIXEL GmbH. It may not be used, reproduced or disclosed without written approval of TIXEL GmbH.

Notice of non-liability:

TIXEL GmbH is providing the information in this document to you AS-IS with all faults. TIXEL GmbH makes no warranties of any kind (whether express, implied or statutory) with respect to the information contained herein. TIXEL GmbH assumes no liability for damages (whether direct or indirect), caused by errors or omissions, or resulting from the use of this document or the information contained in this document or resulting from the application or use of the product or service described herein. TIXEL GmbH reserves the right to make changes to any information herein without further notice.

Used formatting:

Tip
 Additional hint
Note
 Remark, further information
Important
 Crucial note, please obey
Caution
 Neglecting may cause malfunction
Warning
 Neglecting may cause severe malfunction or data loss

1. Introduction

TIXstream MFT is continuously improved and enhanced by new features, that simplify daily life of administrators and users or allow to implement completely new workflows.

TIXstream MFT v1.2.x is used on many nodes that support a typical media file exchange workflow inside a corporate network. However, new features and bug fixes will be applied to v1.8.0 only and newer releases. Fixes for v1.2.x will only be provided in special cases. Thus it is highly recommended to update an existing v1.2.x installation to v1.8.0. Compatibility regarding the update procedure from v1.2.x to v1.8.0 and with respect to operating v1.2.x and v1.8.0 in a network is ensured, if your follow the steps in the followings sections.

In the following the update procedure from a v1.2.x default setup plus the following typical additional options is described (in the following referred to as standard v1.2.x broadcast workflow)

  • Integrated IRT BMF 2 metadata editor plugin

  • Certificate based mutual authentication

  • Enabled data channel encryption

Using a MySQL/MariaDB or MSSQL database server is required for production setups. File based h2 database is only recommended for test or evaluation setups.

Important
 Before updating a v1.2.x installation to v1.8.0 please carefully read the following sections. If you have a HA-cluster setup, the update process might be slightly different. Please contact TIXEL for further details.

If you have any questions regarding the procedure described below, please don’t hasitate to contact TIXEL.

2. New Features and Improvements since v1.2.x

At the time of writing TIXstream MFT v1.2.10 is the most recent version of the v1.2.x release line. In the minor releases from v1.3.x to v1.8.0 several new features have been added. Please see Changelog for all details starting from v1.3.0.

2.1. Features for Operation with Nodes using TIXstream MFT v1.2.x

The key improvements if you only need the feature set already available in v1.2.x are:

  • Users with role TIXstream Transfer Manager Limited User can now access history tab to browse/search/inspect jobs

  • History tab in dashboard now shows correct number of search results and has improved filter options

  • When using query strings at API with pagination, number of search results and pagination is correct now

  • Improved I/O Performance for (S)FTP shares

  • Automatic update of destinations cache

  • ETA and duration format now in hh:mm:ss

  • Fix incorrect data rate calculation for interrupted transfers

  • Improved logging

  • Prevent entering default users admin or demo as maintenance user

  • Restrict destination refresh access to admin only

  • Separate User Guide is now included in documentation package

  • New options to allow creation of fine grain performance log data to optimize throughput or detect bottlenecks

  • Linux only: install script can be provided with a umask parameter

  • Windows only: New file I/O type ("Windows") can be selected in TCC that can significantly improve performance to attached CIFS storage

  • Windows only: Integrated new service wrapper module providing better service status reporting and more convenient log rotation functionality

  • IPv6 support

  • RHEL6/CentOS6 support

  • Ubuntu 18 support

2.2. Features for Operation with Nodes using TIXstream MFT newer than v1.2.x

In case you plan to use TIXstream MFT in other workflows than those feasible with v1.2.x, the following features might be of interest depending on your specific requirements. For details please refer to the Installation or User Guide.

  • Configurable job folder creation (i.e. create new folder for each job)

  • Only create metadata file at destination if metadata exist

  • Preferred datachannel protocol (XP/RWTP) can be set for each peer node

  • Pixspan image compression support

  • Recursive folder copy support

  • Encryption policy can be set for each peer node

  • Pixspan policy can be set for each peer node

  • Include TIXEL metadata example plugin

    • Can be used as blueprint to create a custom editor according to your individual metadata schema requirements

    • Can be easily replaced with 3rd party editors like the BMF2 compliant one from IRT

  • Data integrity check support; Should be used when using data processing plugins like image compression to ensure correct coding/decoding behaviour

  • Watchfolder module to automatically create transfer jobs by monitoring new files in configured source share

3. Updating v1.2.x to v1.8.0

Important
 Before updating a v1.2.x installation to v1.8.0 please make sure you have a backup of your database and configuration files.

3.1. Database Settings

If you have a custom MySQL database configuration, that differs from a standard default, please make sure that NO_ZERO_DATE option is not set. Otherwise automatic TJM database schema migration will fail and TJM service cannot be started.

3.2. Windows

In the following we assume the installation directory %TIXEL_HOME% is D:\bin\tixel. Please adapt the path according to your setup. You need to execute all commands as administrator, i. e. right click "Run as administrator" or run it from "Command Prompt (Administrator)".

If applicable, please close Windows service admin application since this defers service removal. To make sure the new service wrapper is used, please stop all TIXEL services and unregister services:

D:\bin\tixel\admscripts\remove-tix-services.cmd

Then unzip the installation package and run the installation script install.cmd as usual to update the installation.

The following changes have been applied to the %TIXEL_HOME% directory structure:

  • TIXstream transfer-status/checksum persistence:

    • %TIXEL_HOME%/tmp → %TIXEL_HOME%/data/tixstream

Before starting the TIXEL services, please check and adapt the configuration files according to the sections below.

3.3. Linux

Make sure that unzip tool is installed (it is needed by the new install script now and sometimes not installed by default). Untar the installation package, stop all TIXEL services and run the install script install.sh

The following changes have been applied to the ${TIXEL_HOME} directory structure:

  • TIXstream WAN transfer service installation is now in TIXEL_HOME directory

    • /opt/tixstream → /opt/tixel/tixstream

    • You might want to remove all tixstream dirs and links: rm -r /opt/tixstream*, since they are not needed anymore.

  • Some script and directory names have been harmonized

    • Installer: install_mft_bundle.sh→ install.sh

    • Admin scripts: ctl_tixel_services.sh → ctl-services.sh

    • Backup dir: /opt → /opt/.tixel.bak

    • TIXstream transfer-status/checksum persistence: /opt/tixel/tmp → /opt/tixel/data/tixstream

  • Default directory for NGINX log files is now /var/log, so that default Linux log rotation is applied

Before starting the TIXEL services, please adapt/check configuration files according to the sections below.

4. Configuring TIXstream MFT v1.8.0 for Standard v1.2.x Broadcast Workflow

To configure TIXstream MFT 1.8.0 to operate compatibly with v1.2.x peer nodes, please make sure your configuration files and settings in the TCC are adapted as described in the following sections.

To ensure expected behaviour it is recommended to copy/paste all new parameters listed below into the corresponding configuration files.

Note
 If you have done a fresh installation of TIXstream MFT v1.8.0 you only have to walk through the Checklist sections for each service below to make sure that your setup is compatible with the standard v1.2.x broadcast workflow.

4.1. TIXstream Core Service Configuration

4.1.1. New Parameters

Please add the following new parameter in tixstream.conf:

If you have a file based license:

license-type = 380

If you have a usb dongle license:

license-type = 100

4.1.2. Checklist

For standard v1.2.x broadcast workflow please make sure that the following values are applied:

Note
 The port in external-address needs to match the value set for custom.tixstream.external.port in transfer-job-manager.properties. The directory specified in session-data-dir needs to point to the one specified in custom.tixstream.checksum-file-dir (s. below). 
external-address = 127.0.0.1:59999
datachannel-max-chunksize = 1048576
datachannel-max-streams = 10
soap-max-occurs = 1000000
internal-client-tlsverify = 1

On Linux:

session-data-dir = /opt/tixel/data/tixstream

On Windows:

Here we only use the relative path starting from %TIXEL_HOME%:

session-data-dir = data\tixstream

4.2. TIXstream Core Profiles

4.2.1. Checklist

For standard v1.2.x broadcast workflow please make sure that the following values are applied:

On Windows and Linux:

In config/profiles/FileTransferReceiver.config

Replace section:

[Receiver/Stream/NetworkReceive]
Type                 = NetworkReceive
Protocol             = TCP
StartPort            = 60004
EndPort              = 60009

with

[Receiver/Stream/XpNetworkReceiver]
Type                 = XpNetworkReceiver
StartPort            = 60004
EndPort              = 60009

In config/profiles/FileTransferSender.config

Replace section:

[Sender/Stream/NetworkSend]
Type                = NetworkSend
Protocol            = TCP
StartPort           = 60004
EndPort             = 60009

with

[Sender/Stream/XpNetworkSender]
Type                = XpNetworkSender
StartPort           = 60004
EndPort             = 60009

On Windows:

In config/profiles/FileTransferReceiver.config

NumBuffers = 2048

In config/profiles/FileTransferSender.config

NumBuffers = 128

On Linux:

In config/profiles/FileTransferReceiver.config

NumBuffers = 256

In config/profiles/FileTransferSender.config

NumBuffers = 128

4.3. Transfer Job Manager (transfer-job-manager.properties)

Remove the following parameters, since they are only relevant for SDK-/test-setups:

custom.dev.use-fake-transfer=false
custom.transfer-job-manager.dev.enable-job-scheduler=true

4.3.1. New Parameters

Add the following new parameters with their default values:

custom.tixstream.checksum-file-dir=${TIXEL_HOME}/data/tixstream
custom.transfer-job-manager.auto-update-destinations-interval-seconds=300
custom.transfer-job-manager.auto-verify-jobs-enabled=true
custom.transfer-job-manager.checksum-file-dir=${TIXEL_HOME}/data/transfer-job-manager
custom.transfer-job-manager.create-job-folder=true
custom.transfer-job-manager.default-checksum-policy=PREFERRED
custom.transfer-job-manager.default-encryption-policy=AVAILABLE
custom.transfer-job-manager.default-pixspan-policy=NOT_AVAILABLE
custom.transfer-job-manager.default-receiving-chunk-size=1048576
custom.transfer-job-manager.default-receiving-data-rate=100
custom.transfer-job-manager.default-receiving-streams=10
custom.transfer-job-manager.default-sending-chunk-size=1048576
custom.transfer-job-manager.default-sending-data-rate=100
custom.transfer-job-manager.default-sending-streams=10
custom.transfer-job-manager.default-transfer-strategies=XP_RWTP
custom.transfer-job-manager.max-receiving-chunk-size=100000000
custom.transfer-job-manager.max-receiving-data-rate=10000
custom.transfer-job-manager.max-receiving-streams=50
custom.transfer-job-manager.max-retries=10
custom.transfer-job-manager.max-sending-chunk-size=100000000
custom.transfer-job-manager.max-sending-data-rate=10000
custom.transfer-job-manager.max-sending-streams=50
custom.transfer-job-manager.overwrite-allowed=false
custom.transfer-job-manager.watchfolder-enabled=false

4.3.2. Checklist

For standard v1.2.x broadcast workflow please make sure that the following values are applied:

Please adjust default sending/receiving datarate parameters below according to your license or network connection. Note that these values (e. g. 1000 or 5000 [Mbit/s]) are on per-transfer basis.

custom.tixstream.external.address=127.0.0.1
custom.tixstream.external.port=59999
custom.tixstream.internal.port=60003
custom.tixstream.internal.address=${custom.transfer-job-manager.hostname}
custom.tixstream.checksum-file-dir=${TIXEL_HOME}/data/tixstream
custom.transfer-job-manager.auto-verify-jobs-enabled=false
custom.transfer-job-manager.default-sending-data-rate=1000
custom.transfer-job-manager.default-receiving-data-rate=1000
custom.transfer-job-manager.max-parallel-incoming-jobs=4
custom.transfer-job-manager.default-transfer-strategies=XP

4.4. TIXEL Control Center (tixel-control-center.properties)

4.4.1. New Parameters

Add the following new parameters with their default values:

custom.tcc.auto-logout-timer-seconds=0
custom.tcc.default-theme=tixel-default
custom.tcc.folder-copy-enabled=true
custom.tcc.max-file-items-per-directory=50000
custom.tcc.pixspan-enabled=true
custom.tcc.rwtp-enabled=true
custom.tcc.watchfolder-configuration-enabled=false

4.4.2. Checklist

For standard v1.2.x broadcast workflow please make sure that the following values are applied:

custom.tcc.folder-copy-enabled=false
custom.tcc.metadata-editor-directory=${TIXEL_HOME}/plugins/irt-bmf-metadata-editor
custom.tcc.metadata-editor-enabled=true
custom.tcc.pixspan-enabled=false
custom.tcc.rwtp-enabled=false

You might have to manually install the latest version of IRT BMF2 metadata plugin to the directory specified above.

Therefore create a new plugins subfolder, unzip the metadata editor archive into that folder and apply appropriate access rights.

On Linux like this:

mkdir /opt/tixel/plugins/irt-bmf-metadata-editor
unzip irt-bmf-metadata-editor-v2.1.3-0c8612d.zip -d /opt/tixel/plugins/irt-bmf-metadata-editor
chown -R tixstream:tixstream /opt/tixel/plugins/irt-bmf-metadata-editor

On Windows you can use the explorer to unarchive the zip file, copy the content to the folder and set the access rights accordingly.

4.5. Access Manager (access-manager.properties)

No changes in TIXstream MFT 1.8.0

4.6. NGINX

The NGINX configuration has been improved continuously, also during 1.0.x - 1.2.x releases. However, changes are not applied automatically during the standard update process. Most of the changes did not affect actual functionality. So you might not have applied new settings or changes manually in the past.

To benefit from the new configuration parameters and to be up to date with the current defaults, please copy the new config files to the corresponding locations:

Caution
 If you added sections or parameters in the NGINX configuration file according to individual requirements, please make a backup copy the current file and apply changes to the new configuration file after copying it to the target folder.

On Linux:

cp /opt/tixel/nginx/nginx_https.conf /etc/nginx/nginx.conf
systemctl restart nginx

On Windows:

Locate the file nginx/conf/nginx_https.conf in the directory tree of the unzipped installer package and copy it to %TIXEL_HOME%/nginx/conf/nginx.conf.

Then restart the TIXEL nginx server for TIXstream MFT service via the Windows Services console.

4.7. Settings in TCC’s Web UI

Once you have adapted and checked all configuration files, you can start the TIXEL services and log in to the TCC as admin user.

4.7.1. Nodes

In the Nodes tab, there are several new configuration parameters. After an update they have the appropriate default values. If you add new nodes, TCC already fills in some default values.

Checklist

For standard v1.2.x broadcast workflow please make sure that the following values are applied:

  • Data Channel encryption: enforced

  • Data Integrity Check: not available

Sender and receiver datachannel parameter

  • max Chunk Size: 1048576 (or keep the ´0´ - to use the appropriate defaults you have already set in transfer-job-manager.properties)

  • max Streams: 10 (or keep the ´0´ - to use the appropriate defaults you have already set in transfer-job-manager.properties)

  • max Data Rate: 1000 (or 5000) - according to your license

4.7.2. Shares

A new parameter I/O Type has been introduced. During the update procedure this field has been provided with the appropriate default sequential. The other I/O types Windows and sync can be used if you need to optimize file system access. Please contact TIXEL for further details.