Torrus XML Configuration Guide

Common rules

The Torrus configuration consists of several XML files. Once the XML configuration is changed, it must be compiled into the database by executing torrus compilexml. In addition, when Renderer, Collector and Monitor processes notice the configuration changes, they refresh their information automatically.

The top-level XML element is always <configuration>, with sub-elements defining various sections, like datasources or views:

  <configuration>

    <!-- Optional inclusion of other XML files -->
    <include filename="myconfig.xml"/>

    <datasources>
      <!-- Data sources tree definition -->
      ...
    </datasources>
    <views>
      <!-- View definitions -->
      ...
    </views>
    <token-sets>
      <!-- Token sets definitions ->
      ...
    </token-sets>
    <monitors>
      <!-- Monitor definitions ->
      ...
    </monitors>
  </configuration>

Multiple XML files are interpreted in additive matter, i.e. <datasources> section from one file is concatenated with the corresponding sections of previous files. If the same parameter is defined for the same subtree in several input files, the last processed value gets into the configuration.

Additional XML files may be added to the compilation list by means of <include> statement. They will be processed recursively before the content of the XML file they are referenced from. The argument filename determines the name of the file in standard XML files directory. It is safe to include the same file several times, Torrus compiler guarantees that files are only processed once.

Some kinds of sections, like <datasources>, allow to define the same elements two or more times. In this case, the previous parameter values are overridden by the new values.

Each component of the configuration is defined by the set of parameters. They are specified in a common manner, differentiating in parameter names only:

  <view name="default-rrd-html">
    <param name="view-type"     value="html" />
    <param name="expires"       value="300" />
    <param name="html-template" value="default-rrd.html" />
  </view>

The parameter value can be specified either by value XML attribute, or by the text contents of the <param> element.

Some parameter values require other parameters to be defined, like in the above example: a view of type html cannot exist without a template.

After all XML files are compiled, the parameters are checked for validity by the compiler.

Character sets

By default, all XML input is treated as UTF-8 (unicode). This is important because all the HTML output generated by Torrus is encoded UTF-8.

However, you may use Latin1 (ISO-8859-1) encoding in your XML files. In order to ensure correct displaying of non-ASCII characters, the encoding must be specified in your XML files:

  <?xml version="1.0" encoding="ISO-8859-1"?>

You need this only in those files containing non-ASCII characters in Latin1 encoding.

In addition, version 1.54_3 or higher of XML::LibXML is required, and Torrus version 0.0.16 or above.

Macros

In the top level of the configuration tree, a number of macros may be defined. Currently they are used in snmp-object parameter only. Macros are specified with <def> elements, being the direct children of <definitions> element:

  <configuration>
    <definitions>
      <!-- IF-MIB:ifTable  -->
      <def name="ifDescr"           value="1.3.6.1.2.1.2.2.1.2" />
      <def name="ifPhysAddress"     value="1.3.6.1.2.1.2.2.1.6" />
      <def name="ifInOctets"        value="1.3.6.1.2.1.2.2.1.10" />
      <def name="ifInUcastPkts"     value="1.3.6.1.2.1.2.2.1.11" />
      <def name="ifInErrors"        value="1.3.6.1.2.1.2.2.1.14" />
      <def name="ifOutOctets"       value="1.3.6.1.2.1.2.2.1.16" />
      <def name="ifOutUcastPkts"    value="1.3.6.1.2.1.2.2.1.17" />
      <def name="ifOutErrors"       value="1.3.6.1.2.1.2.2.1.20" />

      <!-- Default Interface index lookup -->
      <def name="IFIDX"         value="M($ifDescr, %interface-name%)" />
    </definitions>
    ...

These definitions are global across all XML configuration files, and are referenced with dollar sign and the definition name, e.g.:

    <leaf name="ifInErrors">
      <param name="snmp-object" value="$ifInErrors.$IFIDX" />
      ...

Parameter properties

Some parameters require special handling during the compilation or at the runtime. The parameter properties define such behaviour in the XML configuration.

Currently two properties are recognized: "remspace" and "expand".

  <configuration>
    <param-properties>
      <!-- Parameters where space is removed from values -->
      <prop param="action"             prop="remspace" value="1"/>
      <prop param="display-rpn-expr"   prop="remspace" value="1"/>
      <prop param="ds-names"           prop="remspace" value="1"/>
      ...
    </param-properties>

Datasource definitions

Datasources are organized into a tree hierarchy. All parameters are inherited by the subtrees and leafs from their parents, and can be overridden on lower levels.

The datasources tree consists of two key types of components: subtree and leaf. A subtree can have child subtrees or leaves. A leaf can never have children. A subtree represents logical aggregation, while the leaf always represends the actual datasource.

In XML configuration, a child subtree or leaf belongs to the parent element, like the following:

  <datasources>
    <!-- This is the first child of the tree root -->
    <subtree name="Netflow">
      <param name="ds-type" value="rrd-file" />
      <param name="comment"
             value="Netfow data collected by FlowScan with CarrierIn.pm" />
      <!-- All Flowscan-generated files reside here -->
      <param name="data-dir" value="/var/local/flowscan/graphs" />
      <subtree name="Exporters">
        <param name="comment" value="Netflow exporting devices" />
        ...
        <!-- all exporters defined here -->
      </subtree>
       ...
      <subtree name="Total">
        <param name="data-file" value="total.rrd" />
        <leaf name="bps">
          <param name="comment" value="Bits per second" />
          <param name="leaf-type" value="rrd-def" />
          <param name="rrd-ds" value="bits" />
          <param name="rrd-cf" value="AVERAGE" />
        </leaf>
        <leaf name="pps">
          <param name="comment" value="Packets per second" />
          <param name="leaf-type" value="rrd-def" />
          <param name="rrd-ds" value="pkts" />
          <param name="rrd-cf" value="AVERAGE" />
        </leaf>
        <leaf name="packlen">
          <param name="comment" value="Average packet length in bytes" />
          <param name="leaf-type" value="rrd-cdef" />
          <param name="rpn-expr" value="{bps},8,/,{pps},/" />
        </leaf>
      </subtree>
    </subtree>
  </datasources>

Each subtree or a leaf is identified by a path, the symbolic notation similar to filesystem paths. In any path notation, subtree names always end with slash (/), and this trailing slash is the part of the name. In this case, any subtree is identified by a path ending with slash, while leaf paths always end with a word symbol. The top-level subtree is identified by a single slash.

The other components of the datasouce definition are templates and aliases.

Templates

A template is used when it's needed to define multiple different pieces of the configuration in the same way. For instance, the definition for input/output octets and bits can be specified once in a template, and then applied where appropriate.

The piece of XML configuration inside the <template> element is memorized under the template name, and reproduced in every occurrence of <apply-template> with the corresponding template name. The template definition must be the direct child element of <configuration> XML element:

  <datasources>
    <!-- Default views must be defined -->
    <param name="default-subtree-view" value="default-dir-html" />
    <param name="default-leaf-view" value="default-rrd-html" />

    <!-- Many of our RRDs have the field "bits" - let's define it here -->
    <template name="bps">
      <leaf name="bps">
        <param name="comment" value="Bits per second" />
        <param name="leaf-type" value="rrd-def" />
        <param name="rrd-ds" value="bits" />
        <param name="rrd-cf" value="AVERAGE" />
      </leaf>
    </template>
    ...
      <subtree name="Total">
        <param name="data-file" value="total.rrd" />
        <apply-template name="bps" />
        <leaf name="pps">
          <param name="comment" value="Packets per second" />
          <param name="leaf-type" value="rrd-def" />
          <param name="rrd-ds" value="pkts" />
          <param name="rrd-cf" value="AVERAGE" />
        </leaf>
        <leaf name="packlen">
          <param name="comment" value="Average packet length in bytes" />
          <param name="leaf-type" value="rrd-cdef" />
          <param name="rpn-expr" value="bps,8,/,pps,/" />
        </leaf>
      </subtree>

Aliases

Alias is the alternative symbolic name for a subtree or a leaf. It can be even a name from a different subtree hierarchy. If that alternative hierarchy does not exist, the corresponding subtrees are created:

  <subtree name="62.3.44.55">
    <alias>/Netflow/ExportersByName/rtrTelehouse1/</alias>
    ...

Compile-time variables

Compile-time variables are those defined somewhere in datasource hierarchy, and valid within a given subtree and its children. It is possible to define pieces of XML configuration which are or are not compiled, depending on the value of corresponding variable.

Variables are set by setvar XML element, with mandatory attributes name and value.

Variable values are used in iftrue and iffalse XML elements. Mandatory parameter var specifies the variable name. The child XML elements are compiled if the variable value is true or false, correspondingly. A true value is true or a nonzero number. Undefined variable is identified as false.

Example:

  <template name="cisco-cbqos-classmap-meters">
  ...
    <iftrue var="CiscoIOS_cbQoS::CMNoBufDrop">
      <leaf name="Dropped_No_Buffer">
      ...
      </leaf>
    </iftrue>
  </template>

  <subtree name="QoS_Stats">
    <setvar name="CiscoIOS_cbQoS::CMNoBufDrop" value="true"/>
    ....
  </subtree>

Parameter value substitution

For any given leaf, some parameters may reference the other parameter values, by embracing the parameter name with percent signs:

   <param name="data-file" value="%snmp-host%_hostaverages.rrd" />

The parameter substitution is performed at runtime. The substitution formula may be defined at a higher subtree level, and the substitution itself will occur at leaf level.

The parameter substitution is performed only to those paraneters which are defined with the property "expand".

Common parameters

RRD-Multigraph leaves

The leaves with ds-type=rrd-multigraph are dedicated for displaying of several datasource values in one graph. Such leaves cannot be referenced for a numerical value, hence cannot be monitored.

Example:

  <subtree name="SampleMulti">
    <leaf name="sample1">
      <param name="ds-type" value="rrd-multigraph" />
      <param name="ds-names" value="in,out" />
      <param name="foobarpath"
             value="/SNMP/Routers/213.230.38.4/FastEthernet0_0" />

      <!-- parameter name tail is formed by the DS name -->

      <param name="ds-expr-in"       value="{%foobarpath%/locIfInBitsSec}" />
      <param name="graph-legend-in"  value="Bits per second in" />
      <param name="line-style-in"   value="AREA" />
      <param name="line-color-in"   value="#00FF00" />
      <param name="line-order-in"    value="1" />

      <param name="ds-expr-out"      value="{%foobarpath%/locIfOutBitsSec}" />
      <param name="graph-legend-out" value="Bits per second out" />
      <param name="line-style-out"   value="LINE2" />
      <param name="line-color-out"   value="#0000FF" />
      <param name="line-order-out"   value="2" />

    </leaf>
  </subtree>

Parameters:

View definitions

In our context, view means any kind of object representation. The same subtree or view can be displayed in different ways and in different formats: HTML, graph image, plain text, etc.

Renderer module handles these view definitions. For any subtree or leaf, it renders the specified view, and keeps the cache of rendered files.

Each subtree or leaf must have a default view. This is controlled by two parameters that may be defined in the root subtree: default-subtree-view and default-leaf-view.

The set of views is flat, though they can inherit the parameters one from another. Each view is referenced by its name, and is defined by the set of parameters. Same way as with datasources, certain parameter values imply the neccessaty to define certain other parameters:

  <views>
    <view name="default-rrgraph">
      <param name="view-type"     value="rrgraph" />
      <param name="expires"       value="300" />
      <param name="width"         value="500" />
      <param name="height"        value="250" />
      <param name="width-hint"    value="580" />
      <param name="line-style"    value="##SingleGraph" />
      <param name="line-color"    value="##SingleGraph" />

      <!-- Daily graph, inherits parameters from the above -->
      <view name="last24h">
        <param name="start"         value="-24h" />
      </view>

      <!-- Weekly graph -->
      <view name="lastweek">
        <param name="start"         value="-7d" />
      </view>
    </view>
  </views>

Currently the view is defined by the configuration only. Probably, in the future additional parameters will be supplied dynamically.

View parameters

For every view, the mandatory parameters are:

The following values of view-type are recognized:

Styling Profiles

Styling profiles allow symbolic names to be used for line type and color.

Two hashes in the beginning and a name refer to the line style from the styling profile, e.g. ##BpsIn, ##green, ##one, ##two.

      <leaf name="InOutBytes">
        <param name="comment" value="Input and Output bits per second graphs"/>
        <param name="ds-type" value="rrd-multigraph"/>
        <param name="ds-names" value="in,out"/>
        <!-- IN -->
        <param name="ds-expr-in" value="{ifInOctets}"/>
        <param name="graph-legend-in" value="Bytes per second in"/>
        <param name="line-style-in" value="##BpsIn"/>
        <param name="line-color-in" value="##BpsIn"/>
        <param name="line-order-in" value="1"/>
        <!-- OUT -->
        <param name="ds-expr-out" value="{ifOutOctets}"/>
        <param name="graph-legend-out" value="Bytes per second out"/>
        <param name="line-style-out" value="##BpsOut"/>
        <param name="line-color-out" value="##BpsOut"/>
        <param name="line-order-out" value="2"/>
      </leaf>

When processed the example above effectivly becomes:

      <leaf name="InOutBytes">
        <param name="comment" value="Input and Output bits per second graphs"/>
        <param name="ds-type" value="rrd-multigraph"/>
        <param name="ds-names" value="in,out"/>
        <!-- IN -->
        <param name="ds-expr-in" value="{ifInOctets}"/>
        <param name="graph-legend-in" value="Bytes per second in"/>
        <param name="line-style-in" value="AREA"/>
        <param name="line-color-in" value="#00FF00"/>
        <param name="line-order-in" value="1"/>
        <!-- OUT -->
        <param name="ds-expr-out" value="{ifOutOctets}"/>
        <param name="graph-legend-out" value="Bytes per second out"/>
        <param name="line-style-out" value="LINE2"/>
        <param name="line-color-out" value="#0000FF"/>
        <param name="line-order-out" value="2"/>
      </leaf>

Schema definitions can be modified in two ways (see the Torrus Styling Profile Guide manual for available styles and override details)

Token sets definitions

Token is a symbolic identifier for each subtree or a leaf.

A tokenset is a named list of tokens. Its contents can be rendered, and its members can be added or removed at any time.

Each tokenset can have a number of parameters defined. It also inherits the parameter defined in the top <token-sets> XML element:

  <token-sets>

    <param name="default-tset-view" value="default-tset-html" />
    <param name="default-tsetlist-view" value="tset-list-html" />

    <token-set name="jumps">
      <param name="comment" value="Traffic rate jumps" />
      <param name="rrgraph-view" value="lastweek-small"/>
    </token-set>

    <token-set name="hw-failures">
      <param name="comment" value="Holt-Winters prediction failures" />
    </token-set>

  </token-sets>

Parameter default-tsetlist-view is mandatory for tokenset list. It defines the default view when displaying the list of tokensets.

The following parameters are mandatory for tokensets:

Monitor definitions

Monitor is a named set of parameters that defines the behaviour of monitor module. Each leaf can be given a number of monitors via monitor parameter.

Upon monitor module run, an action is launched if the alarm conditions of a given monitor are satisfied.

  <monitors>
    <!-- First define the actions -->

    <!-- This action will put the graphs of alarmed datasources in
         a single alarm report page -->
    <action name="graph-hw-failures">
      <param name="action-type" value="tset" />
      <param name="set-name" value="hw-failures" />
    </action>

    <action name="graph-jumps">
      <param name="action-type" value="tset" />
      <param name="set-name" value="jumps" />
    </action>

    <monitor name="hw-failures">
      <param name="monitor-type" value="failures" />
      <param name="action" value="graph-hw-failures" />
      <param name="expires" value="3600" />
    </monitor>

    <!-- alarm if 5 minutes away it was 10 times lower -->
    <monitor name="high-jumps">
      <param name="monitor-type" value="expression" />
      <param name="rpn-expr" value="{(-300)},10,*,GT,{},{(-300)},10,/,LT,OR" />
      <param name="action" value="graph-jumps" />
      <param name="expires" value="3600" />
      <param name="comment"
             value="Value jumped more than 10-fold in 5 minutes" />
    </monitor>

    <monitor name="hundred-megs">
      <param name="monitor-type" value="expression" />
      <param name="rpn-expr" value="100,1024,1024,*,*,GT" />
      <param name="action" value="graph-jumps" />
      <param name="expires" value="3600" />
      <param name="comment"
             value="Traffic is higher than 10 mbps" />
    </monitor>
  </monitors>

Event types

Should the alarm condition occur, a series of events is happening in sequentional order:

Monitor parameters

Action parameters

Author

Copyright (c) 2002-2005 Stanislav Sinyagin <ssinyagin@k-open.com>