Advanced Service and Component Configuration

This section describes the advanced technical configuration of the various services and components that make up the Anjana Data Platform. It is aimed at administrators and technical teams responsible for maintaining and operating the environment, once the platform has been installed via Ansible or Kubernetes.

The purpose of this guide is to provide the necessary guidelines for adjusting the behavior of Anjana microservices and modules, covering aspects such as:

• Activation of profiles and specific configurations (spring.profiles.active).

• Management of secrets and credentials in Vaults and cloud Secret Managers (AWS, Azure, GCP).

• Configuration of logs, performance, and load balancing.

• Integration with external systems (authentication, notifications, licenses).

• Administration of TOT plugins and Horus parameters.

Use of profiles

Any of the Anjana services can be started using different profiles. This practice is especially useful when a specific configuration needs to be run at certain times, such as changes in log profiling or adjustments to variables that impact performance. The following section will explain how to enable different profiles across various technologies; therefore, we will now focus on how to use those profiles.

By default, Anjana Data is configured to run the 'default' profile, which includes the default values provided by Anjana. These values are generally sufficient, unless specific connection configurations are required. This configuration can be seen in the service descriptor of any of the artifacts, where the argument --spring.profiles.active=default is included.

It is possible to modify this profile or combine it with others, using a comma to separate them, as in --spring.profiles.active=default,profile1,profile2. Additionally, complementary profiles can be run together. It is important to note that the order of the configured profiles influences the result: the last profile specified "wins". If properties have different values in each profile, the value from the last profile that configures them will be applied.

Configuration distribution

Anjana can be configured in different ways, and each service can even be configured differently due to its microservices architecture, although for convenience it is recommended to use the same mechanism for all of them.

It can be configured via:

• Horus as a centralized configuration repository that can read configurations from different sources such as a Git repository, database (app_configuration table in the portuno database schema), HashiCorp’s Vault, or the file system where Horus is hosted.

• Another configuration option is to use a Vault key repository (HashiCorp, AWS Secret Manager, GCP Secret Manager, or Azure Key Vault) in each service.

• The last configuration option is with a YAML accessible directly from the service to be configured; this method enables the use of environment variables on the same machine.

The methods described above can be used in combination, for example using Vault for credential configuration, the app_configuration table for Anjana configuration, and a yaml to configure the log level.

📝Note:

• If Horus is used, Vault must be used through Horus; Vault cannot be used directly if Horus is in use.

• Hermes cannot use Vault directly

⚠️Recommendation:

Given that most of the technical configuration involves connections and credentials, it is recommended, for security reasons, to prioritize the use of Vault repositories.

Direct YAML

For a service to use a YAML directly, the service descriptor must be configured to include the following argument:

--spring.config.additional-location=<path a directorio de fichero de propiedades>

This argument accepts specific files or directories. If a specific file is chosen, it will override all default values, leaving without a value any configuration not included in that file; therefore, it is recommended to use a directory, ending the chosen path with /.

The name of the file or files must follow the format: application-<profile>.yaml. The use of profiles is explained in the previous section.

Horus as a centralized repository

Horus is the only one that cannot use another Horus to provide configuration, so for this service one of the other two options must be chosen, or both.

Horus accepts several types of repositories to serve as already mentioned, but it also accepts configuring several at once. To do this, an order must be set using the property spring.cloud.config.server.<repository>.order and assigning a numeric order to each one.

Each service that will use Horus must have the following argument configured in its service descriptor: --spring.config.import=optional:configserver:http://<Horus host>:9999/config

The following describes how to configure each repository.

Filesystem Repository

It is only necessary to set the folder where the Anjana microservice configuration files are located; this is the native Horus profile.

spring:
  profiles:
    active: native
  cloud:
    config:
      server:
        native:
         search-locations:
         - file:/opt/data/config
         - file:/opt/data/config/{application}
         - file:/opt/data/config/{application}/{profile}

Git Repository

The branch to use can be specified in default-label; the remaining properties are generic Git repository connection settings.

spring:
  cloud:
    config:
      server:
        git:
          uri: <usuario>@<servidor>/<repositorio>.git
          default-label: <rama>
          skipSslValidation: true
          timeout: 10
          clone-on-start: true
          force-pull: true
          searchPaths: '{application}'
          ignoreLocalSshSettings: true
          privateKey: |
            -----BEGIN RSA PRIVATE KEY-----
                       .......
            -----END RSA PRIVATE KEY-----

Vault Repository

The use of Vault repositories is supported through HashiCorp’s Vault

spring:
  profiles:
    active: vault
  cloud:
    config:
      server:
         vault:
            token: ******
            kv-version: 2
            host: <host de vault>
            port: 8200
            authentication: TOKEN

For proper configuration with Vault technology, it is recommended to refer to the installation section of the “Anjana Data - Config Vault” document or the corresponding official documentation.

Database

Anjana has native communication with the app_configuration table in the Portuno database schema. This table is composed of the following columns:

• Key for the property to configure

• Value for the value to configure

• Label to use as a description of the property

• Application to define which service the property is configured for (NULL value identifies the property for all services)

• Profile to define the profile associated with the property (see what a profile is)

Encrypted Vault

Direct Vault configuration is only available for Tot Plugins, as they may not be on the Anjana core network and may not be able to access Vault through Horus. By default, this option is disabled; to enable any of the Cloud Vaults, the service descriptor must be edited by adding the property that enables the Vault, as defined below. To learn how to configure this property, you can follow the format explained in the previous section “Use of profiles”.

AWS Secrets Manager

Allows storing sensitive configuration parameters of the microservice in the AWS Secret Manager service and retrieving them via API call.

Add the following configuration to the microservice configuration yaml:

aws:
  secretsmanager:
    enabled: true
    region: eu-central-1

Within the AWS Secret Manager, secrets are stored with the prefix /secret, accompanied by the microservice name and the Spring profile separated by _ (default AWS Secret Manager configuration).

For example, to configure Kerno with the default profile, the secret will be /secret/kerno_default and its value will be the properties required for it to work.

To properly install and configure AWS Secret Manager, knowledge of AWS services is required; follow the official AWS documentation at https://docs.aws.amazon.com/es_es/secretsmanager/latest/userguide/intro.html .

For use from Anjana, there are two options:

• Create an IAM user with the policy applied and create environment variables with the credentials:

  • AWS_ACCESS_KEY_ID

  • AWS_SECRET_ACCESS_KEY

• Or if Anjana is installed on an AWS EC2 instance, assign it a role with the policy

Azure Key Vault

Allows storing sensitive configuration properties of the microservice in Azure Key Vault and retrieving them via API call. For more information, consult the official documentation: Azure Key Vault

Points to keep in mind:

• Must be configured in each microservice that needs to connect to the Vault, including plugins.

• When configuring secrets, note that Azure does not allow “/” in secret names.

• The following properties must be configured in the desired microservices with the connection to the Azure Vault:

  • The spring.cloud.config.enabled configuration must be disabled.

  • The secret-keys property indicates which secrets from the vault are to be used (if there are several, a comma-separated list is used); if the property is not specified, all secrets from the Vault will be retrieved.

  • The secret-key must match the name of the secret created in the Vault and will be used in properties with the same name, so that it references the Vault secret, for example:

    • passwordDatabase: ${secretKeyName}

• After configuring Azure following its manuals, the configuration of the microservice that needs to connect to the vault must be adjusted with the credentials obtained. These properties are added at the beginning of the yml configuration file being used:

spring:
  cloud:
    config:
      enabled: false
    azure:
      keyvault:
        secret:
          property-source-enabled: true
          property-sources:
          - credential:
              client-id: <client ID>
              client-secret: <client key>
              endpoint: <Azure-Key-Vault-endpoint>
              case-sensitive: true
              secret-keys: <secret name1>, <secret name2>
              profile:
                tenant-id: <tenant ID>

GCP Secret Management

Allows storing sensitive configuration in GCP Secret Management and retrieving it via API call. For more information, consult the official documentation: GCP Secret Management.

• Must be configured in each microservice that needs to connect to the Vault.

• A project must be created in GCP Secret Management for each microservice

• The variable values will be read with the following format ${sm://<variable>} where sm:// is the prefix, which can be omitted with the property spring.cloud.gcp.secretmanager.secret-name-prefix=sm://

Once the configuration has been done according to GCP manuals, the following code is added to the configuration file of the desired microservice.

spring.cloud.gcp:
 core:
   enabled: true
 secretmanager:
   enabled: true
   project-id: <Código del proyecto gcp secret management>
   credentials:
     location: file:*******.json # Fichero que contiene las credenciales gcp

Also, it is necessary to change the field to be replaced by a variable using the format ${sm://<nombre_secret_gcp}. An example is as follows:

spring:
  datasource:
  #If schema changes, change hibernate.default_schema few linew after
  url: jdbc:postgresql://rdbservice:5432/anjana?currentSchema=zeus
  username:
  password: ${sm://jdbc-password}
...

Other ways to secure credentials

Environment variables

Environment variables can be used to hide sensitive information in configuration yamls, for example.

Create the folder with the service name /etc/systemd/system/xxxxx.service.d and inside it a file of type env.conf (root with 600 permissions); this way only the microservice has access to that env.conf file.

[Service]
Environment=<KEY>=<VALUE>
Environment=<KEY2>=<VALUE2>

In the microservice configuration yaml, the variable can be referenced using curly braces with a preceding dollar sign ${KEY}

spring:
  datasource:
    username: anjana
    password: ${BBDD_PASSWORD}
...

Generic utility and performance configurations

All Anjana microservices have a default HTTP header size of 2KB; this value can be modified by changing this property in the properties file.

server:
  max-http-header-size: 20000

It is possible to enable Hibernate statistics to display the information it provides with the following configuration.

It is important to note that generating statistics can affect performance, so it should only be active temporarily.

jpa:
  properties:
    hibernate:
      generate_statistics: true

For Hermes, Kerno, Minerva, Portuno, and Zeus, a list of URLs that Swagger can use to make REST API calls can be configured in their yml. This configuration is optional; if Swagger is not needed, it can be omitted. These URLs must point to the Viator proxy for Swagger to work correctly; if not included, Swagger generates a valid URL on its own.

swagger:
  server:
    url:
    - http://viatorserver/gateway
    - http://<anjana_host>/gateway
    - http://hostalias/gateway

Garbage collector: All services using jdk17 must have the G1 garbage collector to manage memory more efficiently:

-XX:+UseG1GC -XX:+UseStringDeduplication

Log configuration

The microservices are configured by default to output to standard console so that logs can be managed by the system's log utility; typically in virtual machine environments, syslog, rsyslog, and journalctl will be used to consume them or configure their processing.

All of them can be configured via each one's yaml file following standard Spring Boot practices.

Example of common uses:

logging:
  pattern:
  console: "%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} [HERMES] %clr(${LOG_LEVEL_PATTERN:%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:%wEx}"
  level:
    root: INFO
    com.anjana: DEBUG

Load balancing

To configure load balancing, it is done in Horus; the Anjana microservices that register with Horus must be modified (Kerno, Minerva, Hermes, Viator, Portuno, Tot, Drittesta, and Zeus), changing the value of eureka.client.serviceUrl.defaultZone. In the Horus service descriptor, the value of the HORUS_REPLICAS variable must be changed, as described in the previous section.

Example:

eureka:
  client:
    serviceUrl:
      defaultZone: http://horusserver1:9999/eureka,http://horusserver2:9999/eureka

The same can also be done by configuring it in the service descriptor, adding an environment variable HORUS_REPLICAS with the comma-separated list.

The Java startup command for each Anjana microservice must also be modified by adding in the spring.uri parameter all the URLs of the servers where Horus is listening. It is not possible to add automatic purging of failed configuration servers in this version. This is done by adding the following property to the command:

--spring.config.import=optional:configserver:http://horuserver:9999/config,optional:configserver:http://horuserver2:9999/config

Authentication

Everything is configured in Zeus. In the example Yaml available in Nexus, the configuration can be seen in detail, allowing you to copy, edit, and paste what is needed, including field descriptions.

Additionally, authentication integrations are detailed in the specific documents at Integrations/Authentication:

• SSO OKTA/AUTH0

• SSO AWS

• SSO GCP

• SSO Azure

• LDAP/WAD Login

Email notifications

Anjana Data Platform allows sending automatic notifications by email (for example, validation alerts, status changes, adherences, etc.).
To enable them, it is necessary to configure the SMTP connection in the Hermes service, which is the component responsible for managing the sending of notifications.

The configuration is done through the Hermes parameters file (YAML). In Nexus there is an example YAML with comments that can be used as a base to copy and adapt to each organization.

Required information (what you should request from your organization)

Before configuring Hermes, request the following information from the team responsible for corporate email or the SMTP provider:

1. SMTP server (host)
   Address of the corporate SMTP server or provider (for example, smtp.company.com).

2. Port (port)
   Enabled SMTP sending port. The most common are:

   • 587 (TLS STARTTLS)

   • 465 (SMTPS / implicit TLS)

   • 25 (unencrypted or internal relay, not recommended except in controlled environments)

3. Sender account (from)
   Email address from which notifications will be sent
   (for example, anjanadata@company.com).
   This account must be authorized to send emails from the domain.

4. Authentication credentials (username / password)
   Username and password for the mailbox or SMTP technical account.

5. Security requirements

   • Whether the server requires SMTP authentication (auth = true).

   • Whether it requires TLS/STARTTLS (starttls.enable = true).

   • Whether there are IP, proxy, or firewall restrictions that need to be opened for Hermes.

Prior verification of the SMTP service from the command line

Before configuring Hermes or when any doubt arises during incident resolution, it is recommended to independently verify that the SMTP server and credentials work correctly. This allows isolating connectivity or authentication issues from potential functional errors in the platform.

Example of SMTP test from terminal (SMTPS – port 465)

swaks \
  --server smtp.empresa.com \
  --port 465 \
  --tls-on-connect \
  --auth LOGIN --auth-user 'usuario'--auth-password 'password' \
  --from 'anjanadata_platform@empresa.com' --to 'destinatario@empresa.com' \
  --header "Subject: Test SMTPS 465" \
  --body "Prueba de envío SMTP desde línea de comandos"

If the email is received correctly, it confirms that:

• the SMTP server is accessible from the environment,

• the credentials are valid,

• and there are no network or firewall blocks.

SMTP configuration in Hermes according to the port used

The Hermes configuration varies depending on the SMTP port and the type of encryption supported by the server. It is important not to combine SSL and STARTTLS simultaneously, as they are mutually exclusive mechanisms.

Port 587 – SMTP with STARTTLS (recommended in most environments)

Inside the Hermes configuration YAML, enable the mail block and fill in the values provided by your organization.

######################
### SMTP connection properties
######################
  mail:
    # the host of the SMTP server
    host: smtp.example.com
    # the port to establish a connection with the SMTP server
    port: 587
    # the email address of the account you want to use as the sender of the emails
    from: anjanadata@anjanadata.com
    # the credentials for the SMTP server
    username: <userName>
    password: <password>
    properties:
      mail:
        smtp:
        # to enable SMTP authentication. When set to true, it indicates that the SMTP server requires authentication before sending email
          auth: true
        # to enable the use of TLS when connecting to the SMTP server. When set to true, it ensures that the connection is secured using TLS encryption
          starttls:
            enable: true
            required: true
          ssl:
            enable: false

Typical use: modern corporate servers, authenticated relays, cloud providers.

Port 465 – SMTPS (implicit TLS)

######################
### SMTP connection properties
######################
  mail:
    # the host of the SMTP server
    host: smtp.example.com
    # the port to establish a connection with the SMTP server
    port: 465
    # the email address of the account you want to use as the sender of the emails
    from: anjanadata@anjanadata.com
    # the credentials for the SMTP server
    username: <userName>
    password: <password>
    properties:
      mail:
        smtp:
        # to enable SMTP authentication. When set to true, it indicates that the SMTP server requires authentication before sending email
          auth: true
          ssl:
            enable: true
        # to enable the use of TLS when connecting to the SMTP server. When set to true, it ensures that the connection is secured using TLS encryption
          starttls:
            enable: true
            required: true

Typical use: servers that require an encrypted connection from the start.

Port 25 – SMTP without encryption (not recommended)

Should only be used in internal and controlled environments.

######################
### SMTP connection properties
######################
  mail:
    # the host of the SMTP server
    host: smtp.interno.local
    # the port to establish a connection with the SMTP server
    port: 25
    # the email address of the account you want to use as the sender of the emails
    from: anjanadata@anjanadata.com
    properties:
      mail:
        smtp:
          auth: false
        # to enable the use of TLS when connecting to the SMTP server. When set to true, it ensures that the connection is secured using TLS encryption
          starttls:
            enable: false