Octeth Training Center
Octeth Training Center
PowerMTA, FluentD and Octeth Bounce Processing

PowerMTA, FluentD and Octeth Bounce Processing

Getting started

What’s NewWhat’s NewRoadmapRoadmapOcteth Training ProgramOcteth Training Program

Installation & Setup

RequirementsRequirementsArchitectureArchitectureServer SetupServer SetupOempro InstallationOempro InstallationCommunity Edition InstallCommunity Edition InstallSSL CertificatesSSL CertificatesVersion UpgradeVersion UpgradeTroubleshootingTroubleshooting

Video Tutorials

Video TutorialsVideo Tutorials

Administration

Quick WalkthroughQuick WalkthroughSecuritySecuritySub Admin AccountsSub Admin AccountsSettingsSettingsDelivery ServersDelivery ServersGlobal SegmentsGlobal SegmentsGlobal Custom FieldsGlobal Custom FieldsBounce ProcessingBounce ProcessingSpam ComplaintsSpam ComplaintsSuppression ListSuppression ListPlug-InsPlug-InsDelivery ReportDelivery ReportUser ManagementUser ManagementMaintenanceMaintenance

Audience Management

IntroductionIntroductionSubscriber ListsSubscriber ListsCustom FieldsCustom FieldsSegmentsSegmentsSubscribersSubscribersWebhooksWebhooksWebsite Event TrackingWebsite Event Tracking

Email Marketing

IntroductionIntroductionAuto RespondersAuto RespondersEmail CampaignsEmail CampaignsSender Domain Man.Sender Domain Man.Journeys (Automation)Journeys (Automation)Bounce ProcessingBounce ProcessingEmail PersonalizationEmail PersonalizationFBL ProcessingFBL ProcessingEmail Builder IntegrationsEmail Builder IntegrationsEmail DeliverabilityEmail DeliverabilitySender Domain ManagementSender Domain Management

Plug-Ins & Add-Ons

AI Plug-InAI Plug-InRSS Plug-InRSS Plug-InBounce Catcher Add-OnBounce Catcher Add-OnLink Proxy Add-OnLink Proxy Add-On

Developers

IntroductionIntroductionSingle Sign OnSingle Sign OnAPIAPIPlug-In DevelopmentPlug-In DevelopmentDatabaseDatabase

Advanced

Reverse ProxyReverse ProxyCookbookCookbook

Help

TroubleshootingTroubleshootingContact UsContact Us

PowerMTA, FluentD and Octeth Bounce Processing

Installation

Install Fluentd to process PowerMTA Logs and post them to Oempro.

  • Reference

Run ulimit -n and if the result is 1024, increase maximum number of file descriptors to 65536:

If your console shows 1024, it is insufficient. Please add the following lines to your /etc/security/limits.conf file and reboot your machine:

root soft nofile 65536
root hard nofile 65536
* soft nofile 65536
* hard nofile 65536

Reboot the server:

reboot

Install Fluentd Ubuntu package by following instructions here.

# Install packages
apt update
apt install gcc make ruby ruby-dev

# Install Fluentd Gem
gem install fluentd --no-doc

Run the following commands to verify the Fluentd installation:

# Setup the fluentd directory and initial configuration
fluentd --setup /etc/fluent

# Run and test
cd /etc/fluent/
fluentd -c /etc/fluent/fluent.conf -vv &
echo '{"json":"message"}' | fluent-cat debug.test

Use kill to stop the test fluentd process.

Boot Setup

The Fluentd gem does not come with /etc/init.d/ scripts. To ensure Fluentd starts on system boot, we need to create a systemd service file for Fluentd:

  1. Create a new service file for Fluentd:
vi /etc/systemd/system/fluentd.service
  1. Paste the following into the file:
[Unit]
Description=Fluentd data collector
After=network.target

[Service]
User=root
ExecStart=/usr/local/bin/fluentd -c /etc/fluent/oempro.conf --under-supervisor
Restart=always

[Install]
WantedBy=multi-user.target

You can learn the actual path of fluentd by executing which fluentd.

  1. Save and close the file.
  2. Reload the systemd daemon to read the new file:
systemctl daemon-reload
  1. Enable Fluentd to start on boot:
systemctl enable fluentd
  1. You can start Fluentd immediately without rebooting by running:
systemctl start fluentd
  1. To check the status of the Fluentd service, you can run:
systemctl status fluentd

This will ensure that Fluentd starts automatically whenever your system boots up.

Configuration

First, make sure that PowerMTA logs properly. Edit the /etc/pmta/config file and make sure that the following directive exists:

# Octeth Logging
<acct-file /var/log/pmta-oempro-accounting/oempro.csv>
    delete-after 40d
    move-interval 24h
    
    records d, b, t, tq, r, f, rb, rs

    record-fields d  *, !dlvEsmtpAvailable, header_From, header_Message-ID, header_Return-Path
    record-fields b  *, !dlvEsmtpAvailable, header_From, header_Message-ID, header_X-FBLId, header_Return-Path
    record-fields t  *, !dlvEsmtpAvailable, header_From, header_Message-ID, header_Return-Path
    record-fields tq *
    record-fields r  *, !srcType, !srcMta, !rcvSourceIp, !rcvDestinationIp, header_From, header_Return-Path
    record-fields f  *, header_X-virtual-MTA, header_X-FBLId, header_Return-Path
    record-fields rb *, header_From, header_X-virtual-MTA, header_X-FBLId, header_Return-Path
    record-fields rs *, header_Return-Path

    map-header-to-field f header_X-HmXmrOriginalRecipient rcpt # JMRP Recipient
</acct-file>

The /etc/fluentd/oempro.conf content should be set to:

<source>
  @type tail
  path /var/log/pmta-oempro-accounting/oempro-*.csv
  pos_file /var/log/pmta_oempro_accounting.pos
  tag pmta.log.oempro

  <parse>
    @type csv
    keys type,timeLogged,timeQueued,orig,rcpt,orcpt,dsnAction,dsnStatus,dsnDiag,dsnMta,bounceCat,srcType,srcMta,dlvType,dlvSourceIp,dlvDestinationIp,dlvSize,vmta,jobId,envId,queue,vmtaPool,header_From,header_Message-ID,header_X-FBLId,repSourceIp,feedbackType,format,userAgent,reportingMta,reportedDomain,header_Return-Path,header_X-job,header_Subject,header_X-virtual-MTA,dsnReportingMta
  </parse>
</source>

<match pmta.log.*>
  @type copy
  <store>
    @type http
    endpoint https://your-octeth-domain.com/system/bounce_webhook?type=fluentd
    open_timeout 2
    http_method post
    content_type json
    json_array true
    <format>
      @type json
    </format>
    <buffer>
      flush_interval 10s
    </buffer>
  </store>
  <store>
    @type stdout
  </store>
</match>

In order to monitor stdout of Fluentd, run this command:

journalctl -u fluentd -f

On this page

  • PowerMTA, FluentD and Octeth Bounce Processing
  • Installation
  • Boot Setup
  • Configuration
💠
image

Homepage Customer Area Community Portal Contact Us

Footer Social Icons

©Copyright Octeth, Inc. All rights reserved.